extfieldsofstudy:
Medicine
Computer Science

text:
Two robust long short-term memory frameworks for trading stocks

This paper aims to find a superior strategy for the daily trading on a portfolio of stocks for which traditional trading strategies perform poorly due to the low frequency of new information. The experimental work is divided into a set of traditional trading strategies and a set of long short-term memory networks. The networks incorporate general and specific trading patterns, where the former takes into account the universal decision factors for trading across many stocks, while the latter takes into account stock-specific decision factors. Our research shows that both long short-term memory networks, regardless of whether they are based on universal or stock-specific decision factors, significantly outperform traditional trading strategies. Interestingly, however, on average neither has the edge compared to the other, thus remaining ambivalent as to whether universality or specificality is to be preferred when it comes to designing long short-term memory networks for optimal trading.

Introduction
Stock trading is a market exchange process that involves transactions of company stocks among shareholders. These transactions commonly encompass buying and selling stocks, with some holding period between. The aim of stock trading is to obtain a profit, either dividend payouts or capital profit, where the first is expressed as actual earnings that Timotej Jagrič timotej.jagric@um.si Dušan Fister dusan.fister1@um.si Matjaž Perc matjaz.perc@um.si 1 the company is willing to pay out and the latter as the difference between the buying and selling prices. Stock trading is about making the right decision at the right time, either focused on a single stock or broadened over a portfolio of stocks, i.e., multiple stocks held simultaneously. Importantly, the latter variant diversifies (decreases) the unsystematic risk and is thus commonly employed in practice.
Trading decisions by shareholders are traditionally made by either a top-down or a bottom-up approach. A top-down approach sequentially follows the macroeconomic, industry sector and then the individual company (fundamental) analysis [11], while a bottom-up approach, also called stock picking, does that in reverse. Macroeconomic, sectoral and fundamental analyses require extensive information and understanding; trading decisions derived from such processes are typically very rare, either because of the low frequency of some information, e.g., quarterly in the best case, or because of the lengthy analysis process itself. As an alternate to the top-down or bottom-up approaches, technical analysis relies heavily on only past and current asset prices. Defined as chartists, technical analysts analyze supply and demand functions continuously to identify investor behavior and sentiment and execute immediate actions. Due to its rapidity, technical trading can be defined as a high-frequency trading (HFT) process; these processes are currently gaining in popularity among investors. For example, Zhang [62] reported that HFT in 2010 accounted for more than 70% of transactions; we can assume that this ratio is currently much higher. Interestingly, in the HFT domain, raw financial data are supplied directly from trading platforms to high-performance computers and servers that run automated and computerized decision algorithms. These decode data and synchronously and proactively intervene on the stock market. The HFT process is completely automated, regardless of whether the decision algorithms are a very simple set of rules or extremely complex machine learning algorithms. Both share a common purpose, that is, to identify or extract past and recurrent hidden patterns that might be beneficial for future intervention.
The problem and hypothesis that this article addresses is whether it is possible to earn higher-than-normal (excess) profits on an open and transparent stock market using innovative decision algorithms. Economic theory on efficient markets strongly teaches us that the answer to this question is no -Fama [16] has stated that when the markets are efficient, (1) asset prices fully reflect all available information, and (2) new information is quickly incorporated into the asset price, which means that the only way the investor can increase expected profit is to take higher risk. Since new information comes in an erratic or unpredictable way, asset prices are erratic and unpredictable as well; in other words, asset prices follow random walk movements. According to the theoretical grounds, it is therefore impossible to earn higher profits based on only past (old) information, and whoever attempts to do so cannot surpass even a randomly selected portfolio of stocks of comparable risk (stated by Malkiel [32], cite taken from [41]).
Of course, many market inefficiencies have contradicted these theoretical concepts, such as speculative bubbles, irrational pricing, herd instinct, and market changes. A more comprehensive list can be found in [29]. Many empirical studies have shown that various trading strategies and ideas have indeed achieved excess profits and thus have intrinsically negated the theorem that asset prices strictly follow random walks and are unpredictable [31]. Fernández-Rodríguez et al. [17] reported the applicability of a simple artificial neural networks (ANN) trading rule to stock index trading, where the authors obtained higher profitability than a buy-and-hold (passive) strategy through periods of bear and stable markets and lower profitability for the bull market periods only. Chiang et al. [6] proposed an adaptive trading decision support system with a denoising function, which scored 41.86% of the adjusted return, compared to 12.98% for a buy-and-hold trading strategy. Chong and Ng [8] realized (based on 60-year data for London's FT30 index) that trading with the simple momentum strategies relative strength index (RSI) and moving average convergence divergence (MACD) generated higher returns than the buy-and-hold strategy in most cases. Similarly, Wong et al. [59] showed the benefits of MACD and RSI on the Singapore Stock Exchange. They claimed that by applying simple technical indicators, one can generate significantly positive returns, and interestingly, they also claimed that larger companies have specialized trading teams that apply technical analysis only. Teixeira and De Oliveira [52] proposed a combination of technical analysis with the nearest neighbor classification algorithm, exclusively based on past daily stock closing prices and volumes. The proposed method generated significantly higher profits on 12 out of 15 stocks than the buy-andhold trading strategy. Ruta [40] showed many options for continuous 24/7 machine learning trading, and Creamer [10] proposed the machine learning algorithm Logitboost, which generated highly significant positive returns. The most recent research on the stock trading field encompasses deep reinforcement learning methods for stock trading [60] and many other machine learning, artificial intelligence and deep learning trading algorithms, which have been reviewed by cbailes on GitHub's library awesome-deeptrading. 1 Additionally, many relevant articles considering convolutional neural networks, long short-term memory, generative adversarial networks, etc., for trading, are accessible here. Furthermore, fuzzy inference systems may emerge in the near future [13,33].
The purpose of this article is to implement an online (simultaneously updating) long short-term memory (LSTM) neural network investment trading strategy (ITS) for daily portfolio trading and test whether it is possible to earn higher-than-normal profits compared to benchmark trading strategies over a longer trading period. LSTM's innovative architecture should identify (extract) hidden patterns in financial data, if existing, and sufficiently well intervene in out-of-sample backtesting. A case study on the German stock market during the years 2010 and 2020 was implemented, where the German market was chosen as the example because it is one of Europe's leading, most open, and transparent stock markets. Financial data were fetched from a popular trading platform to construct a dataset and were later split into two parts: offline LSTM training was employed for the first part; while online LSTM training was employed for the second. The quality of the proposed idea was then evaluated.
In this study, we did not rely on either fundamental, sectoral or macroeconomic analyses but instead relied on the current market and stock (financial) quotations only, which were technically expressed for a given moment by movement, volatility, trend in stock prices, and suitable technical indicators. Using the different pattern extraction methods, we struggled to capture any speculative market patterns from these movements and trends. We supposed that some of these patterns would affect multiple stocks in a similar (general) way synchronously and that others would only be relevant for specific stocks. Hence, we constructed the two following frameworks (Sirignano and Cont [46]): 1. A single, universal, general common-model that treats all the stocks in the same way, without differentiating between them. The number of LSTM models equaled 1, regardless of how many stocks were included in a portfolio (Filos [18] also called such decision systems 'universal trading agents'). 2. A stock-specific unique-model that aimed to be used for trading with a single stock only. The number of LSTM models equaled the number of stocks held in a portfolio.
Model universality, as exploited in present paper, was extensively populated by Ruiz-Cruz in [38], who proposed a universal deterministic control law model that was able to simultaneously trade multiple stocks in real-time. Investment profiles were freely managed from conservative to aggressive, and the findings on 4 Mexican stocks showed that the proposed approach can be exploited for universal multiple stock trading. Wilson [58] stated that one chaotic model can be effectively exploited to trade hundreds of stocks over a long time period without changing any system inputs; in his case, he traded with 600 stocks over 10 years using a single model. Sezer and Ozbayoglu [42] employed a universal convolutional neural network (CNN) and performed trading on the 30 Dow stocks, where CNN backtesting gave promising results for the future. In contrast, Chihab et al. [7] explicitly stated that no universal model captured everything that might happen on stock markets due to many external (either objective or subjective) factors. According to Shiller [44], a sample of these includes psychological, sociological and anthropological factors, such as greed, emotional pain, speculation, overconfidence, underreaction, the perceived irrelevance of history, magical thinking, herd instinct and others. These factors may all influence different stocks similarly or completely disparately, similar to more objectively given factors, such as interest rates, GDP growth and other commonalities [1].
Our goal was to find an autonomous, superior and robust ITS for daily trading on a selected set of stocks for which classical fundamental analysis could not be applied due to the too low frequency of data (information). We expected that this study would provide a superior ITS and effectively disclose the benefits of both LSTM frameworks. The architecture of both LSTM neural networks remained equal in both frameworks. We assumed that regardless of whether the hidden (recurrent) patterns across many stocks were present through time, the common-model would output adequate performance. Nevertheless, we expected that the unique-model would adapt to the specifics of each stock and therefore outperform the common-model. This paper was written as an extension of Fister et al. [20] and Fister and Jagrič [19] and comprises the following main novelties: -extend experiments from a single stock to complete portfolio, -widen the online LSTM principle for automated daily stock trading, -validate that LSTM ITS can outperform a passive buyand-hold ITS on a diversified portfolio, -implement two different LSTM ITS frameworks (ideas) and compare their robustness, -focus on European (German) financial markets, -compare and extract the benefits of LSTM ITSs compared to traditional ITSs.
The structure of the paper is as follows: Section 2 addresses the fundamentals of financial markets, particularly the German financial market, the dataset used, traditional investment trading strategies and the fundamentals of LSTM. Section 3 describes the two proposed robust online LSTM ITSs. Section 4 addresses the experiments and results, and Section 5 concludes the paper and outlines directions for the future.

German financial market, stock companies and investment trading strategies
This case study focused on one of the most important and prominent financial markets in Europe -the German financial market, which is located in Frankfurt (Germ. Börse Frankfurt). In Frankfurt, many various financial instruments can be traded, such as stocks, bonds, ETFs/ETPs, funds, and commodities. Every day, the Frankfurt stock exchange calculates the average performance of the stocks quoted there and publishes the so-called stock indices accordingly. The most popular stock index in Germany is the DAX30 (Germ. Deutsche Aktien Index 30, German stock index 30), which comprises 30 blue-chip (very prominent) companies and is calculated using a free-float (free-adjusted) capitalization method. Table 1 lists the DAX30 2 stock companies, their tickers and market capitalizations. 3

Dataset
In the table of German blue-chip representatives, companies from textile, automotive, aviation, civil engineering, healthcare, insurance, banking, and other industry sectors can be found. The largest market capitalization on the given date was held by SAP SE company, which is a representative  29.727 n/a n/a n/a n/a n/a Wirecard AG* WDI.DE 0.47 11.848 n/a n/a n/a n/a n/a The examined data are from 1 Jan 2010 to 12 May 2020 "MC" stands for market capitalization and is given in billion euros. Market capitalization data and beta (5-year monthly) were taken on 6th June from Yahoo Finance. Three stock companies, designated by "*", e.g., 1COV.DE, VNA.DE and WDI.DE, were not suitable for this case study since no financial data were available for full coverage from 2010 on. Bold values denote the highest values. The "min", "max" and "mean" prices are stated in euros. High JB values show that none of the close prices were practically normally distributed. The ADF test of stationarity shows that only the close prices of two companies, e.g., EOAN.DE and RWE.DE, are time stationary. We concluded that the observed non-normality, non-stationarity and high level of multicollinearity of financial data make the classical regression approach infeasible Source: Finanzen.net 2 and Yahoo Finance 3 of a multinational enterprise focused on computer software applications. By contrast, the lowest market capitalization was held by Deutsche Lufthansa AG, i.e., the German national airline company. The presented "betas" express the empirically calculated risk of each company stock value relative to the overall DAX30 stock index. The riskiest among those on the list was the automotive Volkswagen AG company, and the least risky was the personal-care company Beiersdorf AG. Stock companies, declared with "*", e.g., 1COV.DE, VNA.DE, WDI.DE, were excluded from the case study due to missing (insufficient) data, while for the remaining companies, basic underlying financial data features could be fetched from the platform Yahoo Finance, consisting of open price, close price, adjusted close, highest daily price, lowest daily price and volume.
Open price was defined as the price of the security (in the morning) at the opening of the trading day. Similarly, the close price was defined as the current security price in the evening when the market closed. The adjusted close price was defined as the close price corrected for dividend payouts, stock splits, right offerings and others. A high price was derived as the highest price on the given day during the market operating hours, while a low price was derived as the lowest price. The volume determined the number of transactions on the given day as it was summed throughout the trading day. These 6 basic financial data features were adopted to derive several technical indicators and thus (1) expanded the dataset and (2) included additional information in the dataset. The minimum, maximum and mean values of each underlying company across the complete time period observed can be found in Table 1. Additionally, the results of the Jarque-Bera (JB) test of normality [26] and augmented Dickey-Fuller (ADF) test of stationarity [12] are attached to the table to validate that none of the close prices in the observed time period followed a normal distribution and that only two of the time series were practically stationary. Additionally, the fetched prices exhibited high levels of multicollinearity. All mentioned findings severely impair the assumptions of classical regression analysis and thus raise the question of whether the classical approach is feasible. Table 2 further summarizes the list of original market, stock and date data and the derived technical indicators. Such an 'expanded' dataset (hereinafter dataset) was generated for each stock and then used during the case study.

Traditional investment trading strategies
An ITS is a decision algorithm that scrutinizes financial data and, according to the internal state, generates three fundamental trading signals, i.e., "Buy", "Hold" and "Sell". Fister et al. [20] distinguished among (1) traditional trading strategies, such as passive, RSI and MACD; (2) machine Market and stock data each comprised 6-element financial data. Date data were composed of 5-element periodic data. Technical indicators consisted of n-day returns, n-day differences, RSI and MACD indicators, inclinations over n = {5, 10, 15, 20}-day periods, changes (differences) in returns for n = {1, 2}, differences in inclination values between (1) 5-day inclination to a 5-day inclination shifted for 5-days, (2) 5-day inclination to a 10-day inclination, (3) 5-day inclination to a 15-day inclination and (4) 5-day inclination to a 20-day inclination. Finally, two coefficients were calculated, e.g., the relative quotient between the close to open prices and the relative quotient between high and low prices, both decreased by 1 Source: Fister et al. [20].
learning classifiers (MLC), such as decision trees, random forests, SVM, k-NN; and (3) ANNs, e.g., LSTM. Traditional trading strategies rely on rule-based guidelines only, which enable them to conduct instantaneous actions based on signals without any effort to build surrogate models. On the other hand, the surrogate model, which has been described by [34], based ITS performs in two subsequent stages: (1) training and (2) validation (prediction), usually by splitting the dataset into two samples, e.g., training (insample) and validation (out-of-sample). The in-sample is used for training only and the out-of-sample for unbiased prediction only. The quality of predictions is evaluated using a mechanical trading system (MTS), which is a tool that exploits the idea of ex post backtesting [21,36]. Practically, this means that the MTS and ITS are both put into history, without knowing any "future" trends, movements or information. The MTS is given an initial amount of cash that can be used to buy stocks. Then, simulation trading is performed day by day for each stock in the portfolio. The MTS reacts to the ITS trading signal accordingly: it buys, holds and sells stocks on demand. Backtesting is especially useful for testing ITSs since it incorporates a zero initial investment concept (trading is performed with virtual money only). Traditional (classical) trading strategies are a kind of rule-based ITS [61], where instead of the trading signal being derived from some intelligent decision algorithm, the decision is made according to the internal state moving between two values (maximum and minimum). When the internal state hits one of the preset limits, the relevant trading signals are triggered; when the internal state holds within these two margins, it is treated as a hold signal.
Many of the traditional rule-based trading strategies exist. In this paper, two well-known technical indicators from Table 2, the RSI and MACD, were used (more technical indicators can be viewed in [37]). For the RSI indicator, which finds extremes between 0 and 100, this means that when (1) the internal state over the preferred time series, e.g., a stock's close price, hits the upper limit (in the professional trading community set at 70), the stock is considered to be overbought, and a sell signal is triggered immediately. Conversely, when the (2) internal state hits the lower limit (usually 30), this is treated as oversold, and a buy signal is triggered. A MACD indicator is slightly more sophisticated since it derives actual signals by subtracting two exponential moving averages (EMAs), e.g., between the 12-period EMA and 26-period EMA, where the period is typically given in days. The result of the subtraction is the so-called MACD line, which is next compared to the 9-period EMA signal line. The subtraction between the two ultimately provides the MACD histogram, which is used for trading directly. Both the RSI and MACD ITSs incorporate initial blackout periods due to their momentums.
By contrast, a passive trading strategy, i.e., buy at the first trading day and hold the capitalization until the last day, is also a typical representative of traditional ITSs. Due to its simplicity, it does not incorporate any blackout periods. The three traditional trading strategies mentioned, i.e., passive, RSI, and MACD, are referred to in this article as backtesting benchmarks [53].

LSTM investment trading strategy
In this article, MLC-ITSs are not discussed. Rather, the focus of this article is on the design of the ANN trading strategy or, more precisely, the use of LSTM, i.e., a kind of recurrent neural network (RNN). RNNs were proposed by Rumelhart et al. [39] specifically to cope with time series problems. Their benefit compared to a more common multilayer perceptron network lies in the integration of a feedback loop, which connects the previous output to the current input. While the RNNs in general mechanically connect the output from the previous step to the input in the current step directly, an innovative variant of RNNs called LSTM, proposed by Hochreiter and Schmidhuber [23], do this more delicately by selectively filtrating the unnecessary information from the previous step and selectively inputting new data in the current step.
LSTMs consist of three gates, forget, input, and output gates, and a central memory cell. In effect, they are much more complex than usual RNNs. The complexity of LSTMs allows them to prevent the occurrence of exploding and vanishing gradients and simultaneously enables them to capture longer-term dependencies in data without forgetting much. Figure 1 shows the basic outline of a single LSTM cell [3,22,24].
Let us designate the input that is about to be propagated through the LSTM neural network as x t , and the trainable LSTM weights, i.e., the input, recurrent and bias weights, as W, R and b, respectively. Whether the underlying weights comprise the input, forget, output and cell candidate components as follows: then, the propagation of information through the LSTM can be denoted by the following transition equations [24,50]: where σ represents a sigmoid function, and i t , f t , o t ,c t represent input, forget, and output gate components and the cell candidate component, respectively. Vectors c t and h t represent memory cell and hidden states, respectively, and c t−1 represents the previous memory cell state from the constant error carousel. Operator stands for elementwise multiplication. Similar to LSTM propagation stipulating the forward pass of the LSTM network, LSTM training stipulates the backward pass. Training the LSTM is a very complicated and computationally intensive task; therefore, we only mention it briefly. The LSTM training (in effect RNN training) is concluded using a special variant of backpropagation, i.e., Fig. 1 Outline of the LSTM cell. LSTMs consist of forget, input and output gates and a central memory cell. The latter makes the LSTM a representative of recurrent neural networks. Source: Buduma and Locascio [5] Memory Cell

Forget gate
Input gate Output gate backpropagation through time (BPTT) [57]. Typical actions of BPTT encompass (1) presenting the input pattern to LSTM and forward-propagating it to obtain predicted outputŷ, (2) comparing the predicted outputŷ to actual output y and calculating the classification error e =ŷ − y (in this caseŷ and y are classification labels), (3) calculating the LSTM network weight derivatives, and, finally (4) exploiting the calculated derivatives to modify (adjust) the network weights to minimize the classification error e [4]. The original mathematical outline for calculating derivatives was denoted in detail by Hochreiter and Schmidhuber [23], while Sutskever [49] presented novel state-of-the-art RNN training methods.

Online LSTM investment trading strategy
As already mentioned, when dealing with surrogate modeling, the dataset is split into training and testing samples (also called in-sample and out-of-sample). Usually, the model is trained on the basis of the in-sample, and the out-ofsample is used for evaluation of the trained model only.
However, such strict splitting may be irrational when considering the real-world problem of portfolio trading. Here, new information (new instances of data) are disclosed every day, and these can be rationally exploited to update the model. Behavior on stock markets changes daily; therefore, it is essential for the user to keep the model continuously updated with the most recent information. Practically, this means that the learning process continues out-of-sample rather than mechanically stops at the end of the in-sample. Considering that only a single instance (one day) of progress is made in each step, this practically simulates instance-byinstance (day-by-day) trading. This concept only complies with a strict dataset split initially, i.e., the first split to design the initial network model, whereas later, the out-of-sample instances become available to retrain (update) the model one-by-one. According to Fister and Jagrič [19], who called such LSTM "online", the concept is entirely physically viable and substantially beneficial to use in practice.
Algorithm 1 presents the details of the online LSTM procedure [19] as was used during the experiments. First, the complete dataset X was loaded into the programming environment and split into two samples: in-sample and outof-sample. The algorithm parameters, such as the learning rate α, decay and other mandatory parameters, were initialized. After initializing, the prospective LSTM model was trained on the in-sample. The purpose of in-sample training was to capture the general behavior of financial markets. In-sample represented offline training and was run for a number of offline epochs. Due to the mass of in-sample instances, it was a time-consuming process. After the offline training, the so-called backtest vector and fixed learning rate α were initialized. The first was used to store the outof-sample trading signals for later evaluation, while the second was used for online training. As the process turned into the online LSTM training part, the new out-of-sample instances x t were extracted one at a time. This sequential day-by-day process followed real-world circumstances. Therefore, the model only progressed by a single out-ofsample instance at each time, first by making forthcoming trading signal predictions and appending the results into the backtest vector and second by retraining the LSTM model with actual y. After the online training concluded, i.e., all out-of-sample instances went through, the MTS simulation (SIMULATE MTS in Alg. 1) was performed using the generated array backtest vector. Finally, the results for the completed out-of-sample period were visualized and reported to the user.
MTS served as a backtesting routine that, according to the ITS, bought, sold or held the stocks in the portfolio. It is based on real market and financial data that were downloaded from web portal. In general, the MTS can be used in two ways: sequentially (day by day) or once only (when the trading is fully completed). Due to its ease, we used the second approach in our case, but this introduced the need to implement the backtest vector. The following assumptions were considered for the MTS: (1) the MTS does not deal with short selling, (2) the close price is obtained exactly a moment before the market closes so that we can execute orders, (3) the market is highly liquid so that the MTS can execute any orders requested, (4) the market is large enough that any buy or sell orders executed do not affect prices or expected market behavior, (5) the maximum number of stocks is always bought when the buy trading signal is triggered and all of the stocks are always sold when the sell trading signal is triggered -hence, some of the free cash assets usually remain in the portfolio after buying (only integer values of stocks can be traded), (6) once the instrument is bought or sold, no further transactions are allowed in the n-days time horizon.

Response variable
The training signal during the training stage was expressed as a percentage change in the close price in given time t compared to the previous close price t − n. Formally, the response variable equation was derived from [20] as follows: where x (close) t was the close price in given period (day) t, and x (close) t−n was the close price n-days ago. Here, n was called the time horizon and was defined prior to use; Fister et al. [20] varied n from 1 to 5. The n represented the percentage change in the close price for a given time horizon and naturally cannot be used for the classification problem. An additional transformation step was thus employed before actual use comparing percentage changes to a predefined variable threshold. Here, three scenarios were applied: (1) when the percentage change n was higher than the defined threshold, the training signal was a "Buy" signal; (2) when the percentage change n was lower than the defined (negative) −threshold, the training signal was a "Sell" signal; (3) if the percentage change n remained between the positive and negative thresholds (±threshold), the training signal was "Hold": The authors in [20] fixed the threshold at threshold = 0.05. In this article, we agreed that some further experimentation on this value may be useful/convenient, mainly due to the fact that each transaction incorporates trading or transaction costs/fees/provisions (more frequent transactions imply higher trading costs).

Proposed solution
As already stated in the introduction, the purpose of this paper was to implement two robust/superior LSTM neural network ITSs and check whether they can perform efficiently on the financial stock market. The two following frameworks for automated online LSTM ITS were presented in detail: 1. a common-model for all stocks held in the portfolio that traded according to the general (average) behavior of the portfolio, 2. a unique-model for each specific stock in the portfolio that traded according to the behavior of each stock uniquely (specifically).
Although the two mentioned frameworks were different, they shared a common ground of propagating sequence (financial) data. The propagation of sequence (daily) data was performed as follows: first, the length of the time steps T s was defined, and for a given day t, the previous T s time steps were extracted from the dataset. These were next supplied into the LSTM neural network one-by-one. The time step x t was propagated through the LSTM last. Whether the length of the time step T s was sufficiently long, LSTM established stable internal memory cell (c t=T s ) and robust weight settings. This was important, since internal memory cell was next employed to output the predicted signals. These were next compared to the actual response variable in a given time t, e.g., y t , to drive the LSTM training procedure. Finally, t became t + 1 and the process was repeated. Each instance participated multiple times in LSTM propagation. Figure 2 graphically shows the propagation of time steps through the LSTM.

Common-model for all stocks
Common modeling was the first proposed framework for automated online LSTM ITS for portfolio trading. Here, a single LSTM model was used for training and predicting for all stocks. The common-model aimed to capture the general or average behavior of stocks held in a portfolio. The benefit of this idea lies in its decreased complexity, i.e., a single-model could be especially appropriate for (minimalistic) dedicated hardware over a wide range of stocks. The negative aspects of the single-model are (1) that a considerable set of underlying financial data must be available for a complete update and (2) two or more contradictory stock behaviors may cancel each other out and thus contribute little or nothing to the model. It is thus desired that the stocks in this model are similar to each other (homogeneous). The following paragraph describes the single-model working principle.
For the in-sample, every day, k instances or k training samples, where k equals the number of stocks held in a portfolio, were used N off line times for offline LSTMtraining. The offline LSTM-training ran sequentially from the oldest data towards the newest data, always considering all the k instances of all stocks in a given time t. After the LSTM-training concluded for all k instances in a given time t, the t variable was incremented by 1 to obtain t + 1, and the procedure was repeated. During the out-of-sample, for each of the stocks, the past sequence data of length T s was supplied to the common-model. Next, the LSTM-simulation or prediction was run, which was in effect the trading signal predictionŷ for the next trading day (tomorrow). At the end of the (tomorrow's) trading day, the actual trading signal y was extracted, and the difference between the actual y and predictedŷ trading signals was supplied to the online LSTM-training procedure. Figure 3a shows the outline of a common-model.

Unique-model for each stock
A unique model for each stock was tailored to the separate specifics of each stock. Practically, this means that for each stock company, exactly one model was initialized, trained and simulated. Hence, the unique-model allowed diverse financial instruments, such as bonds, cryptocurrencies or commodities, to be held in portfolio simultaneously (although only stocks were considered). The benefit of the unique-model compared to the common-model lies in the much less time consuming updating and more tolerable execution pace for different financial instruments distributed over several computers. However, the main drawback of such design was a much higher complexity and thus the need for better hardware. Figure 3b shows the outline of the unique-model for each stock.

Experiments and results
To test the underlying hypothesis, whether it is possible to earn higher-than-normal profits using the online LSTM neural network, comprehensive experimental work was ... Fig. 2 The model shown is a kind of many-to-one sequence model. The past market and financial data are supplied and propagated via LSTM cell. After the predefined time steps T s , the LSTM establishes stable memory cell state and can output the predicted trading signalŷ t , e.g., during the learning procedure after comparing to actual output y t For the sake of experimental tests, 10 separate simulations of both the common-and unique-models were completed first. The use of 10 separate simulations was inspired by a common validation tool, the k-fold validation, where k is usually set to k = 10. We do not provide cross-validation in this paper but monitor the bias-variance error using a number of experiments. LSTMs perform differently with each run; thus, it is important to extract the best, worst and mean performances (performance is represented by the final portfolio value). Ten different models were built for the purpose of testing the common-model, and 270 models were built for the unique-model experiments, i.e., each of the 27 stocks in the portfolio had 10 different unique-models. Upon completing 10 simulations, the following experimental tests were conducted: 1. proof-of-concept of the common-and unique-models compared to traditional ITSs, 2. robustness tests between common-and unique-models, and 3. detailed evaluation tests of common-and unique-model performance compared to that of traditional ITSs.
Experiments were performed in the Python scripting language with the following libraries: financial data were fetched using Python's pandas-datareader 4 via the Yahoo Finance web portal; dataset transformation was performed using the pandas, 5 numpy 6 and talib 7 libraries. Here, the 6-element basic market and stock data (open, close, high, low, adjusted close prices and volume), were transformed into the dataset of 43-element dataset X (the list of dataset X variables shown in Table 2). The trading costs were implemented manually and were set at 1% according to suggestions of [45,47,55,56]. The procedures for LSTMtraining and LSTM-simulation were implemented using the The decay in the offline (variable) learning rate was defined as · 0.1 · exp(i/N off line ), where i represented the current epoch. The equation guaranteed an exponentially decreasing learning rate. The initial offline learning rate was set as double the keras recommendation, while the online learning rate was held at the default. The time horizon was set at n = 1, which means that one-day returns were used to calculate the response variable. The Adam learning algorithm [30] was used as a robust training algorithm due to its efficacious characteristic of combining AdaGrad and RMSProp. The number of offline training sessions was determined experimentally. The number of offline epochs were set very low (compared to [19]) to ensure that only general relations between data were captured (to avoid overfitting problems, since the batch size was fixed at 1). The time steps T = 32 were held as a compromise between the actualization of the LSTM network on the one hand and the stability of the internal memory cell on the other. The initial cash per stock was set for a moderate investor (due to the relative evaluation of all ITSs, the initial cash is not among the most relevant setup variables) keras deep learning API 8 and the results were visualized with the help of the matplotlib library. 9 Experiments were conducted using the algorithm setup as designated in Table 3. Both common-and unique-models were implemented as a combination of the subsequent (1) LSTM layer and (2) dense layer, where the dense layer consisted of the 3 output neurons (buy, hold, sell) with a softmax activation function. The MTS was given initial cash (free assets) per stock. The offline LSTM-training incorporated the variable (decaying) learning rate and the online LSTM-training fixed learning rate. Although the dataset was fetched and generated from 1 June 2009 to 12 May 2020, only the data from 4 Jan 2010 to 12 May 2020 were used for simulations. The second half of 2009 was used to avoid a blackout period (missing data) due to momentum indicators. Setting the threshold was a purely experimental task for two reasons: (1) a high threshold guaranteed many hold instances and just a sample of buys and sells; a low threshold classified many instances into buy or sell groups but diminished the number of holds proportionally. The proportion of buys and sells turned out to be crucial, as these instances were directly presented to LSTM: when the frequency of buys and sells was too low, the LSTM turned nonreactive; when their frequency was too high, LSTM traded for nothing. (2) When setting the threshold, focus was also kept on the balance between the buys and sells, i.e., the greater the divergence between the two, the less symmetric the operation of LSTM (otherwise special remedies should be taken, such as [48]). It was experimentally determined, for the given sample of data, that threshold = 0.035 gives a reasonable compromise between the frequency and skewness of trading signals. The dataset was split 2/3 (67% was used for the training stage, and the remaining 33% was used for validation) as a compromise between the extended out-of-sample period presented in [20] and the usual data mining practice for 5-or 10-fold validation, e.g., 80% or 90%. Prior to conducting experiments, the variables were normalized between range 0-1 to make them suitable for the LSTM neural network. The next paragraph gives a short comment on financial movements during the observed trading period.
During the observed period, markets performed well. Due to very low interest rates, stock markets acted as substitutes for bank deposits and bonds and therefore attracted many new investors. The performance of markets was so positive in the last time period that market analysts reported potential markets may burst to correct for inflated prices. A large correction occurred in March 2020, when the Coronavirus crisis struck. Although it did not hurt financial institutions as badly as the last financial crisis, it caused significant losses and delays in real sectors. Although monetary policy drivers reacted rapidly in most countries, so that stock markets recovered quickly, many stock prices plummeted momentarily, and stock companies had to address serious financial problems. The period examined in this paper thus addresses both stock market occurrences, bullish and bearish trends, which makes it especially important for ITSs to identify trends and act accordingly.

Proof-of-concept
The proof-of-concept was conducted by (1) selecting the best performances with regard to final portfolio value (on the last trading day) among the 10 runs and (2) adding them together into a "best" portfolio. For the common-model, this means that the model with the highest final portfolio value among the 10 was chosen as the best. For the unique-model, the best portfolio performance was extracted for each stock independently among the 10 runs, and the 27 individual best performances were then summed into a single portfolio. Further steps in the proof-of-concept incorporated providing visuals of both performances on the chart and comparing these to traditional ITSs. Figure 4 shows the obtained results.
In the figure, "LSTM1" represented the best uniquemodel performance and "LSTM2" represented the best common-model LSTM ITS performance. According to the final portfolio value, the former was the most profitable among all the ITSs. The common-model was not as prominent as its predecessor but still scored higher profit compared to the traditional ITS. Although the passive ITS was the least sophisticated among all, it performed the best during the bullish trend, but on the other hand, it lost much of its potential profit during the Coronavirus crisis. Even if the portfolio value of the passive ITS recovered quickly, at the end of the trading period, it suffered an overall loss. The RSI ITS delivered solid performance for the majority of examined period, except for the last 150 trading days, when it missed the potential profit just before the last bullish trend. MACD ITS performed worst of all ITSs on a given set of stocks and did not give any indication of profitability.
The experiments were run on the HP ProDesk 400 G4 MT Business PC using an Intel Core i5-7500 CPU and 8 GB RAM. On average, it took 23 hours 3 min and 40 sec to complete common-model LSTM-training and 29 min and 2 sec to complete online LSTM-training (for the complete portfolio). The offline unique-model LSTM-training lasted an average of 50 min 35 sec per stock, while the online LSTM-training procedure lasted an average of 1 min and 5 sec per stock.

Robustness test between the commonand unique-models
The robustness test was conducted by analyzing all 10 independent runs that were initially carried out. For each of the stocks separately, the final (overall) portfolio value was derived, and the basic statistical momentums, such as minimum, maximum and mean values, were calculated. The purpose of the robustness test was to determine and compare the mean values and min-max spreads between the two proposed LSTM ITSs. Namely, the larger the min-max spreads were, the worse the ITS robustness. Table 4 shows the results of the robustness tests, where maximum (denoted as "max"), minimum ("min") and mean ("mean") values were extracted for each of the stocks in the portfolio, for each of the LSTM ITSs and the differences between mean values of both ITSs were calculated. The results show that some greater disparities in a few stocks arose between the common-and unique-models. For example, the unique-model scored a much higher performance difference, i.e., more than 2,000.00 euros of difference between the common-and unique-models for four stocks: BMW.DE, CON.DE, FRE.DE and HEN3.DE. While the first two belonged to a group of riskier assets, the former two were considered less risky. On the other hand, the common-model scored a much higher mean final portfolio value twice for ADS.DE and DB1.DE assets. We speculated that the large difference between the commonand unique-models was due to the riskiness of the assets, but this was not the case. As a matter of fact, we realized that (1) the variance (and thus the standard deviations) of the common-model was significantly lower than that for the unique-model; (2) the common-model in effect scored a higher mean value 16 out of 27 times, and we concluded that the generalization abilities of common-model were shown Fig. 4 "LSTM1" represents the unique-model, and "LSTM2" represents the common-model. The former was most profitable among all ITSs. The passive ITS was the most prominent during the bullish trend, while RSI missed the potential profit in the last stage of trading. MACD did not show any indication of profitability. The dotted line shows the initial free cash assets The number of samples amounts to N = 10 Differences were calculated between the mean values of the unique-model compared to common-model, i.e., = mean(unique-model) − mean(common-model). Bold values denote positive differences (differences where the unique-models scored higher mean values than the common-models). Maximal profit shows the amount of profit per stock, e.g., if initial free cash assets amounted to 10,000.00 euros per stock, and the maximal final value of the individual stock amounted to 15,845.5 euros (for ADS.DE), then the maximal profit was 5,845.5 euros. For stocks that were traded with a negative maximal profit for both proposed LSTM frameworks, we can assume that they suffered a downtrend during the out-of-sample to be positive; (3) the min-max spread of 16 out of 27 stocks varied minimally in terms of mean values, i.e., less than 1,000.00 euros between the common-and uniquemodels; (4) the common-model proved to be more robust than the unique-model, but due to its higher generalization capabilities, it was also more rigid (lower profitability on certain stocks may be implied); (5) the difference between the sum of mean values for the common-(245,300.59 euros) and unique-models (247,648.17 euros) was also minimal and below that of the passive ITS. This demonstrated that both ITSs traded similarly to each other and that both LSTM ITSs were superior to the passive ITS in the best cases but less superior in the average cases. We conclude that a robust and solid LSTM portfolio performance occasionally beats the market and that extra experiments with adjusting control parameters, architectures and time periods would further increase performance.

Detailed evaluation tests
The purpose of this test was to objectively evaluate the quality of each ITS. The test was carried out by deriving three groups of well-known financial indicators: (1) final profit/loss and beta riskiness, (2) volume (number) of ITS transactions, and (3) the Sharpe ratio, Jensen's alpha and Treynor ratio. The first two groups were extracted directly from the trading results, while the third group employed an additional manipulation step.
The financial beta (β) [14] acted as a standardized financial indicator that showed the ratio of covariance of asset (portfolio) and index prices relative to the variance in index prices. It represents the movement of asset prices relative to the index and thus acts as a substitute for the risk of an asset. Beta was calculated as follows: where r i and r m represent the investment (portfolio) and market (benchmark) daily returns, respectively. Here, the asset returns r i were substituted by an ITS portfolio value, and benchmark returns r m were substituted by "DAX30" returns during the observed period. Next, the Sharpe ratio [43], or the return-to-risk financial indicator, was introduced. It measured the proportion of expected return per unit of risk. It was calculated as follows: where S a represents the annualized asset Sharpe ratio, and N represents the number of days within a year, e.g., set at N = 252 in our case, since one year consists of approximately 252 trading days. The Sharpe ratio was calculated on the basis of raw daily data, and negative values indicate that the expected profit of the underlying investment was below the market return and positive values indicate that the expected profit was higher than the market return.
The following indicators measured value on the basis of final values rather than on daily data. Jensen's alpha α J [27] was provided to measure the excess return over the theoretical expected return. It considered the relative risk of underlying assets (portfolio) compared to the market beta iM and the capital asset pricing model as follows: where α J represents Jensen's alpha, and R i stands for realized returns on assets (portfolio). R f and R M represent the risk-free rate and overall market return, respectively. A positive α J means that the investment is performing better than expected; while a negative α J means that it is performing worse. Finally, the Treynor ratio T [54] measures the amount for the excess reward of underlying investment compared to the risk-free rate per given unit of volatility. It was calculated as the difference between realized return and the risk-free rate, compared to the asset (portfolio) risk as follows: Here, the greater the difference is between the realized return and the risk-free rate, the higher the Treynor ratio, or the greater the beta, the lower the Treynor ratio. A positive T determines that the investment is more suitable than the risk-free investment, while a negative T determines that the investment is less suitable. The obtained results are shown tabularly in Table 5 on the basis of best portfolios only (not on all 10 independent runs). The first group of extracted results considered final portfolio value, overall profit/loss and beta riskiness. Given an amount of cash at the beginning of a trading period (10,000.00 euros per stock) meaned that 270,000.00 euros were initially given to complete portfolio. By trading stocks, this amount varied according to the assets' close prices. Table 5 shows that only a single ITS -unique-model LSTM ITS exceeded the initial amount of cash at the end of the trading period and thus consolidated an overall profit (16.40%). The common-model LSTM ITS followed the unique-model, with a -1.38% of overall loss, but surpassed the passive ITS, which also consolidated very little loss (-3.29%). The RSI ITS accounted for -9.27% of the overall loss, while MACD ITS accounted for an extreme loss of -56.12%. According to the beta indicator, the passive ITS was the riskiest ITS among all, although this riskiness was still lower than the riskiness of the benchmark (complete DAX30 portfolio) due to the three companies not included in the analysis. On the other hand, the common-model and, especially, the unique-model showed very little riskiness compared to the benchmark, and they even scored negative betas. This intrinsically implied that whatever movement happened with the index prices, it did not relate to the LSTM ITSs. We agreed that such implication came as a consequence of prudent, vigilant and reasonable day-to-day trading.
Next, the number of transactions for each ITS is shown, which was derived as follows: for each of the LSTM ITSs, the best portfolio was selected (traditional ITSs were run once only). Then, the number of transactions (no. of times the MTS bought or sold stocks) during the whole period was summed. The maximum, minimum, mean values and standard deviations were calculated from the portfolios, e.g., the unique-model maximally performed 36 trades for one of the stocks in the portfolio, and minimally performed only 1 trade for a stock in the portfolio. Across the complete portfolio, the unique-model performed 9.15 trades per stock on average, with a standard deviation of 7.96. The common-model, on the other hand, traded slightly more frequently, with a mean value of 12.85 per complete For the calculation of Jensen's α J and Treynor's T , we assumed a risk-free rate R f = 2% The highest values are in bold, "common-" stands for the common-model and "unique-" stands for unique-model. The most profitable ITS was the unique-model LSTM. The riskiest among the ITSs was the passive ITS. The latter also demanded the highest expected return. Beta (β) was calculated compared to the benchmark "DAX30" portfolio. The passive ITS executed only a single buy order per stock and thus scored equal maximum, minimum and mean (average) numbers of transactions. The RSI ITS scored 7.37 transactions per stock on average, and the MACD ITS executed 68.04 transactions on average. Very frequent trading was one of the reasons for poor MACD ITS performance -each transaction incorporated trading costs, and the more trades made, the higher the transaction costs. These costs substantially lowered the portfolio value. Additionally, note that transaction costs imposed significant volatility as well. For instance, if transaction costs were set at 1%, a drastic 1% loss was always imposed when buying or selling stocks instantly; among others, this negatively affected the calculation of beta and the rest of the indicators. Given the investment performances, RSI and MACD ITS scored negative Sharpe ratios, while the passive and robust LSTM ITSs scored positive ones. The highest Sharpe ratio among the last three was scored by the unique-model LSTM ITS and was thus denoted as bold in the table. According to Jensen's alpha α J , the passive and unique-model ITSs performed better than (theoretically) expected. Due to two of the negative betas, we could not perform objective and unbiased Treynor ratio analysis, as the results would not be meaningful.

Concluding remarks
We have implemented two promising variants of online (constantly-updating) LSTM neural networks for the purpose of stock trading and compared their trading performance to that of traditional ITSs. We have extended the experiments from a single asset to a complete portfolio. The German financial market represented by DAX30 blue-chip stocks was exploited for this study. We have experimentally shown that LSTM neural networks can be exploited as robust ITSs that score solid portfolio performance in average cases and beat the market occasionally in the best cases. Both proposed LSTM ITSs performed outstandingly well in the bearish market when selling assets, and limiting losses came into play. We determined that additional steps towards improving the efficiency of LSTM ITS during bullish markets should be taken to increase the mean final overall profit of multiple independent runs. The robustness tests that were carried out in the paper showed that LSTM can indeed be a very reliable and robust tool. Therefore, additional experiments with control parameters and architecture optimization are necessary to guarantee that the mean overall portfolio value outperforms that of passive ITS. Considering the given outlines, we rejected any doubts of the underlying hypothesis and thus concluded that it was indeed possible to earn higher-than-normal profits on an open and transparent stock market during both bullish and bearish market conditions using innovative LSTM neural networks. Further, we attached important conclusions of the paper point-wise: -The limiting loss capability importantly contributed to the negativity of the calculated beta, which is in accordance with Kay [28]. -Universal models are to be stipulated when the complexity of the trading system needs to be low. For well-differentiated financial instruments, some form of preprocessing, such as clustering [51], might be beneficial. Nevertheless, a crucial step for combining clustering with later building a trading system is the selection of features (variables). One must remember that all fundamental, sectoral or macroeconomic information was discarded in this paper, as all information originated from market quotations only. -Common-and unique-models incorporated entirely different approaches to modeling, but both showed exceptionally similar performance on average. On the basis of 10 independent runs, the mean final portfolio values between them differed by less than 1%. -The single common-model for intervening on stock market was exploited to verify the existence of subjective factors mentioned earlier that drive asset prices. As the common-model performed well on a broad set of stocks, we concluded that similar factors existed on market that affected the asset prices of many stocks in the same (similar) way. Implicitly, this meant that investor (psychological) behavior was similar across many stocks and dependent on fundamental factors to some degree. Of course, we could not capture investor behavior directly, but we exploited the common-model as a proxy.
Kay defended the concept that a negative beta came as a consequence of a decreasing algorithmic performance relative to the general index, and vice versa, which in effect caused the trading algorithm to underperform in most (bullish) situations and outperform the general index during the index's negative (bearish) returns. This is exactly what we observed in the results. Chourmouziadis and Chatzoglou [9] performed a study very similar to ours; by emphasizing the effect of economic, political and psychological factors on financial markets, they exploited a fuzzy stock trading system and realized that their proposed ITS produced lower returns than the passive ITS on the bull market but effectively avoided large losses on the bear market. Berutich et al. [2] showed that a genetic programming ITSs handled avoiding large losses that were caused in the 07-09 subprime crisis well. Using a portfolio of 21 Spanish stocks, they managed to earn minimal positive returns during the market collapse. These two findings were in accordance with our results. Hu et al. [25] have in literature survey explicitly stated that many algorithmic trading systems commonly suffered from poor trading performance during uptrends but adequate limiting losses in downtrends. The conclusion on similarities of commonand unique-models (mean portfolio values differing for less than 1%) contributed to the field of automatic trading as follows: strong psychological (deterministic) factors exist on financial markets that similarly affect numerous financial instruments. These factors could be divided into two subgroups: (1) universal factors and (2) financial instrument-specific factors (also called universal price formation mechanisms [46] or stylized facts [15]). On average, models exploiting any of these provide similar portfolio returns.
For future work, external data information that are consequence of rapid information technology outbreak should be included. An excellent example of such data seem to be support and confidence models that daily search the internet for comments and news regarding the stocks' or other financial instruments' portfolio and subjectively asses these information as positive or negative. External information at high frequency would drastically increase the resilience and reliability of current trading models (comprehensive examples of analyzing sentiment from information sources, such as blogs and social media, are shown in [35,63]). Finally, performed case study could be extended by increasing the number of unique-and commonmodel runs from N = 10 to N = 100 or 1000. We expected that the trading performance of the unique-and commonmodels would become even better through this extension. Further, experiments could be extended by incorporating other foreign stock markets, such as the US and Asian and incorporating other financial instruments, such as bonds, cryptocurrencies, commodities, forex, etc. Finally, the LSTM trading strategy could be exploited to actually intervene in stock markets, either as a recommendation system or combined with a trading bot and dedicated hardware, as a real-time web trading system. Here, the trading period could be lowered significantly to enable intraday buying and selling orders. Finally, the effect of poor performance during uptrends should be carefully studied, and appropriate remediation should be implemented.


extfieldsofstudy:
Computer Science
Medicine

text:
Eavesdropping Vulnerability and Countermeasure in Infrared Communication for IoT Devices

Infrared (IR) communication is one of the wireless communication methods mainly used to manipulate consumer electronics devices. Traditional IR devices support only simple operations such as changing TV channels. These days, consumer electronic devices such as smart TV are connected to the internet with the introduction of IoT. Thus, the user’s sensitive information such as credit card number and/or personal information could be entered with the IR remote. This situation raises a new problem. Since TV and the set-top box are visual media, these devices can be used to control and/or monitor other IoT devices at home. Therefore, personal information can be exposed to eavesdroppers. In this paper, we experimented with the IR devices’ reception sensitivity using remotes. These experiments were performed to measure the IR reception sensitivity in terms of distance and position between the device and the remote. According to our experiments, the transmission distance of the IR remote signal is more than 20 m. The experiments also revealed that curtains do not block infrared rays. Consequently, eavesdropping is possible to steal the user’s sensitive information. This paper proposes a simple, practical, and cost-effective countermeasure against eavesdropping, which does not impose any burden on users. Basically, encryption is used to prevent the eavesdropping. The encryption key is created by recycling a timer inside the microcontroller typically integrated in a remote. The key is regenerated whenever the power button on a remote is pressed, providing the limited lifecycle of the key. The evaluation indicates that the XOR-based encryption is practical and effective in terms of the processing time and cost.

Introduction
Infrared (IR) communication has been used to wirelessly control electronic devices such as TVs, air conditioners, audio, beam projectors, and healthcare gadgets. Some works utilize the IR module to control home appliances by recognizing human gestures with wearables such as armbands [1], or gloves [2], or other sensing devices [3]. Some research works also use the IR device as a medium to take commands from smartphones and send those to home electronics [4][5][6][7][8]. The IR communication is also used to operate special devices such as robots [9,10]. For communication, it is required to have Infrared Emitting Diode (IRED) on the transmission side and IR photodiode on the reception side. The hardware components on the transmitter and receiver are cheaper than the ones in other communications such as Wi-Fi, Zigbee, and Bluetooth, and the power consumption is low [11]. Since the IR communication uses light, it does not collide with the radio signal such as Wi-Fi, Zigbee, Bluetooth, and 5G. The IR channel can be used freely because it is unregulated [11].
The IR communication has several shortcomings. First, it can be disturbed by natural light, artificial light, and/or atmospheric dust. Another drawback is that that the communication distance is short. The last one is that the reception angle is narrow, meaning that the transmitter and the receiver should better face each other for seamless communication.
If the angle between the transmitter and the receiver is not within a certain range, the communication is not stable or may not even be possible [11][12][13][14][15]. To overcome these limitations in practice, electronic devices are designed with two and more IREDs and/or reflector. This enables a farther and wider communication for user's convenience. Unfortunately, it could incur security breaches.
Traditionally, only simple operations have been supported with the IR remote controller (referred to as remote hereafter). For example, TV can be turned on or off, and TV channels can be changed with a remote. These days, complex and intelligent operations are supported in electronic devices, especially with the introduction of IoTs. For example, smart TVs are equipped with an operating system such as Android [16]. These TVs are connected to the internet and provide various services requiring personal information.
To purchase goods and/or watch content, the private and sensitive information such as credit card number, resident ID, or ID/PW should be entered with the IR remote. Smart TV can also serve as a bridge for IoT devices used at home. Thus, users can operate and/or monitor the IoT devices through the TV. Figure 1 shows the eavesdropping threats using the IR. Eavesdropping is the act of secretly listening to the communication of others without consent, in order to gather information. The eavesdropper can be placed outside the home or company buildings. Thus, the users could be exposed to an environment where sensitive information can be leaked by eavesdroppers. The IR eavesdropping can happen in several other places. For example, hospitals have a variety of life-supporting devices using radio communication and are strictly managed to prevent communication collisions. Some research studies [17,18] show that light such as infrared can be used to send information related to a patient, not disturbing the radio communication. While using the IoT device, it would be a huge threat if patients' sensitive health information such as patients' disease [19] is leaked and/or tampered with. Another example is the automatic payment system in the highway tollgate [20]. The attacker can lower his/her toll by manipulating the entry place. IR communication is open to the public. Anyone with a simple system with a microcontroller unit (MCU), IRED, and IR photodiode can collect data and/or operate a device. This paper demonstrates the eavesdropping threat and proposes an inexpensive countermeasure for security. The countermeasure means an action taken to counter or offset the threat. This paper is organized as follows: Section 2 summarizes the related work; Section 3 discusses background information; Section 4 reports the IR reception sensitivity; and Section 5 demonstrates the eavesdropping threat. Section 6 presents a countermeasure for security and Section 7 concludes this paper.

Related Works
Security is one of the big concerns in the IoT domain due in part to the low-cost requirement. In particular, personal information and privacy should be protected and guaranteed because many IoT devices tend to be intertwined with personal and daily lives. There are various research efforts for IoT security. Rihab et al. [21] use facial and/or voice recognition as a method of preventing unauthorized access to IoT. Mamun et al. [22] worked on cryptography applicable to IoT by analyzing lightweight symmetric encryption algorithms such as CLEFIA and TRIVIUM. Pietro et al. [23] statically analyzed web applications used in IoT. They discussed top 10 vulnerabilities in The Open Web Application Security Project (IoT 2018) and how these can be exploited. Xinyu et al. [24] proposed a Secure Wi-Fi IoT Communication Router (SecWIR) framework designed on top of home Wi-Fi routers. This work was motivated by the fact that some router devices have either no security protocols deployed, or have problematic security protocol implementations. SecWIR claims to provide a secure IoT communication capability to home IoT devices.
There are two research efforts regarding security issues with IR communication. Zheng et al. [25] developed an IR electronic gadget inside a keyboard to steal sensitive information and control IoT devices autonomously. It was designed to attack IoT devices using IR from a remote site. The suggested countermeasures focus on the detection methods: sniffing the IR signal using the IR receiver tool and measuring the power consumption. Zhen et al. [15] designed an IoT system interconnecting legacy IR devices. They aimed to securely control these IR devices over the internet with a smartphone. The Raspberry Pi-based system equipped with a Wi-Fi module and IR transceiver acts as a bridge between the smartphone and the legacy IR devices. The smartphone is connected to clouds using the TLS protocol. The paper described threats against the legacy IR devices such as replay attack, brute-force attack, and drone attack. It demonstrated that the custom-built IR transmitter is able to reach as far as 47.31 m. It also showed that a drone flying outside a building can turn a TV on and off inside a conference room. To prevent such attacks, the paper stressed the importance of secure IR communication with encryption. However, the authors were concerned about the hardware cost and IR frame transmission delay that comes with the encryption. Our paper is different from Refs. [15,25] in that we propose an inexpensive and close-to-zero-delay scheme for secure IR transmission.
In addition to IR remotes, the RF technology is gaining popularity these days in remote controllers. The RF remote typically uses 2.4 GHz for communication. IR and RF technologies adopted in the remote have pros and cons [26]. Table 1 shows the differences between these two technologies. The transmission distance of IR is known to be roughly up to 10 m, whereas it is up to 50 m with RF remotes. The IR signal is blocked by objects and/or walls, whereas the radio signal can pass through those obstacles. This means that the eavesdropping risk is much higher with RF remotes. IR communication is not restricted by international regulations, so it can be used freely. On the other hand, RF communication should follow the regulations of international organizations such as the European Telecommunication Standards Institute (ETSI). In the case of RF remotes, there is also a risk of conflicts with other wireless technologies such as Bluetooth or Wi-Fi [27] since they use the same 2.4 GHz frequency band. The hardware cost of the RF remote is typically more expensive than the one for IR remote.  Figure 2 shows the infrared spectrum with a wavelength of 740 nm or more outside the visible light range. Infrared rays are classified into near infrared (NIR), short-wave infrared (SWIR), mid-wave infrared (MWIR), and far infrared (FIR) [29,30]. The IR frequency is not reserved and thus can be used freely. The remote utilizes NIR, of which the wavelength ranges from 740 nm to 1000 nm [31,32]. The IR communication is also possible in the SWIR [13,33]. The naked eyes are not able to see the infrared.

Communication Protocols Using Infrared
There are several communication protocols using IR such as Aiwa, BoseWave, Denon, Dish, JVC, Lego, LG, Mitsubishi, NEC, Panasonic, RC5, Samsung, Sanyo, Sharp, and Sony [28]. The NEC [15,28,34,35] is one of the widely used protocols. Figure 3 shows the frame format of the NEC protocol, which is designed to transmit a 32-bit in one frame. As shown in Figure 3a, a frame is composed of 1-bit start, 8-bit address, and 8-bit command. The start bit ( 1 ) is required to identify the beginning of the frame because it is based on asynchronous communication. The 8-bit address ( 2 ) is used to identify the device and the 8-bit command ( 3 ) is used to deliver the pressed key information in the remote. The inverted address and command are appended as shown, to verify the received information. Figure 3b shows the way to transmit the frame data, which is modulated with 38 KHz carrier. The logical '1 is represented with 560 µs assertion and 1690 µs deassertion. The logical '0 is represented with 560 µs assertion and 560 µs deassertion.

IR Reception Sensitivity Experiments
This section measures the reception sensitivity of IR modules integrated into consumer electronic devices. The IR reception sensitivity refers to the distance, position, and rotation to which electronic devices can respond to the IR transmitter. Figure 4 shows experimental gadgets including Samsung TV, LG TV, set-top box, and each device's remote. We have first measured the reception sensitivity according to positions and rotations. Figure 5 shows the experimental environment where the remote's position was changed by 30 • , marked as a ~g . The remote's angle was also changed by 30 • in each position. The distance between the TV and the remote was fixed to 3 m in this experiment. The IR communication is known to be affected by external illumination [11,12,15]. Thus, the experiments were conducted under two different ambient lights in an empty room with a size of 8 m × 7 m; dark environment with 0 lx and bright environment with roughly 700 lx. The remote was pressed 10 times in each rotation at each position of a ~g . Figure 6 shows the experiment results. The remote was pressed 10 times in each rotation at each position of a ~g . The x-axis represents the remote's rotated angles. The 90 • means that the remote was directly pointing at the devices (TVs or set-top box). The yaxis is the success rate at which a certain number of trials were responded to by the devices.  Regardless of the positions, the devices responded when the range of angles was 60 •~1 20 • . This includes the angle at which the remote was directly pointing at the devices. However, the devices did not respond when the angle range was 210 •~3 30 • in all positions. The devices responded to a wider angle in c , d , and e positions than the others. For example, in the d position, the Samsung TV responded with the remote angled to 30 •~1 80 • . As the remote positioned further away from the center ( d ), the response tended to be unstable. For example, the Samsung TV responded when the angle was 180 • in the position g , but did not respond in the position f .
In the second experiment, we have measured the distance from which the devices respond as the IR remote moves away. The experiment on distance is important because it indicates how far the attack would be possible from. Figure 7 shows the experiment environment where the distance can be as far as 25 m. The ambient light is roughly 3170 lx under sunlight. The remote was also pressed 10 times at each distance from 1 m to 25 m. Figure 8 shows the experiment results. The Samsung and LG TVs responded up until the maximum distance of 25 m. However, the set-top box responded from a distance of up to 20 m.

Eavesdropping Experiments
In this section, we performed experiments on eavesdropping on the IR frames from the remotes. Figure 9 shows the experimental environment where eavesdropper's positions are marked as a ~l and remotes are located at α, β, and γ. The experiment was performed in a 7 m × 12 m empty room. The eavesdropper was designed using Arduino [36], an IR photodiode, and a 7-segment. The 12 eavesdroppers were positioned 3 m away from the TVs or the set-top box at a ~l . The remotes were pressed at three different positions (α, β, and γ). Table 2 shows the IR protocols and keypad mapping that the remotes utilize. Samsung uses its own protocol, and LG and the set-top box implement the NEC protocol. The difference in the protocols is the format in the address and command sections shown in Figure 3a. The 7-segment on the eavesdropper was used to check the received data visually. For example, the eavesdropper displays a '1 on the 7-segment when the 'On/Off' keypad input from the Samsung remote is received. The experiments were repeated 10 times by alternately pressing the two buttons on each remote.   Figure 10 shows the experiment results. Regardless of the remote's positions (α, β, γ), all the eavesdroppers surprisingly collected the correct data even from behind the remote. The eavesdropping demonstration is available at https://youtu.be/VXxg-IvlqVU (Video S1: accessed on 8 December 2021).  We further experimented with the possibility of collecting keypad inputs from outside. Figure 11 shows the experimental environment where the virtual home surroundings were set up. It is a virtual setup resembling an eavesdropper located behind the windows with curtains at a certain home. In the experiments, the remote points to various angles as shown in Figure 11a. The experimental environment was located in an empty space of 3.7 m × 2.45 m, and the ambient light was roughly 700 lx. The brightness of the light source was 9800 lx. People use curtains or blinds on windows at home. The curtains play a role in both blocking light and protecting privacy. We used diverse curtains ( 1 ~5 ) and hardcover paper ( 6 ) shown in Figure 11b, exhibiting different light transmission characteristics. Table 3 shows the result of the light transmittance measurement depending on the curtain fabrics. The fabric 1 does not block any light whereas the fabric 5 and hardcover paper 6 completely block the light. Table 3. Light transmittance measurement of curtain fabrics in Figure 11b.  Figure 12 shows the success rate of eavesdropping by pressing the remote 10 times. Surprisingly, the eavesdropper successfully collected all keypad inputs from all angles (α~ε) even with the fabric 5 , which completely blocks light.  Simply eavesdropping on the TV remote may not seem dangerous because the changing of channels or increasing volume has nothing to do with personal information. However, a problem arises when a user authenticates himself/herself or places an order by accessing the internet via TV. These days, TVs typically provide network capability, as all the electronic devices are connected in the IoT era. For example, Figure 13a shows a picture accessing the e-mail account via the set-top box. The user enters the ID and password to authenticate himself/herself. Figure 13b shows a picture of the theft of a user's ID and password using a designed eavesdropper. The exposed sensitive data, ID, and password can be used for exploitation.

Countermeasure
This section proposes a countermeasure to prevent the eavesdropping. The countermeasure should be simple and effective since the remote is usually used in consumer electronic devices where the cost is an important metric to consider. Furthermore, in the consumer electronics sectors, it is also required to minimize the customers' involvement yet still provide a secure mechanism. Basically, a simple encryption can be used to effectively block the eavesdropping. In a typical communication system with cryptography, the key is shared between two parties in a handshaking manner. However, in the IR system, the key cannot be shared in the challenge-response manner at the beginning because the IR communication is based on the simplex; one party should pass the key to the other party.
The key itself can be generated with two different methods: the user-generated and the device-generated methods. The former means that a user sets a password (key) by pushing buttons on the remote when using the remote for the first time. However, it is burdensome because the user must remember the password all the time and the password must be set again when replacing the battery in the remote or when using a new remote. In addition, there is a risk of password leakage from social engineering because people often use a meaningful sequence of numbers such as birthdays or phone numbers. The device-generated method automatically generates a key from the device. A key can be generated from the free-running timer in an MCU [37][38][39] typically integrated inside remotes. We use the latter for the key creation. The life cycle of the key is also an important issue because the key could be leaked eventually if the same key is used all the time. Thus, the key should be changed after a certain period of time. This paper takes an aggressive approach to security and generates a new key each time the power button in a remote is pressed. Figure 14 shows the proposed hardware key generator using an 8-bit timer inside a remote. The timer value is incremented every rising-edge of the clock and wraps around to 0 when reaching a maximum (255). This process is repeated continuously. When the power button is pressed on the remote, the timer value at that time is stored in the key register. The stored value is used as a disposable session key. This key is used until the TV is turned off. When the TV is turned on the next time, a new random key is created in the same way.  Figure 15 shows the key transmission scheme. Many electronic devices utilize the NEC protocol for the IR communication. The address and command values are customized and assigned by the manufacturer following the format explained in Section 3. The address is typically fixed for each electronic device. For example, the experimented Samsung TV, LG TV, and set-top box use 0 × E0, 0 × 20, and 0 × 01, respectively, as shown in Table 2. We use the address field to transmit the key only when the power button is pressed as depicted in Figure 15. In other words, when the value for the power button is in the command field, the address field contains the key.  Figure 16a shows the encryption scheme with the key. The command assigned to each button (except the power button) is encrypted using an XOR operation with the key, as in Equation (1). The XOR is a simple operation. Thus, the hardware cost is low and the processing time is negligible. Code 1 shows a pseudocode for encrypting an 8-bit command with a key and creating an IR packet.
Enc(8-bit command, 8-bit key) = 8-bit command ⊕ 8-bit key, One problem with this scheme is that the encrypted command during normal operation can be the same as the command for the power button, in which case the device is turned off. To avoid this problem, we impose a restriction that the device's address is not allowed to be used as a key. This means that when the device's address is delivered, the command should be decrypted with XOR on the receiver side. The strength of the encryption depends on the 8-bit key for the scheme in Figure 16a. The simple XOR scheme using a fixed key could be easily deciphered by eavesdroppers with the brute-force attack. This is because the same key is used until the device is turned off. To cope with the problem, we introduce a counter so that the encrypted command becomes a different value for the same button. Figure 16b shows the proposed encryption scheme using a key and an 8-bit counter. The command is encrypted with the same XOR operation yet with 'key + counter' as in Equation (2). Code 2 shows a pseudocode for encrypting an 8-bit command with a key and creating an IR packet.
Enc(8-bit command, 8-bit key) = 8-bit command ⊕ ((8-bit key + 8-bit counter value) mod 256), The counter is incremented when the button on the remote is pressed each time, like the counter mode in encryption [40]. This means that the counter value should be passed to the receiver side for decryption. To transmit both the 8-bit counter and an 8-bit address, the extended mode [35] in the NEC protocol can be utilized. Figure 17 shows the frame format in the extended mode where a 16-bit address is transmitted instead of an 8-bit address. We use the 16-bit address field to transmit both the 8-bit address and 8-bit counter, as shown in Figure 17. In this scheme, the encryption strength is based on 'key + counter', which is also an 8-bit. However, eavesdroppers should guess the 8-bit value every time the remote button is pressed. For example, the probability of correctly extracting the user ID in Figure 13b is (1/256) 7 because seven buttons were pressed.   Figure 18 shows the experimental setup for the evaluation of the countermeasure. A remote was designed with IRED and an inexpensive MCU called Arduino, and the receiver was designed with an IR photodiode and an Arduino. The key was created using pRNG [41], which is a pseudo-random number generator based on an interruption raised by a watchdog timer.  Figure 19 shows the experimental results showing log messages from the IR transmitter and receiver. It shows the raw, encrypted, and decrypted data for the keypad buttons '1 and '2 . When the power button is pressed, the transmitter sends the encryption key randomly taken from a counter and the receiver recovers it from the IR packet ( 1 in Figure 19). 2 and 3 show the processed and recovered data for the keypad buttons '1 and '2 , respectively. The raw data for the buttons '1 and '2 are 0 × 88 and 0 × 48, respectively. The encryption for the button '1 is performed with the encryption key (0 × A7) and a random counter value (0 × 32). The encryption for the button '2 is performed with the encryption key (0 × A7) and the incremented counter value (0 × 33). The receiver recovers the raw data (0 × 88 and 0 × 48) from the received IR packets with the same XOR operation. To estimate the overhead associated with the proposed method, we have measured the average elapsed time with and without encryption. The elapsed time is the time duration during which the frame information is created upon pressing a remote button. It was measured 10 times and averaged. The average elapsed time is 27,600.80 µs and 27,898.80 µs without and with encryption, respectively. The difference is negligible because a simple XOR operation is used in the countermeasure. The hardware cost of designing a remote with the proposed scheme is also negligible. The proposed scheme in Figure 16b requires 8 XOR gates for encryption and two 8-bit flip-flops for storing an 8-bit key and an 8-bit counter value.

Conclusions
In this paper, we demonstrated the eavesdropping threat with the IR communication and proposed a countermeasure. These days, consumer electronic devices such as smart TVs are connected to the internet for the IoT capability. The IR is an attractive wireless communication method because it is designed inexpensively and does not interfere with the radio signals. In the reception sensitivity experiments, Samsung and LG TV were responded to from farther than 25 m. This means eavesdropping is feasible within a wide range of the area. According to the eavesdropping experiments, the curtains were not able to block the eavesdropping. The proposed countermeasure uses a simple XOR operation with a key. The remote operation time with and without encryption differed by only roughly 300 µs in the experiment. The hardware for encryption requires only eight XOR gates and two 8-bit flip-flops for the key and counter. The proposed solution is simple enough to be implemented in consumer electronic products and does not require the user's involvement in operation.