Hybridizing association rules with adaptive weighted decision fusion for personal credit assessment

ABSTRACT Credit risk has been one of the major challenges emerged from the banking industry in modern financial markets. Served as a typical method, association classification (AC) has been widely used for personal credit risk assessment. It focuses on the relationship between an item and a class based on mined association rules, where three measures, i.e. support, confidence and weighted Chi-square, are generally used to generate association rules. However, most of the existing approaches neglect the discrimination power differences from between items, between measures and between rules. To deal with this problem, in this paper we present a novel approach characterized by hybridizing association rules with adaptive weighted decision fusion (HAR-AWDF) for personal credit assessment. Here in measures and rules are all worked as classifiers, and adaptive weightings are assigned to items via information entropies computed by posterior probabilities of individual items, also to measures and rules with their classification performance. In particular, a threshold scheme is proposed to determine whether a rule is effective, thereby the final decision is made via weighted voting in classification. The experimental results obtained conducting our approach on German Credit Data set show that among all items of the applied credit data, property, savings account and credit history are vital to evaluate personal credit state, in terms of classification accuracy.

Introduction
With the rapid development of inclusive finance, especially the development of online loan, consumer finance and other industries, personal credit industry possesses unlimited demand and potential, which has become the concern in credit industry. Therefore, it is necessary to build a reliable personal credit evaluation system, to differentiate the credit state for a customer (Lv, Li, & Zhang, 2017). Under the background of big data (Jones, Johnstone, & Wilson, 2015), several data mining techniques, including clustering (Huang, Hung, & Jiau, 2006), classification (Jurgovsky et al., 2018), association rules (Ma & Cheng, 2016) and prediction (García, Marqués, & Sánchez, 2019), have been conducted to perfect personal credit rating mechanisms.
Association rules are especially suitable for marketbasket dataset, which reflects the interdependence and correlation between one thing and other things. They are used to mine the correlation between valuable data items from a large amount of data (Jan, 2015). The AC technique (Wanaskar, Vij, & Mukhopadhyay, 2013;Yue & Shi, 2017) employs association rules in the classification process to increase classification accuracy, in which association rules CONTACT Yue Zhang zhangyue@ahpu.edu.cn are generally mined by making use of a priori-based algorithms (Apilettia et al., 2017;Jorge, Marcio, & Mario, 2018;Xie et al., 2019) or their improvements, say, FP-growthbased algorithms (Pei, Wang, & Wang, 2016). These algorithms aim to find all item sets and association rules whose support, confidence or weighted Chi-square are respectively greater than the minimum counterparts that are specified by users according to the applied database. However, they indicate neither significance differences between the candidate items nor classification performance differences between the derived rules. In addition, there are three measures, including support, confidence and weighted Chi-square, linking to a selected rule. In this way, it leads to another problem of how to integrate the decisions from three measures in the classification process. To handle the above problems, we propose the approach of HAR-AWDF to evaluate the personal credit in this work. The adaptive weighting is firstly assigned in our assessment system, according to the entropy calculated by the posterior probability of different items. The adaptive weightings of three classical measures are subsequently computed depending on their classification performance, and the fusion measurement for each candidate itemset is obtained by the weighted sum of three measures. The critical itemsets and rules can be then found, as the deviation between fusion measurements of two rules associated with the same itemset exceeds the given threshold value. Finally, the adaptive weightings of critical itemsets are computed based on their classification performance, and the class label for an instance is obtained by means of weighted voting.
The rest of this paper is organized as follows: In Section 2, we give the background knowledge of this study. The approach of HAR-AWDF is detailed in Section 3. The experimental results are reported in Section 4. And we conclude our work finally.

Association Classification (AC)
The traditional AC technique (Hadi, Aburub, & Alhawari, 2016) is derived as the combination of association rule mining and the resultant classification, which played an important role in the decision-making process in many previous applications. The association rules aim to discover a correlation or association between items, while association rules-based classification is conducted for label prediction. The derived rules that their support and confidence are no less than the given thresholds, can mine some data of strong relevance from the data set, that is, frequent itemsets.
Let T be a transactional database, defined as a set of transactions {t 1 · · · t n }, and X be a set of n different items (symbols or item values){x 1 , · · · , x n , c 1 , c 2 }. An association rule for classification is an implication of the form A → c, where A ⊆ X, and c represents the class value belonging to {c 1 , c 2 }. For example, the interpretation of a rule A → c 1 is that if an itemset A appears in a transaction, then the transaction can be categorized as class one.
In the phase of determining the interestingness of a rule, support (Sup), confidence (Conf) and weighted Chisquare (Wch) are the most widely used measures, which can be described as (1) Support: Sup(x 1 · · · x k → c) = σ (x 1 · · · x k c)/|T| (2) Confidence: (3) Weighted Chi-square: where σ (·) denotes the number of transactions containing the item x or the item set A = {x 1 · · · x k } in a transaction dataset, and |T| represents the size of transactional database T.

Shannon entropy
Shannon entropy (Xiong & Shang, 2016), also known as information entropy, reflects the relationship between the amount of information and the uncertainty of a piece of information. Given an item x, Shannon entropy is computed by where p j represents the posterior probability of the item x, and N denotes the possible category number. The entropy H(x) measures the information content of the item x. The larger the entropy, the smaller amount of information of the item, and the weaker discriminative power of the item to classify an instance with the item value x. Using H(x), the item x is assigned the weight W(x) with

Multi-classifier fusion
Motivated by the fact that individual classifiers perform differently for a same task, the technique in (Kannatey-Asibu, Yum, & Kim, 2017) can be used to enhance the performance of classifier ensemble by reducing classification errors of individual classifiers. Let L be the matrix (7) consisting of the class pattern (the class label) determined by individual classifiers. In Equation (7), l ij , N, m and n denote the class label of observation j by classifier i, the category number, the total number of all the samples, and the number of classifiers in turn. The weight w i of classifier i, indicating its classification performance for m observations, is calculated as where y j is the true class of observation j (j = 1, 2, 3, . . . , m), and For making the final decision on label predication, the voting matrix V is defined as where v(i) is the ith row of the identity matrix I, whose rank is determined by the number of the classes. Then the final decision equation is the dot product of the weighting vector W = [w 1 , · · · , w n ] and voting matrix V: where V(j) denotes the jth column of the voting matrix V.

HAR-AWDF-based assessment system
It is assured that the existing AC-based credit assessment systems neglect the classification performance differences stemmed from different attributions, measures and rules. As the improvements of our approach, the Shannon entropy is used as the criterion to measure the discriminative power of attributions, and the aforesaid multi-classifier fusion technique is employed to increase the accuracy of class prediction of the adopted measures and rules. Furthermore, the strategy of the weighted voting is conducted for final decision-making.

Problem formulation
In this section, we analyze the flaws of conventional AC technique through a simple example, and the raw data are shown in Table 1. For these data, we calculate the measures of Sup, Conf and Wch of single-itemset rules at first, and the results, respectively, are displayed in Table 2. As shown in Table 2, the following observations are very clear.  (1) The support for items a and b is higher than the given threshold 0.2. Hence, items of a and b become frequent items as the Apriori algorithm is applied. However, confidences for rules b → 1 and b → 2 are both 50%, while confidences for rules a → 1 and a → 2 are 85.71% and 14.29%, respectively. It manifests that the contributions of items a and b to categorize transactions are very different. Therefore, it's needed for one to assign weights to items to indicate the difference of discrimination between items.
(2) Confidence for rule a → 1 is 85.71%, which is higher than that of rule a → 2 of 14.29%. Nevertheless, the weighted Chi-square of rule a → 1 is 1.0, which is lower than that of rule a → 2 of 2.250. Thus, from the view of both support and confidence, a transaction with item a is classified in class 1, while with weighted Chi-square, it is categorized to class 2. These indicate that it is necessary to integrate the decisions supported by Sup, Conf and Wch. (3) According to the five items of the transaction dataset shown in Table 1, ten rules linking to single-itemsets are mined. By analogy, a large number of candidate itemsets and rules could be mined from a big dataset, which is bound to bring a huge challenge to algorithmic efficiency. So it is appropriate to set a threshold to prune redundant itemsets and rules. (4) There are many vital categorization rules to be mined for each category. Individual rules give different predictions. Hence, it is reasonable to fuse the decisions made by all selected rules for prediction.

Our schemes
The proposed HAR-AWDF system improves previous AC approaches by providing new schemes to the typical issues described above. The technical details of our approach are schematically illustrated in Figure 1. The HAR-AWDF algorithm functions in two phases: mining crucial single-itemsets and multi-itemsets and making decision fusion. The first phase contains four  steps: weighting items, weighting and integrating measures of candidate single-itemsets, selecting crucial singitemsets, and selecting crucial multi-itemsets. The second phase consists of weighting crucial itemsets and rules, and making a final decision. The followings are step-bystep demonstration on how the HAR-AWDF algorithm works: Step 1: Weighting items According to the confidences for items a, b, c, d and e in Table 2, we calculate Shannon entropies and weights of items via Equations (5) and (6), respectively. The results shown in Table 3 are posteriori probabilities, Shannon information entropies, and weights of five items. By comparing the entropy values of the five items, we observe that the entropy values of items b, c, d are higher than those of items a and e. This means that the discriminative powers of items b, c, d are weaker than those of items a and e which are assigned to weights 0.2627 and 0.2109, respectively. Thus, the equal weight scheme for items, widely used in the previous approach, is improved as the weighted items.
where W(x) denotes the weight of item x and is computed via Equation (6); Sup(x → c) is computed via Equation (2). Three measures of Wsup, Conf and Wch for a single-itemset rule x → c can then be weighted via Equation (8), where a measure is regarded as a classifier. The integrated measure of a rule x → c is eventually obtained by the weighted sum of Wsup, Conf and Wch. The integrated measures of rules with the same singleitemset to class 1 and class 2 are denoted as M 1 and M 2 , respectively. For the example shown in Table 1, the integrated measures of single-itemset rules are displayed in the second and third columns in Table 4. For the singleitemset of a, the integrated measures of rules a → 1 and a → 2 are 1.5599 and 0.4683, respectively. As a component of for algorithmic improvement, this procedure responds to the second issue addressed in Section 3.1.
Step 3: Selecting crucial single-itemsets The selection criteria of crucial single-itemsets can be described as where M 1 and M 2 denote the integrated measures of rules with the same candidate single-itemset linking to class 1 and class 2, respectively; δ is the threshold for selecting crucial single-itemsets. Specifically, the threshold δ is set to 0.1. The difference between the integrated measures between e → 1and e → 2 is 0.3. It is obvious that Equation (13) is then satisfied, so the single-itemset e is treated as a crucial single-itemset. As displayed in Table 4, the crucial single-itemsets satisfied Equation (13) include a, b, c and e. This scheme is distinct from the existing AC approaches in which five items are considered to have the same level.
Step 4: Selecting crucial multi-itemsets Followed by the above procedures, we consider the combinations of items in crucial single-itemsets. Then the programme is repeatedly conducted from step 1 to step 3 in order to select crucial multi-itemsets. Thus the weighted support (Wsup) of a rule x 1 · · · x k → c is calculated by where W(x i ) denotes the weight of item x i and is computed via Equation (6); Sup(x 1 · · · x n → c) is computed via Equation (2). As shown in Table 5, the crucial multiitemsets consist of five two-itemsets ab, ac, ae, bc and ce, three three-itemsets abc, abe and ace, and one fouritemset abce. Explicitly, the scheme of selecting crucial single-itemsets and multi-itemsets, executed in Step 3 and Step 4, aims to speed up algorithmic implementation. This distinctly differs from previous works to use all singleand multi-itemsets for prediction.
Step 5: Weighting crucial itemsets and rules In step 3 and step 4, we get the crucial singleitemsets and multi-itemsets whose classification performance shows actual differences. For this purpose, a crucial single-itemset or multi-itemset rule is endowed with the weight computed via Equation (8), when it works as a classifier. As shown in Table 5, the weights of crucial single-itemset rules e → 1 and e → 2 corresponding to the crucial single-itemset e are both 0.3, while the weights of crucial multi-itemset rules abce → 1 and abce → 2 corresponding to the crucial multi-itemset abce are both 0.1. Obviously, the scheme of weighting crucial itemsets and rules is another element of our approach in algorithmic improvement.
Step 6: Making final decision The weighted voting depicted by Equation (10) is then used to make a final decision via Equation (11) as the end of our algorithm's execution. It is clear that this procedure can further boost the performance of our proposed HAR-AWDF-based system.

Experiments and results analysis
In this section, we conduct several experiments to verify the effectiveness of the proposed HAR-AWDF-based approach. The experimental data are the German Credit Data set 1 in which each data point contains 20 items and class label to indicate good or bad credit risk within 150,000 loan applicants. We achieve the assessment task on a laptop with Intel i5 × 8250u processor under the MATLAB (Version 2016a) programming environment.

Selection of the threshold δ
To compromise classification accuracy and algorithm complexity, we firstly select the appropriate setting for threshold δ experimentally. The results are displayed in Table 6. As shown in Table 6, as to set δ = 6, the accuracy arrives at 88.31%, while this implementation is only 10 s. In the case of δ = 0.1, the accuracy increases to 93.6%.
However, it takes 267 s to execute algorithm far exceeding that in the first case. So, we set δ = 3 for the following experiments.

Integration of measures
By considering that the classification performances of three measures (Sup, Conf, Wch) are sometimes inconsistent, we realize the measures' integration with weighting sum. In this experiment, we randomly select a subset consisting 1000 data points from the original dataset, and then carrying out our algorithm by randomly splitting the training and testing samples in three cases where the ratios of training samples are set as 25%, 50% and 75%, respectively. Figure 2 shows the classification accuracies of four measures (Sup, Conf, Wch and integrated measure) gained in these cases. From Figure 2, the experimental results show that the integrated measure proposed in our approach consistently outperforms those of three other measures (Sup, Conf, Wch). As can be seen, it possesses the performance advantage more than ten percentage points in each trial.

Comparison with representative AC-based approaches
To compare our approach with representative AC-based approaches to validate our improvement, we conduct the experiment by selecting 10,000 instances from the original dataset. The ratios of 12-fold random sample selections from experimental set to make up the training sets are 4%, 5%, 6%, 7%, 8%, 9%, 10%, 12%, 14%, 16%, 18%   and 20% in turn. We perform our approach together with three classical AC-based approaches including CMAR (Li, Han, & Pei, 2001), WCBA (Alwidian, Hammo, & Obeid, 2018) and CBA (Azmi, Runger, & Berrado, 2019) for each trial. The experimental results are shown in Figure 3. With the results displayed in Figure 3, we can observe that our approach becomes the best performer in most of cases except the first one. In addition, in the cases that the numbers of training samples are no less than 800, our assessment system holds the stable and remarkable advantage to the opponents. For the further evaluation of our proposed approach, we randomly select 1000 training samples from experimental set for each trial for a 10-fold cross-validation. The experiment results are shown in Table 7. The experiment shows that the average recognition rate of HAR-AWDF is above 92% with the standard deviation 0.0185, while the average running time is only 14.7 s. The average recognition rates of three reference approaches are all under 71%, and their standard deviations are above 0.02, while the average running times are all more than 30 s. These indicate that our approach outperforms these congeneric AC-based approaches, in terms of the significance and performance of items, measures and rules in classification.

Rationality of mined crucial rules
In this section, we are to interpret the rationality of the first seven crucial itemsets in terms of their significances to evaluate personal credit. These itemsets shown in Table 7 are mined in the second experiment in Section 4.3. The items in crucial itemsets include (1) A0: all credits paid back duly or no credits taken in credit history; (2) B3: owning real estate; (3) E2: savings account or bonds ≥ 1000 DM.
As shown in Table 8, we find the attributes of property, credit amount and credit history are closely related to one's credit score. The item B3 has a 0.94 weight, while the weights of the items A0 and E2 are 0.78 and 0.92, respectively. So we can conclude that the item of 'owning real estate' is more important than items of 'all credits paid back duly in credit history' and 'savings account or bonds ≥ 1000 DM', in accordance with the real-life cognition that a person owning real estate has a good credit rating.

Conclusions
In this paper, we put forward the HAR-AWDF-based personal credit assessment system, as an improvement of traditional AC technique. Our approach is characterized by weighting items, measure integration and thresholding to mine crucial itemsets and rules. In addition, the performance of the proposed assessment scheme is strengthened by the weighted voting in classification step. The effectiveness of our approach is verified by the desirable experimental results obtained by the comparisons with different measures and AC-based algorithms.
In particular, the average recognition rate of HAR-AWDF is above 92%, and the standard deviation is under 0.02 in the case that 10% training samples is randomly selected from German Credit Data set. This manifests that three attributes of the credit data, that is, property, savings account and credit history are vital to evaluate personal credit state, and one owning real estate has a good credit.


extfieldsofstudy: Mathematics Physics

text:
Time series forecasting of stock market index based on ENWMPCA model

Aiming at the high dimension of factors affecting the stock market index, and the problem that MPCA algorithm cannot distinguish the contribution of different eigenvectors to financial time series forecasting, eigenvalue normalization weighted multilinear principal component analysis (ENW-MPCA) is put forward. Due to the existence of correlation between stock markets, 34 technical indexes of 7 stock markets are selected to construct a three-dimensional tensor model. Eigenvectors of different eigenvalues have different contributions to the prediction accuracy of MPCA algorithm, thus adopt ENW-MPCA algorithm for the feature extraction, which can distinguish the different contribution of eigenvectors corresponding to different eigenvalues to the prediction while reducing dimension. Then using support vector machine for regression prediction, the prediction value of financial time series is obtained. Experimental results on the Hang Seng Index shows that, compared with the MPCA, the forecasting error of ENW-MPCA is smaller and the prediction accuracy is improved to a certain extent. This indicates that the proposed algorithm can fully retain the internal structure of stock time series, proving its validity and practicability.

Introduction
As main body of financial market, stock market plays an important role in development of national economy. The change of the stock market index reflects the law of the stock market to a certain extent. Therefore, time series forecasting of stock market index has become a hot spot for both investors and academic researchers [1][2][3][4].
Due to the characteristics of high dimension and numerous related indexes of stock market index time series, directly using the prediction model for classification or regression often lead to the dimension disaster, greatly increasing the computational complexity. Therefore, many existing methods combine feature extraction algorithm and prediction model to predict [5]. Grigoryan.H et al. [6] put forward an integrated forecasting model (ICA-SVM) based on ICA and SVM. Guo.Z et al. [7] proposed a time series prediction method of stock market index based on feature fusion, combining independent component analysis and canonical correlation analysis to extract and fuse features, and then SVM model was adopted to predict.Using the correlation between related stock markets, after building a three-dimensional tensor model with a certain amount of input indexes, Guo.Z et al. [8] extracted features of the model by using multilinear principal component analysis (MPCA). And then regression prediction was carried out, and good results were obtained. However, MPCA algorithm treats the eigenvectors of each dimension equally, and does not distinguish the influence of each eigenvector in the feature matrix on the feature extraction. Therefore, in view of the fact that the MPCA algorithm cannot distinguish the influences of different eigenvectors on the prediction of financial time series, this paper proposes a eigenvalue normalization weighted multilinear principal component analysis (ENW-MPCA). The contribution of eigenvectors of different eigenvalues to the prediction accuracy is distinguished by weighting the eigenvectors of the projection matrix in all the modes of the tensor. It can not only preserve the characteristics of the original MPCA algorithm, but also increase the prediction accuracy of the algorithm by adding feature weights.

Multilinear principal component analysis (MPCA)
MPCA is a multilinear method for dimensionality reduction in all tensor modes, seeking bases in each mode that allow projected tensors to capture the majority of variation of the original tensor. It captures most of the changes of the original tensor, including the structural information and correlation, so it can achieve dimension reduction while avoiding the destruction of correlation between the sample data of PCA and overcome the shortcoming of the destruction of data structure. , project the tensor to subspace so as to achieve dimension reduction:

ENW-MPCA algorithm
Aiming at the problem that MPCA algorithm cannot distinguish the contributions of different eigenvectors on the prediction of financial time series, a eigenvalue normalization weighted multilinear principal component analysis (ENW-MPCA) is proposed in this paper. By weighting the eigenvectors of projection matrix in all the modes of the tensor, the contributions of eigenvectors of different eigenvalues to the prediction accuracy are distinguished. It can not only preserve the characteristics of the original MPCA algorithm, but also increase the prediction accuracy by adding feature weights. In this paper, the eigenvalues after normalizing are used as the weights of corresponding eigenvectors, so as to highlight the important role of eigenvectors of large eigenvalues and also reflect the auxiliary function of eigenvectors of small eigenvalues. After normalizing the eigenvalues selected The eigenvalue normalization formula is: (4) Where,P n is the dimension of eigenvectors and N is the order of the tensor sample. λ ni represents the i-th eigenvalues in the n order of the tensor sample and λ n represents the sum of eigenvalues chosen in the direction of n order.
For feature matrix U (n) (n=1,…,N), the weights are diagonal matrices consisting of , which is calculated as equation (5) Therefore, the linear transformation formula of the ENW-MPCA algorithm can be reformulated by equation (6):

Steps of ENW-MPCA algorithm
The steps to implement the algorithm are as follows: (1) Input the tensor samples: (4) Calculate the data samples after projection: ; (5) Utilize circulation to update projection matrix U (n) (n=1,…,N) constantly，and narrow the ψ y steadily until it is less than the threshold, and output the optimal projection matrix U (n) (n=1,…,N); (6) Normalize the eigenvalues of eigenvectors chosen as the projection matrix: , 1, 2 , , , 1, 2 , ,

Experimental results and analysis
In order to verify the feasibility and robustness of feature extraction of three-order tensor financial data by ENW-MPCA algorithm, experimental researches and analyses were carried out on data of the Hang Seng Index and the Nikkei 225 index, and the prediction effects were compared with MPCA algorithm and feature extraction algorithms based on vector, KPCA and PCA. The SVR of SVM

Data set preparation
The stock data collected from January 4, 2010 to December 9, 2014 contains of 34 stock indexes of 7 stock markets. Three-order tensor model is constructed and is segmented by moving sliding window. Take the empirical value 20 days for sliding window length. So the data are divided into 1000 groups and the first 800 groups are training data, and the latter are testing data. To measure the performance of the ENW-MPCA, 5 performance indicators are selected. The descriptions and formulae of these indicators are described in Table 1. Among them, y(t) is the predicted value, ( ) y t is the real value, ( ) y t is the average value of the predicted value sequence and ( ) y t is the average value of the real value sequence.

Experiments on Hang Seng Index
For SVM, there are some parameters which need to be chosen, including kernel function type t, penalty parameter c, kernel function parameter g and loss function p. Experiments show that when the value of t is 0, the effect is better than other kernel functions, that is, linear kernel function is the best. p is determined by repeated experiments and the cross validation method is used to find the optimal parameters c and g. For the prediction of the Hang Seng Index, the parameters of SVM of all algorithms are as shown in Table 2.  1. SCC verses the change of testQ When using ENW-MPCA and MPCA algorithms for the feature extraction of Hang Seng Index,testQ , which is the ratio of eigenvalues and sum of eigenvalues, is an important parameter. It determines the dimension of projection matrix of each mode, and has a strong influence on the prediction results. The dimension of tensor data is 20*34*7 . To determine the best projection space, change the value of testQ , and Figure 2 shows the curves of the SCC versus the variation of testQ . From Figure 1, ENW-MPCA and MPCA algorithm are basically increasing with the increase of SCC. Curves do not change in the former period, for the reason that the dimensions of features retained remain the same, which causes the changeless of SCC. When testQ increases to 0.9994, the feature tensor extracted is 14*7*6 and SCC of both algorithms reach the maximum. SCC for ENW-MPCA is 0.9133, and for MPCA is 0.9058. But when testQ continues to increase, SCC of the two algorithms fall instead, because the increase of redundancy leads to the decrease of prediction accuracy. Figure 2 shows the closing price prediction of Hang Seng Index based on ENW-MPCA, MPCA, KPCA and PCA algorithms. As can be seen from Figure 3, comparing ENW-MPCA with MPCA, the change of the two curves are similar, but ENW-MPCA has better agreement, smaller fluctuation and smoother curve. MPCA agrees better than KPCA and PCA, and the results are better than the two algorithms, which shows that the feature extraction algorithm based on tensor is better than algorithms based on vector.  When achieving the optimal prediction accuracy, measure indicators on Hang Seng Index by the four algorithms are shown in Tab.4. Compared with MPCA, KPCA and PCA, the performance of ENW-MPCA is optimal, and the prediction accuracy is the highest. MSE of ENW-MPCA is 69050, and the SCC is 0.9136, better than 77783 and 0.9065 of MPCA, which indicates that its predictive effect is better. This shows that using the normalized eigenvalue as the weight of corresponding eigenvector can effectively distinguish the role of different eigenvalues and has a good inhibitory effect on the inherent redundancy information extraction of MPCA algorithm, which makes a better prediction accuracy. But the error limit is -852, error of the twenty-seventh trading days, so the error range of ENW-MPCA is from -900 to 600, the scope is slightly larger than that of MPCA, which is from -800 to 600.
The predictive effect of MPCA algorithm is better than KPCA and PCA, and is more stable. This shows that MPCA, based on tensor, can better retain the original information of data, and has better prediction performance. The measure indicators of KPCA are superior to PCA, which is because the introduction of the kernel function makes KPCA enable to consider the nonlinear correlation between sample data, so the error is small. ,700] The selection of feature dimension is also related to the prediction accuracy and efficiency. The relationship between the square correlation coefficient and the feature dimension of the prediction result is analyzed. Figure 3 shows the curve of the square correlation coefficient of Hang Seng Index with the change of feature dimension. For ENW-MPCA, MPCA, KPCA and PCA, when the feature dimension alters, SCC will be different. This indicates that the chosen of feature dimension can affect the discriminant feature of the four algorithms. From the experimental results, we can see that when the feature dimension is very small, the value of SCC is slow, which is because a small number of features cannot retain enough information of data. SCC of the four algorithms increases with the increase of feature dimension firstly, then increases slowly, and then decreases after the highest point is reached. But in the whole process, the movement trend is slightly fluctuation and not smooth enough. Compared with MPCA, SCC of ENW-MPCA fluctuates more slowly with the increase of feature dimension, and the maximum of it is higher than that of the MPCA algorithm.

Conclusion
This paper puts forward a prediction method of stock market index time series based on ENW-MPCA algorithm, using tensor method to predict stock price. Making no distinction, MPCA algorithm equally treats the role of different eigenvalues, without reflecting the different influences of the major and minor eigenvalues on the eigenvectors. Therefore, improve the MPCA algorithm, using the eigenvalues after normalizing as weight of the corresponding eigenvector, and ENW-MPCA algorithm is proposed. A three-dimensional tensor model is constructed by using the historical data of the stock market indexes. After adopting ENW-MPCA algorithm to reduce the dimension and extract the features, the support vector machine is used for the regression prediction, and finally the prediction results are obtained.
Experimental results show that compared with MPCA, ENW-MPCA has better predictive effect and higher accuracy. Compared with KPCA and PCA, which are based on vector, the performance of feature extraction of ENW-MPCA is better. This shows the relationship between the structure of internal information and data retention data algorithm based on tensor can better retain the structure information of internal data the relationship between data, showing a better prediction performance. This paper verifies the feasibility of applying tensor model to predict the stock market index time series. With the development of stock market in China, it is believed that the proposed algorithm can be applied in a wide range of fields and areas.


extfieldsofstudy:
Computer Science
Medicine
Mathematics

text:
Few-Shot Deep Adversarial Learning for Video-based Person Re-identification

Video-based person re-identification (re-ID) refers to matching people across camera views from arbitrary unaligned video footages. Existing methods rely on supervision signals to optimise a projected space under which the distances between inter/intra-videos are maximised/minimised. However, this demands exhaustively labelling people across camera views, rendering them unable to be scaled in large networked cameras. Also, it is noticed that learning effective video representations with view invariance is not explicitly addressed for which features exhibit different distributions otherwise. Thus, matching videos for person re-ID demands flexible models to capture the dynamics in time-series observations and learn view-invariant representations with access to limited labeled training samples. In this paper, we propose a novel few-shot deep learning approach to video-based person re-ID, to learn comparable representations that are discriminative and view-invariant. The proposed method is developed on the variational recurrent neural networks (VRNNs) and trained adversarially to produce latent variables with temporal dependencies that are highly discriminative yet view-invariant in matching persons. Through extensive experiments conducted on three benchmark datasets, we empirically show the capability of our method in creating view-invariant temporal features and state-of-the-art performance achieved by our method.

I. INTRODUCTION
A N essential task in visual surveillance system is to automatically associate individuals across disjoint camera views, which is known as person re-identification (re-ID). It has gained considerable popularity in video surveillance, multimedia, and security system by its prospect of searching persons of interest from a large amount of video sequences. Fig. 1: A pair of image sequences regarding the same person from iLIDS-VID dataset [13]. The videos exhibit significant visual variations especially in terms of illumination changes and occlusions.
Most existing approaches focus on matching still images represented by spatial visual appearance (shape, texture, and colour). Specifically, one matches a probe (or a query) person observed in one camera view against a set of gallery candidates captured by another disjoint view for generating a ranked list according to their matching distance or similarity. To this end, a body of methods have been developed to extract invariant features [1]- [5] or learn discriminative matching models [6]- [12]. However, people appearance is intrinsically limited due to the inevitable visual ambiguities and unreliability caused by appearance similarities among different people and appearance variations of the same person from significant cross-view changes in terms of poses, illumination, and cluttered background. This motivates the demand of seeking additionally helpful visual information sources for person re-ID.
On the other hand, videos or image sequences are often largely available from surveillance cameras, which inherently contain more information than independent images. One question raised is whether we can obtain more useful information from videos as opposed to still images? Videos are abundant and rich source of human motion and appearance information. For example, given sequences of images, temporal information related to a person's motion can be captured, which may disambiguate difficult cases that arise in the case of recognising the person in a different camera. However, working on videos creates new challenges such as dealing with video sequences of arbitrary length, and the difficulty of learning effective representations while disentangling nuisance factors caused by visual variations.
Most approaches to video-based person re-ID are based on supervised learning to optimise a discriminant distance metric under which minimised intra-video and maximised inter-video distances can be achieved [13]- [15]. They typically extract spatial-temporal features (e.g., HOG3D [16]) on each fragment from which videos are represented as a set of extracted features. To learn data-dependent high-level video features, [17], [18] use Long-Short Term Memory (LSTM) We showcase the feature distribution divergence in cross-view setting on the MARS dataset [21]. We randomly select 100 video pairs across views and extract the hidden vector for each video by using the pre-trained LSTM network [22]. Then, for all videos in the probe/gallery view (shown as the blue/red histogram), we compute the average hidden features and the variance accordingly. (a): Initial probability distribution of latent variables in cross-view exhibit arbitrarily unmeasurable divergence. It is noted that the cluttering of two distributions is caused by the high-dimensional hidden vectors in two views, while applying one-dimensional projection onto the hidden vectors is not suitable for unary matching. After the crossview adversarial learning, view-invariant latent variables are formed and allows comparable matching in their Gaussians. As shown in (b), we select two identities and plot each one's cross-view feature distributions in the subgraphs. In each identity's subgraph, the crossview features are regulated to allow unary Gaussian matching. And the divergence is minimised (as shown in (c)).
networks to aggregate frame-wise CNN features into videolevel representations while mapping into hidden states with temporal dependency. However, videos are much higher dimensional entities, and it becomes increasingly difficult to do credit assignment on each frame selection to learn long-range relationship among them, unless we collect much more labeled data or do a lot of feature engineering (e.g., computing the right kinds of flow features) to keep the dimensionality low. As a matter of fact, it is not realisable for person re-ID to collect massive labeled pairs of video sequences. a) Challenges in Video Matching based Person Re-ID: Matching video footages of pedestrians in realistic surveillance raises several challenges. First, it demands a faithful model with long distance dependency to map sequences into latent variables where both the dynamics of input patterns and effective representations can be learned simultaneously. Second, the model should be designed without demand on large amount of labeled training samples [19]. To this end, it desires a representation learning with access to few labeled data. Last but not the least, learning latent variables as video representations across camera views inherently exhibits distribution divergence, which should be eliminated to make the matching comparable [20]. An illustration on feature distribution divergence is shown in Fig.2. It is seen that the initial probability distributions of latent variables across views are very arbitrary (depicted in Fig.2 (a)). Thus, it requires a unified-view approach for constructing rich latent variables, and allows comparable matching in cross-view setting. Fig. 2 (b) shows that after cross-view adversarial learning, the features are transformed to be view-invariant and comparable for matching (see Fig.2 (c), as measured by the KL-divergence). b) Our Method: In this work, we propose few-shot deep adversarial neural networks to learn latent variables as representations for cross-view video-based person re-ID in the context of few labeled training video pairs. Video observations are modelled using variational recurrent neural networks (VRNNs) [23] that use variational methods to produce latent representations while capturing the temporal dependency across time-steps. To achieve view-invariance, we perform adversarial training to produce the latent representations invariant across camera views. Specifically, the VRNNs contain variational auto-encoders that provide a class of latent variables to capture the input dynamics, all of which conditioned on the previous encoders via hidden states of an RNN. To promote the learned features view-invariant, the network is augmented with crossview verification to update adversarially into the view changes, and it encourages view-invariant features to emerge in the course of the optimisation. The proposed approach is generic as it can be created atop any existing feed-forward architecture that is trainable by back-propagation. Meanwhile, the network is easy to be optimised by adding a gradient reversal layer [24] that leaves the input unchanged during forward passing and reverse the gradient by multiplying it by a negative scalar during back-propagation.
The inputs to the model are high-level percepts for a pair of videos, extracted by applying a convolutional net, namely VGG-Net [25] trained on ImageNet [26]. These percepts are the states of last layers of rectified linear hidden states form a CNN, which are put through the VRNNs to capture the latent dependencies at different time-steps. However, the derived latent variables from VRNNs are less regulated and exhibit varied distributions caused by view changes [27]. To this end, we propose adversarial learning which applies a cross-view verification to explicitly promote the view-invariant feature learning. In order to evaluate the learned representations, we qualitatively and quantitatively analyse the predictions and matching rates in people recognition made by the model. c) Contributions: The major contributions of this paper are three-fold: • We propose deep few-shot adversarial learning to produce effective video representations for video-based person re-ID with few labeled training paired videos. The proposed model atop VRNNs [23] is able to map video sequences into latent variables which serve as video representations by capturing the temporal dynamics. • Our approach addresses the distribution disparity of learned deep features by performing adversarial training [24] to ensure the view-invariance. Also, the algorithm is based on few-shot learning, and thus it is advantageous in generalisation capability to widen its application to large-scale networked cameras. • Extensive experiments are performed on three challenging benchmarks: iLIDS-VID [13], PRID2011 [28], and MARS [21] to show the notable improvement achieved by our method.

A. Person Re-identification
The majority of approaches to person re-identification are image-based, which can be generally categorized into two categories. The first category [1], [3]- [5], [29], [30] employs invariant features that aims to extract features that are both discriminative and invariant against dramatic visual changes across views. The second stream is metric learning based method which are working by extracting features for each image, and then learning a metric where the training data have strong inter-class differences and intra-class similarities [6], [31]- [35]. These approaches only consider one-shot image per person per view, which is inherently weak when multi-shot are available, due to the intrinsically ambiguous and noisy people appearance, and large cross-view appearance variations.
The use of video in many realistic scenarios indicates that multiple images can be exploited to improve the matching performance. Multi-shot approaches in person re-identification [1], [2], [11], [13], [36], [37] use multiple images of a person to extract the appearance descriptors to model person appearance. For these methods, multiple images from a sequence are used to either enhance local image region/patch spatial feature descriptions [1], [2], [11], [36] or to extract additional appearance information such as temporal change statistics [37], [38]. These methods, however, deal with multiple images independently whereas in the video-based person reidentification problem, a video contains more information than independent images, e.g., underlying dynamics of a moving person and temporal evolution.
Recently, a number of frameworks are developed to address the problem of person re-ID in the video setting [13]- [15], [39]. The Dynamic Time Warping (DTW) model [40], which is a common sequence matching algorithm widely used in pattern recognition, has been applied into video-based person re-ID [41]. Wang et al. [13], [42] partially solve this problem by formulating a discriminative video ranking (DVR) model using the space-time HOG3D feature [16]. In [39], both spatial and temporal alignment is considered to generate a bodyaction model from which Fisher vectors are learned and extracted from individual body-action units, known as STFV3D. However, these approaches presumably suppose all image sequences are synchronised, whereas they are inapplicable in practice due to different actions taken by different people. An unsupervised approach is introduced by [43] where a spatialtemporal person representation is introduced by encoding multi-scale spatial-temporal dynamics (including histograms of oriented 3D spatiotemporal gradient and spatiotemporal pyramids) from the unaligned image sequences.
A supervised top-push distance learning model (TDL) [14] is proposed to enforce top-push constraint [44] into the optimisation on top-rank matching in person re-ID. It uses videobased representations (HOG3D, colour histograms and LBP) to maximise the inter-class margin between different persons, and likewise in [15]. Nonetheless, these pipelines perform feature extraction and metric learning separately in which lowlevel features are very generic to determine which of them are useful in matching process. Meanwhile, the process of distance metric learning is principled on individual examples (e.g., triplets), which carries little information about neighbourhood structure, and thus not generalised across data sets.
With remarkable success achieved by deep neural networks (DNNs) in visual recognition, person re-ID has witnessed great progress by applying DNNs to learn ranking functions based on pairs [45]- [48] or triplets of images [49].
These methods, which use network architectures such as the "Siamese network", learn a direct mapping from the raw image pixels to a feature space where diverse images from the same person are close, while images from different person are widely separated. Another DNN-based approach to re-ID, uses an auto-encoder to learn an invariant colour feature, whilst ignoring spatial features [50], which turn out to be very crucial in matching pedestrians [?], [51]- [54]. However, existing architectures do not exploit any form of temporal information, and thus not applicable into video-based person re-ID. In order to introduce temporal signals into a DNN, McLaughlin et al. present a recurrent neural network for videobased person re-identification [17]. It combines recurrence and temporal-pooling of appearance data with representation learning to yield an invariant representation for each person's video sequence. Wu et al. [22] deliver an end-to-end approach to learn spatial-temporal features and corresponding similarity metric given a pair of time series.

B. Deep Generative Models for Videos
Ranzato et al. [55] proposed a generative model for videos by using a recurrent neural network to predict the next frame or interpolate between frames. In this work, the author quantise image patches into a large dictionary and train the model to predict the identity of the target patch. However, it introduces an arbitrary dictionary size and altogether removes the idea of patches being similar to dissimilar to one other. A recent model of using Encoder-Decoder LSTM to learn representations of video sequences is proposed by Srivastava [56] where Encoder LSTM maps an input sequence into a fixed length representation, and Decoder LSTM produces the future sequence. Nonetheless, existing studies are dependent on Recurrent Neural Networks (RNNs) where the functions are deterministic and cannot capture variability in the input space. In this work, we study the visual variability in video sequences and capture the variability by developing a recurrent gaussian process model. The family of Recurrent Gaussian Process (RGP) models are defined by Lincoln et al. [57], which, similarly to RNNs, are able to learn temporal representations from data. Also, the authors propose a novel RGP model with a latent auto-regressive structure where the intractability brought by the recurrent GP priors are tackled with a variational approximation approach. While our method is similar to RGP [57] by employing auto-regressive structure as latent states, this work addresses the distribution problem of latent variables effectively such that it provides the best approximation to the true posterior which is not resolved in [57]. In contrast to the model of [57] that needs expensive iterative inference scheme to continuous latent variables with intractable posterior distributions, we perform an efficient inference and a learning algorithm that even works in the intractable case. Our inference model based on the variational Bayes [58], [59] is able to re-parameterise the variational lower bound, which can be straightforwardly optimised using standard stochastic gradient descent techniques.

C. Few-Shot Learning
Few-shot learning is to learn novel concepts from very few examples [60]. It typically demands an effective representation learning pipeline that have good generalisation ability. Generative models can be used to generate additional examples so as to improve the learner's performance on novel classes [60], [61]. Intuitively, the challenge is that these training examples capture very little of the category's intra-class variations. For instance, if the category is a particular pedestrian, then we may only have examples of the person's frontal view in one camera, and one of he/she in back view. Amongst representation learning approaches, metric learning such as triplet loss [49], [62]- [64] or Siamese networks [45], [48], [65], [66] have been used to automatically learn feature representations where objects of the same class are put closer together. However, these approaches cannot be directly applied into video-based person re-ID because they do not explicitly address the crossview variations which may lead to the feature distribution divergence.

III. FEW-SHOT DEEP ADVERSARIAL NEURAL NETWORKS FOR CROSS-VIEW VIDEO-BASED PERSON RE-IDENTIFICATION
In this section, we present a deep adversarial learning to learn video representations for person re-ID in a few-shot learning. In the case of video-based person re-ID, we need to learn from a few sequence examples regarding each person to produce discriminative representations. This motivates the setting we are interested in: "few-shot" learning, which consists of learning a persons/class from little labelled sequence examples. While deep learning requires large training datesets to update the set of parameters, we develop our model by using the variational recurrent neural networks (VRNNs) with continuous latent random variables to model the video sequences, which allow us to perform efficient inference and learning. Then the learned latent temporal representations are put through adversarial training to make them view-invariant. In the following, we first formally describe the problem setting and definitions in Section III-A. Then, we present the VRNNs and the principle of adversarial learning in Section III-B and Section III-C, respectively. Section III-D contains the optimisation and Section III-E details the inference for testing.

A. Problem Formulation
Given a set of variable-length video sequences captured in a network of cameras with N video observations In the case of person re-ID, a video sequence regarding a person appearing in one camera is known as the probe video, and the re-identification is to find the correct match for the probe from a set of gallery videos. In this setting, we further assume each video sequence in the probe camera in the gallery set come with L labels y i ∈ {0, 1} L (each person corresponds to a class). Without loss of generality, N p does not have to equal to N g . Hence, each training pair is consisted of a randomly selected probe sequence X i p and its correspondence X g regarding the same person i with the label y i ∈ L where L denotes the total number of persons. In overall, the training objective of our framework is to jointly achieve two goals: predict the label for each coupled probe and gallery video pair; and impose adversarial learning to regularise the learned representations to be view-invariant.

B. Variational Recurrent Neural Networks (VRNNs)
To model the temporal dependencies between the latent random variable across time steps, we employ the variational recurrent neural networks (VRNNs) [23], which contains a variational auto-encoders [58] at each time step. These encoders are conditioned on previous ones via the hidden state h t−1 of an RNN such as an LSTM [67]. Thus, for each time- where KL(Q||P ) is Kullback-Leibler divergence between two distributions Q and P .
is the generative model. θ e and θ g are parameters of the VRNN's encoder and decoder, respectively. Thus, for each frame sequence as the overall feature representations for the following classification task since it captures temporal latent dependencies across the time steps.

C. Deep Adversarial Learning for View-Invariance
Recall that in the context of few-shot person re-ID there are limited labeled pairs of video sequences in training, we perform the training objective by jointly optimising two tasks: the classification on each sequence and the verification on the correspondence. Intuitively, imposing the classification loss on inputs is able to optimise discriminative classifiers regarding identities with relative similarity in the context of all training classes because the optimal similarity is probabilistically determined by respecting all training categories [22], [27]. The verification loss is principled to regularise the classifier to be view-invariance. The proposed cross-view adversarial learning network includes a deep temporal feature encoder (green) and deep label predictors for the paired input (yellow), which form a standard feed-forward architecture. The cross-view invariance is achieved by embedding a verification loss (red) into the feature encoder via a gradient reversal layer that multiplies the gradient by a certain negative constant during the back-propagation.
Formally, let G y (z i p ; θ y ) and G d (z i g , θ d ) represent the probe classifier (predict class labels y i ∈ {0, 1} L for X i p ) and gallery classifier (predict class labels d i ∈ {0, 1} L for X i g ) respectively with parameters θ y and θ d for a given paired input [z i p ,z i g ]. Here, G y (·) and G d (·) can be modelled using deep neural networks. For the ease of notations, we set L = N p = N g , which suggests each pair input is coupled with one person and the training progressively considers one person out of L identities. Therefore, the classification loss towards X i p and X i g (i ∈ L) can be defined respectively as L y (X i p ; θ y , θ e ) = L B G y (V e (X i p ; θ e ); θ y ), y i ; where L B represents the categorical cross-entropy loss function, and V e (X i * ; θ e ) is the VRNN encoder that maps input X i * into its latent representationz i * . Hence, for notation convenience, we define the combined classification loss as Combing the VRNN training and classification task can yield the following optimisation: (6) As we are aimed in achieving the representationsz i with view-invariance, we can adversarially train the above objective function by incorporating the verification loss. In other words, the verification loss is introduced as the regulariser of Eq. (6): Jointly combining the optimisation problems in Eq. (6) and Eq. (7) leads to our objective function, which can be mathematically express as: where λ is a tuning hyper-parameter to balance the trade-off between optimising on making view-invariant representations and optimising the classification accuracy.

D. Optimisation
The optimisation of Eq. (8) involves minimisation on some parameters, and maximisation on others, i.e., we iteratively solve the following problems by finding the saddle pointsθ e , θ g ,θ y ,θ d such that: (θ e ,θ g ) = arg min θe,θg,θy,θ d E(θ e , θ g ,θ y ,θ d ); This problem can be tackled with a simple stochastic gradient procedure, which is to make updates in the opposite direction of the gradient of Eq. (8) for the minimisation of parameters, and in the direction of the gradient for the maximisation of parameters. In practice, stochastic estimates of the gradient can be computed by using a subset of the training examples to calculate their averages. Thus, the gradient updates can be calculated as: We use stochastic gradient decent (SGD) to update θ g . For the other parameters, we use SGD and gradient reversal layer [24] to update a feed-forward deep networks that comprises feature extractor (VRNN's encoder) fed into the classification task (L y and L d ) and the cross-view verification loss. The role of gradient reversal is to make the gradients from the classification and view difference subtracted instead of being summed. This ensures the classification loss is maximised while the feature representations are view-invariant. Thus, the resulting feature representations can capture temporal dependencies (due to the VRNN objective function L V ) and also cross-view invariance (due to the regressor of L R ). The optimisation is depicted in Fig.3.
Why should this framework learn good video features for video-based person re-ID? The learned latent variables z i are not comparable across videos in cross-view setting which exhibit view variability and distributions. The formalism of cross-view verification offers a systematic mechanism of reducing view variations through generalising the classifiers where the loss function is to penalise the differences between the classifiers undr each view. Thus, this can draw connections between this loss and regularisation of feature activations.

E. Inference and Complexity Analysis
Once the training is accomplished, the inference model q θe (·) can be used to produce the feature representations for a video sequence following the equation: z t |x t ∼ N(µ z,t , diag(σ z,t )), where µ z,t and σ z,t denote the parameters of a Gaussian N(·) whose mean µ and variance σ 2 are the output of a non-linear function of x t , that is, [µ, σ] = φ r (φ x r (x t ), h t−1 ). Given two unknown sequences in the test X p and X g , we are able to obtain the latent variables z p and z g as their respective video representations to compute their similarity value via the element-wise inner product between the latent variables z p and z g [22], that is, Sim(X p , X g ) = z p (z g ) T . The illustration on the inference is shown in Fig. ??.
The main computational cost in the training comes from the learning parameters of Gaussian posterior, that is, ). In our model, we use the pre-trained neural networks [25] to extract features from x t and z t . Thus, φ x r and φ z r are parameterised to be the last fully connected layers of the deep convolutional neural networks [25]. Fortunately, the feature extraction φ x r (φ z r ) can be performed off-line and hence, we consider the complexity regarding φ Enc r and φ Dec r , which are typically a stacked three-layer RNN, and each layer has 1024 LSTM units. As the standard vanilla LSTM is high computational cost because of its complex structure [68], e.g., it has three gates to control the memory cell in addition to the input activation, and thus the computational cost of LSTM is four times larger than a simple RNN with the same dimension of the memory cell. To reduce the computational cost, we compress the output vector of the LSTM by a linear projection referred to as a recurrent projection [69]. Assume the input vector of x t is D-dimensional, and the dimension of the hidden state h t is M , the sizes of weight matrices of multiplicating the input and the hidden state vectors W ·x and W ·m are M ×D and M × M , respectively. Therefore, the computational cost of a single LSTM to calculate the h t from x t at a time step is approximately 4(D + M )M . Since the feature dimension from the deep ConvNets [25] is 1024, and M = D = 1024, the number of multiplications becomes 8 × 10 6 . In our model, we compress the output vector h t into the R-dimensional vector via a weight matrix W rh of size R × M . Then the compressed vector is fed as the input to the memory block of the next layer and the hidden state to the gate activations on the same layer at next time step. Thus, the computational cost of this light LSTM becomes 4(D +R)M +RM . If M = 1024 and D = R = 256, the number of multiplications becomes 2 1 4 × 10 6 , resulting in 72% reduction from the vanilla LSTM.

IV. EXPERIMENTS A. Datasets
To evaluate the performance of the proposed approach, we conduct extensive experiments on three image sequence based person re-ID datasets: iLIDS-VID [13], PRID2011 [28], and MARS [21]. Example images are shown in Fig. 4.
• The iLIDS-VID dataset consists of 600 image sequences for 300 randomly sampled people, which was created based on two non-overlapping camera views from the i-LIDS multiple camera tracking scenario. The sequences are of varying length, ranging from 23 to 192 images, with an average of 73. This dataset is very challenging due to variations in lighting and viewpoint caused by cross-camera views, similar appearances among people, and cluttered backgrounds. • The PRID 2011 dataset includes 400 image sequences for 200 persons from two adjacent camera views. Each sequence is between 5 and 675 frames, with an average of 100. Compared with iLIDS-VID, this dataset was captured in uncrowded outdoor scenes with rare occlusions and clean background. However, the dataset has obvious colour changes and shadows in one of the views. • The MARS dataset contains 1,261 pedestrians, and 20,000 video sequences, making it the largest video re-ID dataset. It provides rich motion information by using DPM detector [70] and GMMCP tracker [71] for pedestrian detection and tracking. As suggested by [21], the dataset is evenly divided into train and test sets, containing 631 and 630 identities, respectively.

B. Experimental Settings
We extracted percepts using the convolutional neural net model from [25]. To fit the input frame into the model, we resize each frame to be 224 × 224, and run it through the convnet to produce the RGB percepts. This is implemented by simply re-scaling the largest side of each image to a fixed length, i.e., 224 × 224 (We use openCV to resize each frame to have 224 width and 224 height). In the case of person re-ID, cropping image has the drawback of potentially excluding critical parts of pedestrians whilst simply resizing an image can still get plausible performance because frames from person re-ID datasets do not have very different aspect ratios. Additionally, we compute flow percepts by extracting flows using the Brox method and train the temporal stream convolutional network as described by [72]. The optical flow is computed using the off-the-shelf GPU implementation of [73] from the OpenCV toolbox. In spite of the fast computation time (0.06s for a pair of frames), it would still impose a bottleneck if done on-the-fly, so we pre-computed the flow before training. To avoid storing the displacement fields as floats, the horizontal and vertical components of the flow were linearly rescaled to a [0, 255] range. We use the 4096dimensional fc-6 layer as the input representation with RGB and flow percepts.
For the iLIDS-VID and PRID2011 datasets, we randomly split each dataset into two subsets with the same size. One is used for training and the other one for testing. For iLIDS-VID and PRID2011, each of which dataset has only two disjoint cameras and each person has one sequence under each camera. Thus, the sequences from the first camera are used as the probe while the sequences from the second camera are used as the gallery. To constitute the training pairs, each person is regarded as a class, and we only select one probe frame sequence and its gallery correspondence as a training paired input. Please note that for the MARS dataset, we maintain the IDs with the probe and its correspondence and thus we have 625 IDs for training. In the testing on the MARS dataset with more than two cameras, we randomly select one camera as the probe view while a different camera is randomly selected as the gallery. Training was performed by following the Adam optimiser and ran the model for 50 epochs with a learning rate of 10 −3 . We set an early stop criteria that the model does not show a decrease in a validation loss for 10 epochs. To set the hyper-parameter such as λ, we select the parameter by using cross-validation where the labeled paired videos {X p , X g } in the training are further split into the training set (S T ) and the validation set (S V ) containing 90% of the original examples and the rest of 10%, respectively. We use the labeled set S T to learn classifiers G y (·) and G d (·). The learned classifiers are evaluated on the validation set S V and parameters are selected corresponding to the classifiers with the lowest validation risk. All our models are implemented on the public Torch [74] platform, and all experiments are conducted on a single NVIDIA GeForce GTX TITAN X GPU with 12 GB memory.
a) Evaluation Measure: To evaluate the performance of our method and also compare the performance against other methods, we employ the standard Cumulated Matching Characteristics (CMC) curve as our evaluation metric, which indicates the probability of finding the correct match in the top R matches within the ranked gallery. In our experimental results, we report the Rank-R average matching rates, which are obtained by randomly splitting the dataset into training and testing 10 trails and the average result is computed.

C. Hyper-parameter Selection
In this section, we set up the hyper-parameter selection on λ by using a variant of reverse cross-validation approach [75] to optimise the λ in the context of view-invariance adaptation. The evaluation on reverse validation risk associated to varied parameter of λ is proceeded as follows. Given the video pairs [X p , X g ] from the probe (X p ) and the gallery view (X g ), we split each set into training sets (X * p and X * g respectively, containing 90% of the original examples) and the validation sets (X V p and X V g respectively). We use the X * p and X * g to learn a classifier CLASS( ) to classify each identity across views. Then, we learn a reverse classifier CLASS( ) r using the set {x, CLASS(x)} x∈X * g and the unclassified part of X * p . Finally, the learned reverse classifier CLASS( ) r is evaluated on the validation set X V p , and the classifier CLASS( ) has a reverse validation risk in the process of addressing crossview invariance. This process is repeated with multiple values of hyper-parameter λ and the selected parameter is the one corresponding to the classifier with the lowest reverse validation risk. As shown in Fig. 5, the adaptation parameter λ is chosen among 9 values between 10 −2 and 1 on a logarithmic scale, and the optimal value of λ is chosen to be 0.6 when the method has the lowest validation risk.

D. Ablation Studies
In this section, we conduct extensive experimental analysis to answer the following questions: (a) How does our method perform compared to other feature learning models for videos? (b) How do we show that the learned temporal latent features are effective in matching persons against view changes? (c) How is the performance of our method affected with respect to varied input lengths of videos? (d) How large the computational complexity is as opposed to existing deep recurrent models? a) Comparison with Other Feature Learning Models for Videos: To evaluate the effectiveness of latent video represen- tations learned by our model, we compare with state-of-the-art deep generative models: • R-DANN: The domain adversarial neural networks [24] is a deep domain adaptation model which uses two components to create domain-invariant representations: a feature extractor that produces the data's latent representations, and an adversarial domain labeller that helps the feature extractor produce features domain-invariant. In our experiment, the feature exactor is a RNN (LSTM) with fc6 as the input representation of data where each LSTM has 2048 units.  Table I. The proposed method is superior in all datasets with large improvements over V-LSTM [56]. The lower recognition accuracy obtained by LSTM model is possibly due to the difficulties of training with large visual variations and cross-view divergence. VAE [58] performs better than V-LSTM [56] by introducing latent random variables which are effective in learning the mapping between videos and latent representations. R-DANN [24] outperforms VRNN [23] by performing domain-invariance training, however, R-DANN is not focusing on the underlying temporal dependencies. In contrast, our approach explicitly addresses the cross-view feature distribution changes by adversarial training on the latent variables produced by VRNNs.
b) View-invariance Study: The second experiment is to demonstrate the property of our features in view-invariance. To this end, we evaluate the matching performance of our method against the difference between feature distributions. Fig.6 with dual Y-axis shows two evaluations: The red line indicates the performance of our method for video matching by reducing the parameter distributions across camera views as the training epochs proceeds. The blue line represents the difference between feature distributions under two camera views measured by the KL-divergence. In this experiment, we use the testing data from the iLIDS-VID dataset, and keep the representations by increasing the number of training epochs.
We can see an improvement in the matching accuracy as more training epochs are proceeded, which is aligned with the decrease in feature distribution divergence as measured by the variational bound of KL-divergence. c) Variable Length: The third study is to investigate how the performance varies against the length of the probe and gallery sequences in testing. Evaluations are conducted on three datasets, and the lengths of the probe and gallery sequences are varied between 1 and 128 frames in step of power of two. Results are shown in Fig.7 where a bar matrix shows the rank-1 re-identification matching rate as a function of the probe and gallery sequence lengths. We can see that increasing the length in either probe or gallery can increase the matching accuracy. Also longer gallery sequences can bring about more performance gain than longer probe sequences. d) Computational Cost Analysis: The final study is to analyse the computational cost of our model as opposed to existing deep recurrent models. We consider the vanilla LSTM [68] with three layers as comparison. The vanilla LSTM and the light LSTM in our model are trained with 1024-dimensional memory cells and the light LSTM has an additional 256-dimensional recurrent projection layer. The comparison results are shown in Table II. It can be seen that the LSTM with projection outperforms the vanilla LSTM with the number of weight parameters one third of the vanilla LSTM.

E. Impact of Adversarial Learning
It is an interesting question to know what is the right time to perform the adversarial operation? It should be at every time step or the last time step of a latent representation learning. If the adversarial training is performed at every time step, namely early fusion, the network learns to produce the viewinvariant representations conditioned on the subsets of the input sequence x ≤T . On the other hand, a late adversarial learning at the last time-step (i.e., late fusion) might be incapable of eliminating feature variations during the representation learning. To study the impact of adversarial learning on the optimal view-invariant representations, we empirically test the two different strategies by assessing the CMC values at R = 1 computed at each time-step. The comparison results on MARS dataset is shown in Fig.8. The results indicate that progressively performing adversarial learning at each time-step on the learned features lead to more optimal representations.

F. Comparison with Sate-of-the-art Approaches
In this section, we compare our method with state-of-theart video-based person re-ID approaches. We consider the following competitors:  • XQDA [11]: Cross-View Quadratic Discriminant Analysis is a static appearance feature based supervised approach that learns simultaneously a discriminant low dimensional subspace and a QDA metric. • eSDC [5]: An effective unsupervised spatial appearance based method, which is able to learn localised appearance saliency statistics for measuring local patch importance. The comparison results on three datasets are given in Table  III and Fig.9. It can be seen that our method has large improvement gains compared against multi-shot methods including XQDA [11], ISR [76], and eSDC [5], in which temporal dependency across frames are not considered. Compared with 3D feature based methods such as HOG3D [16]+DVR [13], SDALF [1]+DVR [13], and STFV3D [39], our method has a high matching rate by learning hidden representations which are able to faithfully describe person videos with variations. Competing methods of RCN [17] and RFA-net [18] are based on RNN structure and use deterministic functions to compute transitions from inputs into hidden states, which cannot capture data variability to be addressed in cross-view video matching, and thus leads to inferior performance to our approach. In contrast, our approach explicitly address cross-view difference by adversarial learning, and the learned video features are more effective in person re-ID. For example, on iLIDS-VID dataset, RCN [17] attains 58.0% at rank-1 matching rate while the proposed method achieves 60.1% at rank-1 value. On PRID-2011 dataset, our method outperforms SI 2 DL [15] by 2.5% at rank-1 while using few labeled training pairs. One thing to be noticed is that our method performs secondary to the method What-and-where [22] in rank-1 and rank-5 on the MARS dataset. The possible reason is What-andwhere [22] is based on supervised learning to match persons with annotations on body parts. In contrast, our method does not require any region annotations. In the combination with KissME [8], Ours+KissME [8] has further improvement on recognition values on all datasets. For instance, Ours+KissME [8] achieves the best results on rank-1 (64.6% and 84.2%) on the iLIDS-VIDS and PRID011 datasets, and the best mAP (52.1%) on the MARS dataset. This allows us to potentially improve the performance in combining with metric learning algorithms.

G. Comparison with Other Few-Shot Learning Methods
We also compared to two recently proposed few-shot learning methods: matching networks [78] and model regression [79]. The matching networks propose a nearest neighbor approach that trains an embedding end-to-end for the task of few-shot learning. Model regression trains a small MLP to regress from the classifier trained on a small dataset to the classifier trained on the full dataset. Both of the two techniques are high-capacity in learning from few examples and facilitates the recognition in the small sample size regime on a broad range of tasks, including domain adaptation and fine-grained recognition. Comparison results are shown in Fig. 10. In terms of the overall performance, our method outperforms the two competitors constantly over the MARS dataset. Matching networks exhibit similar performance to our 53  method, however, matching networks are based on nearest neighbors and use the entire training set in memory, and thus they are more expensive in testing time compared with our method and model regressors. Please note that we do not perform the comparison on iLIDS-VID and PRID2011 datasets because these two datasets have only two cross-view sequences regarding each person.

V. CONCLUSION AND FUTURE WORK
We propose a novel few-shot deep adversarial model with a latent variational structure to learn deep temporal representations for video-based person re-identification with few labeled training examples. The proposed method is based on variational recurrent neural network which provides a family of latent variables to describe input videos with temporal relationships. To eliminate feature distribution divergence caused by view changes, the learned latent features are adversarially trained through a cross-view verification loss to make them view-invariant. The proposed method requires few labeled training examples while generic enough and scalable to larger networked cameras. In future work, we would explore oneshot or zero shot learning with memories to augment the generalisation of the model.