Personalized Context-aware Re-ranking for E-commerce Recommender Systems

Ranking is a core task in E-commerce recommender systems, which aims at providing an ordered list of items to users. Typically, a ranking function is learned from the labeled dataset to optimize the global performance, which produces a ranking score for each individual item. However, it may be sub-optimal because the scoring function applies to each item individually and does not explicitly consider the mutual influence between items, as well as the differences of users' preferences or intents. Therefore, we propose a personalized context-aware re-ranking model for E-commerce recommender systems. The proposed re-ranking model can be easily deployed as a follow-up modular after ranking by directly using the existing feature vectors of ranking. It directly optimizes the whole recommendation list by employing a transformer structure to efficiently encode the information of all items in the list. Specifically, the Transformer applies a self-attention mechanism that directly models the global relationships between any pair of items in the whole list. Besides, we introduce the personalized embedding to model the differences between feature distributions for different users. Experimental results on both offline benchmarks and real-world online E-commerce systems demonstrate the significant improvements of the proposed re-ranking model.

INTRODUCTION
Ranking is crucial in E-commerce recommender systems. e quality of the ranked list given by a ranking algorithm has a great impact on users' satisfactory as well as the revenue of E-commerce recommender systems. A large amount of ranking algorithms [5,6,8,17,22,31,36] have been proposed to optimize the ranking performance. Typically ranking in recommender system only considers the user-item pair features, without considering the local context information from other items in the list, especially by those items placed alongside [9,39]. ough pairwise and listwise learning to rank methods try to solve the problem by taking the item-pair or item-list as input, they only focus on optimizing the loss function to make be er use of the labels, e.g., click-through data. ey didn't explicitly model the mutual in uences between items in the feature space.
Some works [1,38,41] tend to model the mutual in uences between items explicitly to re ne the initial list given by the previous ranking algorithm, which is known as re-ranking. e main idea is to build the scoring function by encoding intra-item pa erns in feature space. e state-of-the-art methods for encoding the feature vectors are RNN-based, such as GlobalRerank [41] and DLCM [1]. ey feed the initial list into RNN-based structure sequentially and output the encoded vector at each time step. However, RNNbased approaches have limited ability to model the interactions between items in the list. e feature information of the previous encoded item degrades along with the encoding distance. Inspired by the Transformer architecture [23] used in machine translation, we propose to use the Transformer to model the mutual in uences between items. e Transformer structure uses self-a ention mechanism where any two items can interact with each other directly without degradation over the encoding distance. Meanwhile, the encoding procedure of Transformer is more e cient than RNN-based approach because of parallelization.
Besides the interactions between items, personalization is another important factor to be considered for re-ranking in E-commerce recommender system. Re-ranking for recommender system is userspeci c, depending on the user's preferences and intents. For a user who is sensitive to price, the price feature should be more important in the re-ranking model. Typical global ranking strategy may be not optimal as it ignores the di erences between feature distributions for each user. In addition, the interaction between items is a ect by user as well. For instance, when users are in the intention of price comparison, similar items with di erent prices tend to be more aggregated in the list. When the user has no obvious shopping intention, items in the recommendation list tend to be more diverse. erefore, we introduce a personalization module into the Transformer structure to represent user's preference and intent on items. e interaction between items in the list and user can be captured simultaneously in our personalized context-aware re-ranking model. e main contributions of this paper are as follows: • Problem. We propose a personalized re-ranking problem in recommender systems, which to the best of our knowledge, is the rst time to explicitly introduce the personalized information into re-ranking task in large-scale online system. e experimental results demonstrate the e ectiveness of introducing users' representation into list representation for re-ranking. • Model. In our personalized context-aware re-ranking approach, we employ the Transformer equipped with personalized embedding to compute representations of initial input ranking list and output the re-ranking score. e selfa ention mechanism enable us to model user-speci c mutual in uences between any two items in a more e ective and e cient way compared with RNN-based approaches. • Data. We release a large scale dataset (E-commerce Reranking dataset) used in this paper. is dataset is built from a real-world E-commerce recommender system in one of the largest online shopping mobile App. Each record in the dataset contains a recommendation list for each user with users' basic information, click-through labels and features for ranking. • Evaluation. We conducted both o ine and online experiments which show that our methods signi cantly outperform the state-of-the-art approaches. e online A/B tests show that our approach achieves higher click-through rate and more revenue for online shopping mobile App.

RELATED WORK
Our work aims to re ne the initial ranking list given by the base ranker. Among these base rankers, learning to rank is one of the widely used methods. e learning to rank methods can be classi ed into three categories according to the loss function they used: pointwise [13,25], pairwise [7,21,22], and listwise [6,8,16,22,31,36,37]. To apply the learning to rank into the recommendation system, the pointwise method uses a single item as input to predict a score for ranking. e pairwise method taking two items as a pair to classify which item should be ranked rst. e listwise method uses the whole list as input and judges the likelihood possibility of the full permutation of items. ey learn a global scoring function within which the weight of a certain feature is globally learned. However, the weights of the features should be able to aware of the interactions not only between items but also between users and items.
ey use the whole initial list as input and model the complex dependencies between items in di erent ways. [1] uses unidirectional GRU [11] to encode the information of the whole list into the representation of each item. [41] uses LSTM [20] and [4] uses pointer network [33] not only to encode the whole list information, but also to generate the ranked list using a decoder. For those methods which use either GRU or LSTM to encode the dependencies of items, the capacity of the encoder is limited. is is because the RNN network encodes the items in a sequentially way according to their positions in the initial list. us the item at the end of the initial list may have li le in uence to the item at the head of the initial list, which is not true in reality. In our paper, we use transformer-like encoder, based on self-a ention mechanism to model the interactions for any of two items in O(1) distance. Besides, for those methods which use decoder to sequentially generate the ordered list, they are not suitable for online ranking system which requires strict latency criterion. As the sequential decoder using the item selected at time t-1 as input to select the item at time t, it can not be parallelized and needs n times inferences, where n is the length of the output list. [2] propose a groupwise scoring function which can be parallelized when scoring the items, but its computation cost is high because it enumerates every possible combinations of items in the list.
Besides the above ranking and re-ranking methods, our work is inspired by the following works. [32] proposed Transformer which is widely used in machine translation as its e ectiveness in encoding high-order and long distance dependencies in sequential list compared with RNN/CNN-based approaches [35,40]. FPMC [18,28] are personalized recommendation methods which learn user embedding from user's click history. YouTubeRec [14] and Bert [15] con rm that it is feasible to use the pre-trained embeddings to enhance certain task-speci c model in practice.

FORMULATION OF RE-RANKING
In this section, we rst give some preliminary knowledge about learning to rank and re-ranking methods in E-commerce recommendation system. en we formulate the problem we aim to solve in this paper. e notation used in this paper is in Table 1.
Learning to rank (o en labelled as LTR) method is widely used for ranking in real-work system to generate an ordered list for information retrieval [21,26] and recommendation [16]. In E-commerce recommendation, the LTR method learns a global scoring function based on the feature vector of items. Having this global function, the LTR method outputs an ordered list by scoring each item in the candidate set. is global scoring function is usually formulated as the following Equation: where R is the set of all users' requests in recommendation. I r is the candidate set of items for request r ∈ R. x i represents the feature space of item i. i is the label on item i, i.e., click or not. P( i |x i ; θ ) is the predicted click probability of item i given by the model with parameters θ . is the loss computed with the label i and the corresponding estimated possibility. e goal of LTR is to nd the best parameters θ that minimize the loss function L.
However, x i is not enough to learn a good scoring function. We nd that ranking in E-commerce recommender system should consider the following information: (a) dependencies between item pairs [9,39]; (b) interactions between user and items. e dependencies between item pairs can be directly learned from the initial list S = [i 1 , i 2 , ..., i n ]. We refer this kind of information as local context information, which is labeled as [4] propose approaches to make be er use of local context information. However, few works consider the interactions between the user and items. e speci c information for user-item pairs represents the preferences of users on di erent aspects of items and can not be learned directly from initial list. In this paper, we introduce a personalized vector pv i for each item i to represent the user's preference on latent characteristics of item i. For example, some people may be sensitive to price of items while others may be sensitive to brand of items. We refer this kind of information as global context information, which is labeled as PV = [pv i 1 , pv i 2 , . . . , pv i n ] in this paper.
We propose a personalized context-aware model to re-rank the initial list by utilizing both local and global context information. Compared with the global scoring function learned by LTR, our re-ranking model seeks to learn a user-speci c scoring function which is able to weight features distinctly for di erent users. e loss function of the model can be formulated as Equation 2.
where S r is the initial list given by the ranking model.θ is the parameters of re-ranking model. Our personalized contextaware model is di erent from the typical ranking approaches in the following aspects: (a) We use a ordered list S r instead of a candidate set I r . (b) When learning the scoring function for each item i, we use the total feature matrix X of all items in the list instead of one single feature vector x i to model the mutual in uences between items. (c) We use a personalized matrix PV to learn the interactions between user and items. Note that both X and PV can be used by the existing ranking approaches to achieve be er performance.

PERSONALIZED CONTEXT-AWARE RE-RANKING MODEL
In this section, we rst give an overview of our proposed Personalized Context-aware Re-ranking Model (PCRM). en we introduce each component of our model in detail.

Model Architecture
e architecture of PCRM model is shown in Figure 1. e model consists of three parts: the input layer, the encoding layer and the output layer. It takes the initial list of items generated by LTR method as input and outputs a re-ranked list. In input layer, we utilize the interaction between user and each item i to generate a personalized vector pv i . pv i , i = i 1 , ..., i n is the personalized vector of user u on item i which represents the user's preference on latent characteristics of item i. We concat pv i with raw feature vector x i , then add it with a position embedding pe i to get the input vector for encoding layer. e encoding layer is a Transformer-like encoder made up of multi-blocks, each of which contains a multihead a ention module and a feed forward network (abbreviated as FFN), the detailed structure is shown in Figure 1 (a). A er encoding layer, the model generates a score for each item using a so max function. e detailed structure will be introduced separately in the following sections.

Input Layer
e function of this layer is to make a comprehensive representation of all input features and feed it to the next layer. First we have a xed length of initial sequential list S = [i 1 , i 2 , ..., i n ] given by the learning to rank method. Same as the learning to rank method, we have a feature matrix X ∈ R n×d feature . Each row in X represents the raw feature vector x i for each item i ∈ S.
Personalized Vector. As shown in Figure 1 (b), we concat the raw feature matrix X ∈ R n×d feature with a personalized matrix PV ∈ R n×d pv to get the intermediate embedding matrix E ∈ R n×(d feature +d p ) , which is shown in Equation 3. PV is produced by a pre-trained model which will be introduced in the following section.
Position Embedding. In order to utilize the sequential information in the initial list, we inject a position embedding PE ∈ R n×(d feature +d p ) into the input embedding. en the embedding matrix for encoding layer can be calculated using Equation 4. In this paper, a learnable PE is used which we found that it led to slightly outperform the xed position embedding used in [32].
At last we use one simple feed-forward network to convert the . . .    .

Encoding Layer
e function of encoding layer in Figure 1(a) is to learn a highlevel representation for item list S by combining both local and global context information. Transformer [32] has been proved to be e ective in many NLP tasks, specially machine translation for its powerful encoding and decoding ability compared to RNN-based approaches [11,12,20]. Self-a ention mechanism in Transformer is particularly suitable in our re-ranking task as it directly models the mutual in uence for any two items regardless the distance between them. Without the decay with distance, Transformer can capture more interactions between items which are far away from each other in the initial list. As shown in Figure 1(b), our encoding module consists of N x blocks. Each block (in Figure 1(a)) contains a multi-head a ention layer and Point-wise Feed-Forward Network (FFN) layer.
Attention Module. e a ention function we used in this paper is de ned as Equation 6: where matrices Q, K, V represent queries, keys and values respectively. d K is the dimensionality of matrix K to avoid large value of the inner product. so max is used to convert the value of innerproduct into the adding weight of the value vector V . In our paper, we use self-a ention where Q, K and V are projected from the same matrices. Each row in E is corresponding to each item in the ranking list. E contains not only the features of items but also the interaction between user and items.
In our model, any two items can a ention with each other. e self-a ention result of each item is the weight sum of the all the items in initial list. Compared with Language Model [27] and SASREC [23], which use the masked self-a ention by preventing the a erward item i n+1 from being encoded into the representation of the item i n .
is is because that [23] focuses on next bucket recommendation: given a sequence of clicked items by the user from 0 th time slot to t-1 th time slot, the algorithm predicts which item the user will click at t th time slot. us it is not allowed to leak the information of ground truth (item at at t th time slot). However, in re-ranking se ings, the information of all the items can be used.
To model more complex mutual in uences, we use the multihead a ention as shown in Equation 7: where h is the number of headers. e in uence of di erent value of h will be studied in the ablation study in the next section. Moreover, to improve the generality of the model and make the model stable for training, we apply the dropout layer (abbreviated as dropout) and residual connections (abbreviated as RC). e dropout is proposed by [29] to address the problem of over ing. e residual connections is used to alleviate the gradients vanishing problem as it directly propagates the raw input of lower layer to higher layer and adds with the self-a ention [19] results. e RC is an essential part when we stacks multiple blocks of Transformer encoder. e ablation study in the next section will compare the performance when removing these two components separately. In total, the a ention layer can be formulated as Equation 8.
where LayerNorm is the standard normalization layer proposed by [3].
Position-wise Feed-Forward Network. e function of this position-wise Feed-Forward Network (FFN) is mainly to enhance the model with non-linearity and interactions between di erent dimensions of the input vectors. Similar to A ention module, we also use Dropout to avoid over ing and residual connection to enhance the power of the model by hierarchically learning. LayerNorm is Personalized Context-aware Re-ranking for E-commerce Recommender Systems SIGIR19, 2019, France normalization which is e ective on accelerating and stabilizing the training process. Finally, the output of the FFN layer can be calculated using Equation 9: (1) and b (2) are learnable parameters.
Stacking the Encoding Layer. Here we use a ention module followed by the position-wise FFN as a block of Transformer [32] encoder. A er encoding by one block, the output of encoding layer for item i (abbreviated as F (0) i ) has already contained the interaction between di erent items. We believe that taking F i contains more complex and high-order mutual information. In the next section, we study how many blocks is best for our task. e output of the encoding layer a er stacking N x blocks can be represented as Equation 10.
where N x is the stacking number of the encoding layer. Generally, N x is larger than 0 since it is obvious that the embedding vector would directly connect to the output layer when N x = 0. So the model reduces to a two-layer feed-forward network and the performance would greatly degrade.

Output Layer
e function of the output layer is mainly to generate a score for each item i = i 1 , . . . , i n (labeled as Score(i) in Figure 1 (b)). We use one linear layer followed by a so max layer. e output of so max layer is the probabilities of click for each item, which is labeled as P( i |X , PV ;θ ). We use P( i |X , PV ;θ ) as Score(i) to re-rank the items in O(1) time complexity. e formulation of Score(i) is shown in Equation 11.
where W F is learnable projection matrix, and b F is bias term. n is the number of items in the initial list.
In the training process, we use the click-through data as label and minimize the loss function shown in Equation 12. We update the parametersθ of the PCRM model by Adam optimizer [24].

Personalized Module
In this section, we introduce the approach to calculate the personalized matrix PV , which represents interactions between user and items. e straightforward approach is to learn PV with PCRM model in an end-to-end manner on the re-ranking loss. However, as explained in Section3, the re-ranking task is to re ne of the output of learning to rank approaches. e task-speci c representation learned on re-ranking task lacks users' global preference. erefore, we utilize a pre-trained neural network to produce user's personalized embeddings PV which are then used as features for PCRM model. e pre-trained neural network is learned from the whole click-through logs in e-commerce platform. Figure 1(c) shows the structure of pre-trained model in our paper. is sigmoid layer outputs the click probability on item i for user u (P( i |H u , u; θ )) given user's all behavior history (H u ). P( i |H u , u; θ ) is the predicted click probability. H u is the sequence of items clicked by the user u. e side information of user is also used as input, e.g., gender, age, purchasing level, etc. e loss of the model is calculated by a point-wise cross entropy function which is shown in Equation 13.
where D is the set of items displayed to user u within the whole clickthrough logs in e-commerce platform. θ is the parameter matrix of pre-trained model. i is the label (click or not) on item i. Inspired by the work [14], we employ the hidden vector before the sigmoid layer as the personalized vector pv i (in Figure 1(c)) that feeds into our PCRM model.

EXPERIMENTAL RESUTLS
In this section, we rst introduce the datasets and baselines used for evaluation. en we compare our methods with baselines on these datasets to evaluate the e ectiveness of our PCRM model. At the same time, the ablation study is conducted to help understand which part of our model contributes most to the overall performance.

Datasets
We evaluate our approach based on two datasets: Yahoo! Webscope v2.0 set 1 1 (abbreviated as Yahoo Letor dataset) and E-commerce Re-ranking dataset 2 . To the best of our knowledge, there is no publically available e-commerce re-ranking dataset with context information for recommendation. erefore, we construct E-commerce Reranking dataset from a popular e-commerce platform. e overview of two datasets are shown in Table 2.
Yahoo Letor dataset. We process the Yahoo Letor dataset to t for the ranking model of recommendation using the same method in Seq2Slate [4]. Firstly, we convert the ratings (0 to 4) to binary labels using a threshold T b . Secondly, we use a decay factor η to simulate the impression probabilities of items. All the documents in Yahoo Letor dataset are rated by the experts under the assumption that all documents for each query can be viewed by the users compeletely. However, in the real world recommendation scenario, items are viewed by the users in a top-down manner. As the screen of the mobile App can only show limit number of items, the bigger the ranked position of one item, the smaller probability of that this item can be viewed by the user. In this paper, we use 1/pos(i) η as the decay probability, where pos(i) is the ranking position of item i in the initial list and η is the hyper parameter. E-commerce Re-ranking dataset. e dataset contains a largescale records in form of click-through data from a real world recommendation system. Each record in the dataset contains a recommendation list for each user with users' basic information, click-through labels and features for ranking.

Baselines
Both learning to rank (LTR) and re-ranking methods can act as our baselines.
LTR. e LTR methods are used in two tasks. Firstly, the LTR methods can generate an initial list S r for the re-ranking model from a candidate set I r for each user request r . Secondly, the LTR methods which use pairwise or listwise loss function can act as re-ranking methods by taking the initial list S r as input and conducting the ranking algorithm for another time. e representative LTR methods used in this paper include: • SVMRank [22]: is is a representative learning to rank method which use the pairwise loss to model the scoring function. • LambdaMart [6]: is is a representative learning to rank method which use the listwise loss to model the scoring function. LambdaMart is the state-of-the-art LTR among those LTR methods equipped with the listwise loss function according to [34]'s evaluation. • DNN-based LTR: is is the learning to rank method which is deployed in our online recommender system. It use the standard Wide&Deep network structure [10] to model the scoring function via the pointwise loss function.
Re-ranking. As mentioned in the related work section, the existing re-ranking methods include DLCM [1], Seq2Slate [4] and GlobalRerank [41]. DLCM [1] and GlobalRerank [41] focus on reranking in information retrieval. Seq2Slate [4] focuses on re-ranking in both recommendation and information retrieval. In this paper, we only choose DLCM as baseline method. Seq2Slate and GlobalRerank are not chosen as baselines because they all use the decoder structure to generate the re-ranked list. Seq2Slate uses pointer network to generate re-ranked list sequentially. GlobalRerank uses RNN equipped with a ention mechanism as the decoder. e decoder structure outputs the item one by one. Whether an item is selected depends on the items which are chosen before it. As a consequence, both Seq2Slate and GlobalRerank can not be parallelized in online inference. e time complexity for Seq2Slate and GlobalRerank at interference phase is O(n) × RT , where n is the length of the initial list and RT is the time for a single ranking or re-ranking request. e latency for re-ranking by Seq2Slate and GlobalRerank is unacceptable because of the strict latency criterion for online recommender service.
• DLCM [1]: It is a re-ranking model used in information retrieval based on the initial list generated by LTR methods. e GRU is used to encode the local context information into a global vector. Combing the global vector and each feature vector, it learns a more powerful scoring function than the global ranking function of LTR.

Evaluation Metrics
For o ine evaluation, we use Precision and MAP to compare di erent methods. More speci cally, we use Precision@5, Precision@10 for precision and MAP@5, MAP@10 and MAP@30 for MAP. As the maximum length of initial list in our experiments is 30, MAP@30 represents total MAP and is denoted by MAP in this paper. e de nitions of the metrics are as follows.
Precision@k is de ned as the the fraction of clicked items in the top-k recommended items for all test samples, as shown in Equation 14.
where R is the set of all user requests in the test dataset. S r is the ordered list of items given by the re-ranking model for each request r ∈ R and S r (i) is the i-th item. 1 is the indicator function whether item i is clicked or not.
MAP@k is short for the mean average precision of all ranked lists cut o by k in the test dataset. It is de ned as follows.
For online A/B tests, we use PV, IPV, CTR and GMV as metrics. PV and IPV are de ned as the total number of items viewed and clicked by the users. CTR is the clickthrough rate and can be calculated by IPV/PV. GMV is the total amount of money (revenue) user spent on the recommended items.

Experimental Settings
Training Dataset. For both Yahoo Letor dataset and E-commerce Re-ranking dataset, the length of the initial list given by LTR is 30. When preprocessing the Yahoo Letor dataset for recommendation, the decay parameter η = 0.2 and threshold T b used to generate binary labels is 1.5. Same number of features are used by all the algorithms on each dataset to ensure the fairness of comparison.
Model. For both baselines and our PCRM model, we use the same value for those critical hyper parameters. e hidden dimensionality d model is set to 1024 for Yahoo Letor dataset and 64 for E-commerce Re-ranking dataset. e learning rate of Adam optimizer in our PCRM model is the same with [32]. Negative log likelihood loss function is used as shown in Equation 12. p dropout is set to 0.1. e batch size is set to 256 for Yahoo Letor dataset and 512 for E-commerce Re-ranking dataset. We use 4 blocks of transformer with 3 multi-heads. We also tried di erent number of blocks and multi-heads to see the performance variances by ablation study. e rest se ings belonging to the customized parts of our model will be listed at the corresponding parts in the evaluation section.

O line Experiments
In this section, we rst conduct o ine evaluations on Yahoo Letor dataset and E-commerce Re-ranking dataset. en we show the results of online A/B tests. We also conduct the ablation study to help nding which part of our PCRM model contributes most to the performance.

O line Evaluation on Yahoo Letor dataset.
In this section, we conduct evaluation on Yahoo Letor dataset to discuss the following questions: • Q0: Does our PCRM model outperform the state-of-the-art methods and why? • Q1: Does the performance vary according to initial lists generated by di erent LTR approaches?
e evaluation results are shown in Table 3. We compare the baselines and our PCRM-BASE model based on two di erent initial lists which are generated by LambdaMART and SVMRank respectively. PCRM-BASE is the variant of our PCRM model without the personalized module. Note that Yahoo Letor dataset does not contain user-related information, thus we only conduct PCRM-BASE for comparison. SVMRank and LambdaMart are also used for reranking. For SVMRank, we use the implementation in [22]. For LambdaMart, we use the implementation from RankLib 3 . Table 3 shows that our PCRM-BASE achieves stable and signi cant performance improvements comparing with all baselines. When based on the initial list generated by SVMRank, PCRM-BASE outperforms DLCM by 1.7% at MAP and 1.6% at Precision@5. e gap gets larger when comparing with SVMRank which has 5.6% increase at MAP and 5.7% increase at Precision@5. When based n the initial list generated by LambdaMART, PCRM-BASE outperforms DLCM by 1.1% at MAP and 0.2% at Precision@5. PCRM-BASE also achieves 3.1% improvements on MAP and 1.3% improvements on Precision@5 comparing with LambdaMART .
PCRM-BASE uses the same training data as DLCM and does not contain the personalized module. e performance gain over DLCM mainly comes from the powerful encoding ability of Transformer. Multi-head self-a ention mechanism is more good at modeling mutual in uence between two vectors, especially when the length of encoding list gets longer [23]. In our model, the a ention mechanism can model the interactions of any item-pairs in O(1) encoding distance.
As PCRM-BASE uses the Transformer-like structure, there are many sub-modules which may contribute to the performance gain. We conduct the ablation study to help us understand which subdesign helps the most to beat the baselines. e ablation study is conducted on the initial list generated by SVMRank. Similar results were found when using the initial list generated by LambdaMART and we omits the results in this paper as space is limited. Table 4 show the results of ablation in three parts: e rst part ( rst row) shows the performance of the baseline DLCM. e second part (second row) "Default" is the best performance of our PCRM-BASE model. e third part (the rest rows) shows di erent ablation variants of our PCRM model which include: remove position embedding (PE), remove residual connection (RC), remove dropout layer, use di erent number of blocks and use di erent number of 3 h ps://sourceforge.net/p/lemur/wiki/RankLib/ heads in multi-head a ention. Note that we set b=4 and h=3 in our "Default" PCRM model.
As shown in Table 4, the performance of our model degrades greatly a er removing position embedding. is con rms the importance of sequential information given by the initial list. By removing the position embedding, our model learns the scoring function from the candidate set instead of an ordered list. Note that even without position embedding, our PCRM-BASE still achieve comparable performance with DLCM, which further con rms that our PCRM-BASE model can encode the initial list more e ectively than DLCM.
the MAP of our model slightly decreases by 0.1% and 0.7% respectively when removing residual connections and dropout layer, which indicates that our model is less severe to the problems such as gradients vanishing and over ing.
e performance of our model rst increases with the number of blocks (1-¿2-¿4) and decreases a erwards (4-¿6-¿8), as over ing happens when we stack 8 encoding blocks together.
We also tried di erent se ings(h = 1, 2, 3, 4) in the multi-head a ention layer. No signi cant improvements are observed in Table 4, which is di erent from the conclusions derived from NLP tasks [30]. e experiments in NLP show that when using more heads in multi-head a ention mechanism, it is usually helpful since more information can be captured for the following reasons. (1) From Equation 7 we nd that the function of each head is playing a role of mapping the original feature vector into a di erent subspace.
us using more heads, we can model more interactions of items in di erent sub-spaces. (2) [30] indicates that using more heads is helpful in encoding the information of long sequence. is is resonable because the output vector for a certain item is the weighted sum of all item vectors in the list. When the sequence becomes longer, each item in the list contributes less to the output vector. However, in our re-ranking se ings, all the items in the initial list are highly homogenous and the average length of the initial list is 18. ere are minor improvements when mapping the original feature vector into di erent subspaces. Besides, our sequence is not long enough to use multiple heads. As a consequence, we suggest to use only one head to save computation costs because the performance improvements are not obvious.

O line
Evaluation on E-commerce Re-ranking dataset. We conduct the o ine evaluation on E-commerce Re-ranking dataset to answer the following question.
• Q2: What is performance of our PCRM model equipped with personalized module?
e evaluation results are shown in Table 5. We compare different variant PCRM model with DLCM baseline. For our PCRM models, we not only Waeluatethe performance of PCRM-BASE, but also evaluate the performance of the variant of model equipped with the pre-trained personalized vector PV , which is labelled as PCRM-Personalized-Pretrain. do not compare wevith SVMRank and LambdaMART because the public released implementations can not handle our large dataset and out-of-memory error happens. As our previous evaluation on Yahoo Letor dataset already con rms that DLCM achieve be er performance in all metrics and [1] has consistent results, we omit the comparison with SVMRank and LambdaMART on our E-commerce Re-ranking dataset. e initial  list is generated by a DNN-based LTR method which is deployed in our real world recommender system. All the re-ranking methods listed in Table 5 use the same training dataset, which is also the same with DNN-based LTR method. e rst row in Table 5 shows the performance of the DNN-based LTR without any re-ranking methods. e numbers in the brackets are the related improvement compared with DNN-based LTR without re-ranking. Table 5 shows consistent results with Table 3 when comparing with PCRM-BASE and DLCM. Our PCRM-BASE outperforms DLCM by 2.34% at MAP and 4.1% at Precision@5. Recall that on Yahoo Letor dataset, PCRM-BASE achieves 1.7% improvements on MAP and 1.6% improvements on Precision@5. e performance gain on our E-commerce Re-ranking dataset is much larger than on Yahoo Letor dataset. is is highly related with the properties of Yahoo Letor dataset. As mentioned in the previous section, in order to simulate the training data in recommendation, we convert the ratings ({0,1,2,3,4}) into binary label (relevant and irrelevant) to simulate click-through data. Our statistics of the converted Yahoo Letor dataset show that the average click through rate is 30%, which mean that for each query with 30 recommended documents, about 9 documents are clicked by the users. However, the average clickthrough rate in our real world E-commerce Re-ranking dataset is no more than 5%. It means that ranking on Yahoo Letor dataset is much easier than on E-commerce Re-ranking dataset. is is also con rmed by the value of MAP for the same ranking methods on two datasets: DLCM can achieve 0.64 MAP on Yahoo Letor dataset but can only achieve 0.28 MAP on E-commerce Re-ranking dataset. Combining Table 5 and Table 3, we nd that the harder the ranking task, the larger improvements of our model. Table 5 shows that our PCRM-Personalized-Pretrain achieves signi cant performance improvements comparing with PCRM-BASE. PCRM-Personalized-Pretrain outperforms PCRM-BASE by 4.5% at MAP and 6.8% at Precision@5. is is mainly imported by the global context information PV , which is learned by a pre-trained model whose architecture is illustrated in Figure 1 (c). PCRM-Personalized-Pretrain has three strengths: (1) e pre-trained model can use longer period of logs which is helpful in learning long-term user interest. (2) e pre-trained model can use logs from the whole e-commerce platform, which is not limited in the recommendation logs. (3) e personalized embeddings learned by pre-trained is more generic to represent user's preference on items.

Online Experiments
We also conduct online A/B tests at a real world e-commerce recommender system on online metrics which includes PV, IPV, CTR and GMV. e meaning of these metrics is explained in the previous "Evaluation Metrics" section. ese metrics evaluate how much willingness for users to view (PV), click (IPV, CTR) and purchase (GMV) in a recommender system. e online A/B tests last for one day. For each algorithm, there are hundred thousand users and millions of requests for online test. Table 6 shows the relative improvements of three methods to an online base ranker. Firstly, the online A/B test shows that using re-ranking helps increasing the online metrics no ma er what kinds of re-ranking methods are. Again, we can conclude that re-ranking helps improving the performance by considering the local context information. It is note that 0.7% increase on PV is signi cant in our large-scale online system because it means that about billions of extra items is viewed by the users a er using the re-ranking method. Secondly, we can conclude that our PCRM-BASE model brings an extra 0.5% increases on viewed items and extra 2.44% increases on clicked items and extra 0.23% increases on the total GMV. Lastly, by using the personalized module, our PCRM-Personalized-Pretrain can further improve the GMV by 6.31%. Recall that in o ine experiments on E-commerce Re-ranking dataset, PCRM-Personalized-Pretrain has 4.5% extra increases compared with PCRM-BASE. e result shows that personalization with pre-trained user's representation can a ect the interaction of item-item, and bring signi cant gain for re-ranking.

Visualizing Attention Weights
We visualize the a ention weights learned by our model to answer the following question.
• Q3: Can self-a ention learn meaningful information in respect to di erent aspects, for example, positions and characteristics of items?
Attention on Characteristics. We rst visualize the average a ention weights between items on two characteristics: category and price. e results caculated on the test dataset are shown in Figure 2. Each block in the heatmap represents the average a ention weights between items belonging to seven main cateogries. e darker the block, the larger the weight. From Figure 2(a) we can conclude that the a ention mechanism can successfully capture mutual-in uence in di erent categories. e items with similar categories tend to have larger a ention weights, indicating larger mutual in uences. For example, "men's shoes" has more in uences on "women's shoes" than on "computer". It is also easy to understand that "computer", "mobile phone" and "home appliance" have large a ention weights with each other because they are all electronics. Similar cases can be observed in Figure 2(b). In Figure 2(b), we classify the items into 7 levels according to their prices. e closer price between items, the larger the mutual in uences.
Attention on Positions. e visualization of average a ention weights on positions in the initial list is shown in Figure 3. Firstly, Figure 3(a) showed the self-a ention mechanism in our model can capture the mutual in uences regardless of the encoding distances as well as the position bias in recommendation list. Items ranked ahead of the list usually is more likely to be clicked and thus have more in uences on those items at the tail of the list. For example, we observe that items at the rst position have larger impacts on items at 30th position than those items at 26th position even though      the la er is more closer to it. e e ect of position embeddings is also obvious compared with the Figure 3(b), whose a ention weights between each position are more uniformally distributed.

CONCLUSION AND FUTURE WORK
In this paper, we proposed a personalized context-aware re-ranking model (PCRM) to re ne the initial list given by state-of-the-art learning to rank methods. e introduction of personalized information into the re-ranking model can bring signi cant performance improvements. In the re-ranking model, we used Transformer network to encode both the dependencies among items and the interactions between the user and items. Both the online and o ine experiments demonstrated that our PCRM model can greatly improve the ranking performance on either public benchmark dataset or our released dataset. In addition, the new released dataset can help researchers to optimize the ranking algorithms for e-commerce recommendation.
Our work explicitly models the complex item-item relationships in feature space. We believe that optimization in label space can also helps. e pairwise or listwise loss function aims to dig more information in label space. However, limited improvements are achieved in our experiments for re-ranking. e labels of clickthrough data in the paper is click or not. It is interesting to construct more partial ordering relations (e.g. order by stay time) in clickthrough data. Another future direction is learning to diversify by re-ranking. Even thougth our model does not hurt the ranking diversity in practice. It is worth to try to introduce the goal of diversi cation into our re-ranking model. We will further explore this direction in the future work.


Gray linear regression model based on adaptive particle swarm optimization power load forecasting method

Aiming at the trend of “S” growth in annual electricity consumption in Ningxia, the gray linear regression combined model (GLM) load forecasting method is adopted, which can improve the lack of exponential growth trend in linear regression model and the lack of linear factors in the GM(1,1) model. However, the traditional GLM model has theoretical defects in the process of solving parameters, and there are limitations in the application range. Therefore, this paper introduces an adaptive particle swarm optimization algorithm which is more efficient than the standard particle swarm optimization algorithm, and combines it with the gray linear regression combination model. The adaptive particle swarm optimization algorithm is used to solve the parameters of the GLM model, then carry out residual correction. Adaptive particle swarm optimization gray linear regression combination model (APSO-GLM) is proposed. The example analysis shows that the model overcomes the defects of the GM(1,1) model and the linear regression model. Compared with the single model, the prediction accuracy is higher and it has a wider application range than the traditional GLM model.

Introduction
Load forecasting is an important part of power system economic dispatch and an important module of an energy management system (EMS).Since the load forecast is based on the past and present of the power load to speculate its future value. Therefore, the object of the load forecasting work is the uncertainty event [1]. Methods of load forecasting are mainly divided into classical forecasting methods and modern forecasting methods [2][3][4][5]. Electric measurement is one of the important tasks of the power sector. Accurate load forecasting can be economically and reasonably arrange the start and stop of the internal generator set of the power grid, maintain the safety and stability of the power grid operation, reduce the unnecessary rotating reserve capacity, and arrange the unit maintenance plan reasonably, guarantee the normal production and life of society, effectively reduce the cost of power generation, and improve the economic and social benefits [6].
As the national economy enters the "new normal", as well as the adjustment of relevant national policies and the advancement of power system reform, many social and economic indicators in Ningxia are no longer exponentially growing, but have certain "S"-shaped curve characteristics [7]. At the same time, the factors affecting load changes are also complicated. These factors have strong uncertainties. It is not easy to find out all the factors. Therefore, the power load can be regarded as a gray system [8]. The gray prediction method requires less data and higher prediction accuracy [9], but the gray model is mainly applicable to a single exponential growth data sequence. The principle of linear regression prediction [10] is the continuity of things development and the correlation of things IOP Conf. Series: Earth and Environmental Science 218 (2019) 012144 IOP Publishing doi:10.1088/1755-1315/218/1/012144 2 causality. Considering the future as the current continuation, under the condition that the various conditions are relatively stable, the future development will be predicted, and the short-term forecast can achieve better results, but the long-term forecast is often not effective. In view of this, in order to predict the annual electricity consumption of Ningxia region with both linear trend and exponential trend, we adopts the gray linear regression combination model [11] constructed by combining the gray model GM (1,1) and the linear regression model in this paper. It can improve the lack of exponential growth trend in the linear regression model and the lack of linear factors in the GM(1,1) model, but the traditional GLM model has theoretical defects in the process of solving parameters.
Particle swarm optimization algorithm (PSO) is a cluster intelligent optimization algorithm developed in recent years. It has many advantages such as fewer parameters, faster convergence, and global search capability. It has been widely used in various fields [12]. In this paper, a more efficient adaptive particle swarm optimization algorithm is introduced. Combined with the GLM model, APSO-GLM is proposed. Then the residual correction is performed to further improve the prediction accuracy. Theoretical research and practice show that the proposed model has higher precision than a single prediction model and traditional GLM, can enhance the stability of prediction, and has higher ability to adapt to future environmental changes.

Gray linear regression combined model
(1) Generate a sequence of first order accumulation with 1-AGO Then: is the GLM model. Parameters v, C 1 , C 2 and C 3 are undetermined. In order to obtain the above parameters, set the parameter sequence ) 1 , Considering that m takes different values, the final fitting parameters v m (t) will also be different, so we can take m=1,2,…,n-3, v m (t) is calculated separately, and the mean value of all the obtained v m (t) is taken as the fitting value of the identified parameter v.
The estimated values of C 1 , C 2 and C 3 can be obtained by using the least square method The predicted value of the generated sequence is The prediction data can be obtained by reduction and reduction. )

Adaptive particle swarm optimization algorithm
The particle swarm optimization algorithm is a global optimization evolution algorithm proposed by Kennedy and Eberhart, which are derived from the simulation of bird predation behavior [13]. The specific process is as follows: Suppose that in a D-dimension target search space, there are N particles forming a community, where the ith particle position and velocity are respectively The optimal position so far searched by the first particle is called individual extremum, and the optimal position so far searched by the whole particles swarm is called global extremum, respectively ) , , ( (12) When these two extremums are found, the particles update their speed and position according to the following formula: Where w is the inertia factor, c 1 and c 2 are learning factors, usually c 1 =c 2 =2, r 1 and r 2 are uniform random numbers in the range [0,1], v id is the velocity of the particle. In the standard particle swarm algorithm, the inertia factor w is generally set to 1. Since the inertia factor is a variable that affects the current particle velocity, a larger value is advantageous for global search, and a smaller value is advantageous for local search, in order to better balance the algorithm. This paper proposes a linearly decreasing inertia weighting factor, as shown by equation (16).
Where, w max is the inertia weight maximum, generally set to 0.9; w min is the inertia weight minimum, generally set to 0.4; k is the current iteration number; k max is the maximum number of iterations.

Gray linear regression combined model based on APSO optimization
The use of GLM model for power load forecasting requires less load data, and does not need to consider the load distribution rule and trend. However, when the model predicts a faster-growing load, the prediction accuracy deteriorates. The reason is that in the process of solving the parameter v, the average value of v m at m different values is simply taken.
In order to avoid errors caused by improper parameter values, this paper uses the APSO algorithm to solve the parameters v, C 1 , C 2 , C 3 , so that accurate prediction can be made even when the load increases rapidly. According to the basic steps of the APSO model described above, set particle is 4dimension, and its position is [v, C 1 , C 2 , C 3 ]. The APSO algorithm is used to solve the parameters v, C 1 , C 2 , and C 3 . The basic steps are as follows: Step 1: Initialization: The initial position and velocity of particles are set according to formulas (9) and (10).
Step 2: The fitness value of each particle is calculated: The individual optimal position of the particle to the current position P best and the global optimal position g best are set to the position of the best particle in the initial particle population, and calculate the adaptation of each particle through the objective function f. The objective function f(Y i ) is Where, x (0) (t) and x (0) (t) are the kth actual and predicted values of the load respectively.
Step 3: Determine whether the convergence criterion of the algorithm is satisfied. If yes, execute step 6, end the iteration, and output the values of C 1 , C 2 , C 3 , and v; otherwise, execute step 4.
Step 4: Use the formulas (13) and (14) to update the velocity and position of each particle. If the particle fitness is better than the corresponding fitness of g best , g best is set to the new position.
Step 5: Determine whether convergence criterion of the algorithm is satisfied. If yes, execute step 6, and output values of C 1 , C 2 , C 3 , and v; otherwise, return to step 4 to iterate and continue to optimize.
Step 6: Output the global optimal position g best , and obtain the global optimal solution of the gray linear regression combined model parameters C 1 , C 2 , C 3 and v, and the algorithm runs.

Case analysis
In order to verify the validity and practicability of APSO-GLM proposed, annual electricity consumption in Ningxia region from 2011 to 2017 was used as sample data, and the original data is shown in table 1. It can be seen from table 1 that annual electricity consumption in Ningxia region shows an increasing trend, but there is obvious volatility. Linear regression model, GM(1,1) model, GLM model and APSO-GLM model proposed in this paper are used for fitting, and the model with the highest fitting precision is used to predict the annual electricity consumption from 2018-2020.

Gray model prediction
The original data is accumulated and generated immediately and the mean is generated. The accumulated gray model is an equation of the form x (1) (t)=ae vt +b. The equation is as follows by using MATLAB for curve fitting: = t x

Grey linear regression combined model prediction
According to formula (1) ~ (6) and MATLAB, the calculated value of v is: v=0.03; thus, the gray linear regression combination model is: (19) According to formula (7), MATALB is used to obtain the values of parameter C as C 1 =38929.08， C 2 = -443.33，C 3 = -38940.51 The gray linear regression combination model is obtained 51 . = t x

Comparative analysis of forecasting models
The predicted and actual values of the four models are fitted, and the fit is shown in the Figure 1. The prediction accuracy of the four models is tested. The error test of the model is shown in Table 2.  Through the comparison of the results of the above average relative error and the fitting of the actual value and the predicted value, it can be seen that although the four models can be used for power load forecasting, the APSO-GLM model proposed in this paper is more accuracy than the traditional GLM model, linear regression model and GM (1,1). Based on the effective synthesis of the GM(1,1) model and the linear regression model, the prediction results are optimized. The local large deviations that may occur in the prediction process of the single prediction model are avoided, and the prediction accuracy is improved.

Conclusion
This article has the following three main tasks: a. Aiming at the limitations of a single theoretical method, a GLM model that combines linear and exponential information and does not require typical distribution rules is proposed. It is superior to single gray model and linear regression model in prediction accuracy, improves the accuracy of the model, reduces the prediction error, and makes the prediction effect better.
b. In view of some theoretical problems of the traditional GLM model in solving the parameters, this paper uses the APSO algorithm to replace the traditional least squares method to solve the parameters of the GLM model, and proposes an APSO-GLM prediction model, which overcomes the limitations of the GLM model, has certain theoretical significance and application value.
c. The APSO-GLM model proposed is applied to power system load forecasting, and an experiment verifies that the model has certain feasibility and high prediction accuracy.


extfieldsofstudy:
Computer Science
Medicine

text:
TAEO-A Thermal Aware & Energy Optimized Routing Protocol for Wireless Body Area Networks

Wireless Body Area Networks (WBANs) are in the spotlight of researchers and engineering industries due to many applications. Remote health monitoring for general as well as military purposes where tiny sensors are attached or implanted inside the skin of the body to sense the required attribute is particularly prominent. To seamlessly accomplish this procedure, there are various challenges, out of which temperature control to reduce thermal effects and optimum power consumption to reduce energy wastage are placed at the highest priority. Regular and consistent operation of a sensor node for a long-time result in a rising of the temperature of respective tissues, where it is attached or implanted. This temperature rise has harmful effects on human tissues, which may lead to the tissue damage. In this paper, a Temperate Aware and Energy Optimized (TAEO) routing protocol is proposed that not only deals with the thermal aspects and hot spot problem, but also extends the stability and lifetime of a network. Analytical simulations are conducted, and the results depict better performance in terms of the network lifetime, throughput, energy preservation, and temperature control with respect to state of the art WBAN protocols.

Introduction
Advancement in Information and Communication Technology (ICT) has revolutionized all domains of applied sciences. One such domain that has gained much attention of the research arena, as well as engineering industries, over last two decades is Smart Health using Wireless Body Area Networks (WBANs). WBAN has a potential to solve social problems, like efficient use of limited medical infrastructure with respect to population, raising aging population, and over wrought facilities. Hence, there is a dire need for the prevailing conditions to attract general public towards WBAN technology to address the aforementioned problems.
Recent trends and enhancements in communication and sensory technology have led to the further optimization of WBAN. WBAN is a type of Sensor Network, which has numerous applications both for medical and non-medical purposes [1,2], like in defense activities, by physicians for remote health monitoring, for coaches and players in sports, etc. The generic purpose of WBAN is the automation of certain processes for the ease of people and, consequently, bringing improvements in their lifestyle. Therefore, the technological significance of WBAN is eminent from its day-to-day applications.
WBAN constitutes a set of miniature sensors, like Electrocardiogram (ECG) and Electromyography (EMG), which are implanted inside the human body or placed outside the skin; Figure 1 shows a basic WBAN architecture. These nodes are integrated into the exchange of information. Each sensor collects the associated data from its surrounding and processes it for onward submission to the intended medical rep or physician. The adoption of technology always imposes certain challenges; similar is the case with WBAN. Aside from its proved assistance to both patients and doctors, some aspects of WBAN need to be controlled for its practical implementation.
In addition to the saved energy through intelligent design of electric circuits of wireless sensor nodes, energy efficient routing plays a vital role in preserving the energy of these nodes. Moreover, another vital problem in such networks is the thermal dissipation of nodes, which is harmful for human tissues. In WBAN, a node is declared as a hot-spot node when its temperature is raised to an extent where it can damage human tissues. This is a safety hazard and it is not acceptable in WBAN implementation. Therefore, the temperature of a hot-spot node needs to be intelligently managed for a hazard-free deployment of WBAN. There are various factors that cause the temperature rise of the node. Two primary sources [3] are: • EM transmission and • power consumption within internal circuitry. WBAN constitutes a set of miniature sensors, like Electrocardiogram (ECG) and Electromyography (EMG), which are implanted inside the human body or placed outside the skin; Figure 1 shows a basic WBAN architecture. These nodes are integrated into the exchange of information. Each sensor collects the associated data from its surrounding and processes it for onward submission to the intended medical rep or physician. The adoption of technology always imposes certain challenges; similar is the case with WBAN. Aside from its proved assistance to both patients and doctors, some aspects of WBAN need to be controlled for its practical implementation.
In addition to the saved energy through intelligent design of electric circuits of wireless sensor nodes, energy efficient routing plays a vital role in preserving the energy of these nodes. Moreover, another vital problem in such networks is the thermal dissipation of nodes, which is harmful for human tissues. In WBAN, a node is declared as a hot-spot node when its temperature is raised to an extent where it can damage human tissues. This is a safety hazard and it is not acceptable in WBAN implementation. Therefore, the temperature of a hot-spot node needs to be intelligently managed for a hazard-free deployment of WBAN. There are various factors that cause the temperature rise of the node. Two primary sources [3] are: • EM transmission and • power consumption within internal circuitry.

Figure 1.
Wireless Body Area Networks (WBAN) basic architecture depicting deployment of sensors for gathering patient's data and submitting it to a remote server via the internet.

Problem Statement and Contribution
In WBAN, a node communicates with its neighbors or sink using radio frequency (RF) signals. A transceiver in a sensor device is responsible for the communication of data through electromagnetic waves. This radio frequency transmission dissipates heat, which is absorbed by tissues, and consequently raises the temperature. The second factor causing temperature rise is the internal circuitry of a sensor node that generates heat while processing the collected data. The amount of heat that is generated by a node circuitry is dependent on its architecture. There is a need to control a node from dissipating excessive heat. Therefore, heat that is generated by the sensor nodes needs to be controlled for effective implementation of WBAN. An increase in the thermal dissipation of a sensor node due to the transmission of packets is one core safety issue in WBAN [4]. This is the major theme of the proposed protocol i.e. to keep the heated node away from the transmission process until its temperature revamped in order to provide a safe WBAN system for the health monitoring of patients. As discussed earlier, heat dissipation mainly depends on electromagnetic communication and the internal circuitry of a node. Therefore, this issue needs to be intelligently addressed in a practical implementation of WBAN technology. Otherwise, an increase in the temperature of a sensor node to

Problem Statement and Contribution
In WBAN, a node communicates with its neighbors or sink using radio frequency (RF) signals. A transceiver in a sensor device is responsible for the communication of data through electromagnetic waves. This radio frequency transmission dissipates heat, which is absorbed by tissues, and consequently raises the temperature. The second factor causing temperature rise is the internal circuitry of a sensor node that generates heat while processing the collected data. The amount of heat that is generated by a node circuitry is dependent on its architecture. There is a need to control a node from dissipating excessive heat. Therefore, heat that is generated by the sensor nodes needs to be controlled for effective implementation of WBAN. An increase in the thermal dissipation of a sensor node due to the transmission of packets is one core safety issue in WBAN [4]. This is the major theme of the proposed protocol i.e. to keep the heated node away from the transmission process until its temperature revamped in order to provide a safe WBAN system for the health monitoring of patients. As discussed earlier, heat dissipation mainly depends on electromagnetic communication and the internal circuitry of a node. Therefore, this issue needs to be intelligently addressed in a practical implementation of WBAN technology. Otherwise, an increase in the temperature of a sensor node to a menacing level results in the damage of human tissues or DNA. In WBAN, this is also known as hot-spot node problem.
In this work, a novel routing protocol for WBAN is proposed, which provides a solution to the following two problems: • effective detection and avoidance of a hot-spot node based on Specific Absorption Rate (SAR) and • enhancement in stability period and network lifetime.
The proposed routing protocol has out classed two state-of-the-art routing protocols in terms of stability period, throughput and residual energy. Moreover, an analysis of the ratio of heated and normal nodes is also conducted.
Rest of the paper is organized, as follows: Section 2 discusses the related work. Section 3 presents the system model. The proposed methodology is described in Section 4. Section 5 presents the results and discussion. Finally, Section 6 concludes the paper.

Related Work
The researchers have done immense work to bring the continual advancement of WBAN technology. This section explains the notable work of authors on WBAN.
In [3], the authors proposed the Thermal Aware Routing Algorithm (TARA), which is a pioneer protocol in the thermal-aware WBAN. The work before TARA did not encompass the effects of temperature rise on the human body. RF communication and internal circuitry of the sensor are two major sources of heat generation. On the base of these sources, the hot-spots are identified. Data sensing, processing, and transmitting activate ultimately increase the temperature of a node. In TARA, a node monitors the transmission of packets by its neighbors. On this base of this data, it estimates the temperature rise of a node. When the estimated temperature reaches a threshold, the node is marked as a hot-spot. The traffic towards the hot-Spot node is withdrawn and then returned to the previous node in situations where all of the neighbors are hot-spot. The packet withdrawal strategy adds up the overhead of rerouting and leads to overall transmission delay. Power consumption is also increased due to additional rerouting.
In [5], Takahashi proposed the Least Temperature Rise (LTR) scheme to address the shortcomings of TARA. The idea of the coolest neighbor is introduced, i.e. lowest temperature node is opted as a data forwarder. The protocol implemented a counter for the number of hops to limit the stray packet and to avoid looping. When the limit is reached, the packet is eliminated or dropped from the network. A list of recently visited nodes is maintained to optimize the node selection process. If the coolest node is in that list, then the node with the second lowest temperature is selected. As LTR is based on a greedy approach, the packets undergo longer delays to reach destination, thus undermining the reliability of communication. In the same paper, the author proposed ALTR, which is an improved version of LTR. In this scheme, the Adaptive Threshold is used and, when the hop count limit is reached, the packet is not dropped or discarded, but it is forwarded to the destination using the shortest-path algorithm. The protocol successfully reduced the transmission delay. However, when a shortest-path algorithm is applied, the protocol allows the traffic through a hot-spot node.
In the same paper [5], the Least Total Route Temperature (LTRT) protocol works by pre-identification of routes from source to destination, and thereby selecting the path with the least total route. In LTRT, the temperature of a sensor is treated as its weight and the total route from source to destination is calculated and the minimum cost total route is opted for communication.
HPR [6] uses the Shortest-Hop algorithm for transmission of packets to the destination. It works by defining a temperature threshold and allowing transmission if the nodes temperature is below the specified limit. If a nodes temperature surpasses the threshold, then the packets are rerouted to a neighbor with the lowest temperature. The exchange of information related to the node's heat in between nodes for hot-spot detection is a major overhead. The Thermal-Aware Shortest Hop Routing (TSHR) scheme for in-vivo implementation of WBAN is presented in [7]. Tabandeh and Jahed presented a novel architecture with a two-phase implementation approach. The configuration of nodes and the exchange of necessary information are managed in the first phase. Whereas, the second phase implements the routing scheme and the packets are forwarded on the path with a minimum number of hops to the destination. In TSHR, two temperature thresholds are specified. The first limit is for maximum acceptable temperature by a node. The second limit is dynamic and established on the base of a neighbor's temperature. Afterward, the node inspects and, if neighbor's temperature exceeds the preset dynamic value, it is declared as a hot spot.
In [8], Mobile-ATTEMPT (M-ATTEMPT) is proposed, which is an extension of ATTEMPT routing scheme. M-ATTEMPT is a thermal-aware WBAN protocol with efficient mobility support. It works by defining the zones with levels, depicting a parent-child relation. When a node disconnects and enters another zone, it requests the respective parent node for becoming a member. The parent node only accepts the request if the child node count is less than three.
The authors in [9] proposed a novel routing scheme i.e., Energy Efficient Thermal and Power Aware (ETPA), which is focused to provide a thermal-aware and low power consumption implementation of WBAN. Routes are defined on the base of communication expense, which is calculated when considering the energy level, temperature, and signal strength of the neighbors. ETPA works by establishing a four-frame cycle, in which every sensor exchanges its current heat-value and energy level with all other nodes. Afterward, each node in the cycle calculates the transmission cost based on the signal power of the received signal from an adjacent node. The low-cost route is opted for transmission. ETPA has managed the temperature of a sensor by diffusing it amongst other nodes, and thus results in improved lifetime due to lowered energy depletion.
In [10], the authors apprised that the increase or decrease in Specific Absorption Rate (SAR) value primarily depends on the strength of the surrounding electromagnetic field. Other parameters, such as the characteristics of tissue, its dimensions, including size, frequency, and exposure duration to radiations forms the basis of good SAR estimates. SAR estimates and the associated parameters differ, depending on the type of tissue, like brain, muscles, or nerve.
The authors of [11] performed a lot of experimental work to identify safe SAR boundaries. There work proved to be highly beneficial in the development of safe wireless systems. SAR values below 0.02 W/kg are proven to be safe. Experiments that were performed using high SAR values illustrate that the 1.0-1.2 W/kg SAR range has shown the disparaging impact on the chromosome, and consequently damaged the DNA. In [12], Bag and Bissiouni established an effective routing scheme for WBAN to cater to heated node avoidance and successfully reduced the average delay. The transmission path is determined on the base of a number of hops and least hop count route is selected. Rerouting is required in the case of a hot-spot in the finalized route.
The authors in [4] proposed a novel routing model comprising of upper and lower temperature thresholds. When the node temperature attains the lower threshold, the traffic is controlled in order to decrease the rate of temperature rise. Upon the surpassing of the upper limit, the node is marked as a heated node and all of its traffic is re-routed to avoid a further rise in temperature.
In [13], a WBAN protocol that is based on single-hop and multi-hop implementation for critical and non-critical transmission is presented. It works by selecting a data aggregator and forwarder on the base of a cost function. The minimum cost node is opted to perform the necessary transmission to directly linked node or to the sink. The result shows the effectiveness of the proposed scheme in comparison to M-ATTEMPT protocol.
In [14], Monowar and Bajaber developed a routing scheme that focused on the traffic rate of a sensor node. Increased transmission rate elevates the chance of congestion in the network and, consequently, disrupts the Quality of Service (QoS) requirements necessary for a reliable system. This high rate of traffic and network congestion gives rise to the temperature of a node. Congestion is estimated by using queue occupancy, whereas the traffic intensity also provides vital information to measure the level of congestion. Authors in [15] proposed a routing scheme focused on efficient and reliable delivery of critical data in the specified duration. The protocol works by segregating critical from non-critical data and it addresses the temperature management issue.
In [16], the authors proposed RE-ATTEMPT an extension of M-ATTEMPT protocol to overcome its shortcomings. The M 2 E 2 protocol proposed in [17] is focused on low energy consumption and improved network lifetime. The protocol works by selecting parent nodes that are sensors with comparatively high data rate placed closer to the sink. The first level and second level child nodes are identified, depending on the hierarchy in which they are linked or transmits data. The protocol works in two modes. When at home, a node on the human body is connected to the Home nodes that perform the required routing using single-hop communication. In case of non-availability of Home-Signal, the routing tables are maintained by nodes that were implanted on the human body and use multi-hop for the transmission of packets.
To solve thermal problem of WBAN, K.S.Kathe and U.A.Deshpande proposed a thermal aware routing algorithm [18]. In the proposed algorithm, the packets are transmitted on the basis of assigned priority. The objectives of this work are to maintain the temperature of the sensor nodes, reduce the packet delay, and increase packet arrival ratio.
An Adaptive Thermal-Aware Routing (ATAR) protocol is proposed in [19] on the basis of adaptive route diversion, avoiding the heated nodes eliminating the chances of the hotspot node. The multi ring routing approach is applied in this protocol that finds an alternate optimum route.
In [20], a secondary base station is selected that helps to reduce the temperature of the neighbor nodes of master base station, as these neighbor nodes will not be a part of the new data routes from the sensing odes to the base station. In this time, the sensor node will only transmit the priority packets to the secondary base stations. Accordingly, the network load reduces by this way, and at this time the nodes have the opportunity to cool down. However, this protocol only works in the network of low and intermediate traffic and it increases the delay during high traffic. Table 1 presents the summary of commendable work that was conducted in previous years when considering thermal and energy aware routing for WBANs.

System Model
In the following section, the system model of the proposed routing protocol is presented.

Radio Model
In WBAN, the radio frequency model for packet transmission plays a pivotal role and it needs to be intelligently selected when considering the design requirements. TAEO is based on the first order electromagnetic transmission scheme [21]. Equations (1)-(3) estimate the power consumed by a sensor during transmission (transmit or receive). P Tx (N, l) = P Circuit−Tx (N) + P Tx−Amp (N, l) P Tx (N, l) = P Circuit−Tx .N.l 2 (1) where, l is the distance between transmitter and receiver sensor. l 2 indicates the power that was consumed by the transmission channel and P Tx is the energy utilized to send a packet from transmitter to receiver node. It is sum of the power that is consumed by the internal circuit P Circuit−Tx and signal amplifier P Tx−Amp . P Rx shows the power consumption of the sensor on the receiving side, and it depends on the energy that is utilized by internal circuitry only i.e., P Circuit−Rx , as no signal amplification is required.
In WBAN, the human body also adds some attenuation in the signal. Therefore, Equation (1) includes coefficient c depicting the path loss: The equations that are derived above also highlights the importance of hardware selection in a routing protocol. In TAEO, the Nordic nRF 2401A sensor device is opted due to its low power consumption, which has been further improved by effectively controlling the transmission power.

Thermal Model
The Specific Absorption Rate (SAR) is an effective tool to identify the rate at which the heat is absorbed by tissue per unit of its mass. It estimates the heat that is absorbed due to RF waves exposure for a specified duration and for a specific device. Using SAR, limits can be identified for a sensor node to curtail the dissipation of heat accordingly. SAR depicts the probable biological concern on tissues due to RF communication. For mobile phones, the Federal Communication Commission (FCC) has provided a safe SAR range for adherence [22,23]. These are highly beneficial in designing wireless communication systems. The surpassing of SAR limits leads to damage to the tissues [24]. SAR can be evaluated through the Equation (4): where, value gives conductivity of the tissue is represented by σ, ρ is the density, and E is the induced electromagnetic field indicating the strength.

Proposed Methodology-TAEO
As discussed earlier, the proposed routing protocol addresses two-core implementation issues associated with in-vivo sensors: 1.
effective detection and avoidance of a Hot-Spot node based on SAR and 2. enhancement in stability period and network lifetime.

Routing
The proposed routing protocol is based on a multi-hop schema and it works by selecting a Data Forwarder (DF), who is responsible for the collection of packets from surrounding nodes and thereby forwarding them to sink for onward submission to a remote server.
A node with less temperature and distance from the sink along with higher residual energy is selected as DF. The temperature of DF is estimated, and is only allowed for transmission if it is below the specified threshold; otherwise, the respective DF is suspended for upcoming few rounds, until its temperature is revamped to normal. As a multi-hop routing protocol, a neighbor node can also be selected as a forwarder, especially, in the case where the distance of DF to sink is comparatively greater than a node's direct distance to sink. In such a scenario, the node itself will directly forward

Initialization
In this phase, each node broadcasts a packet containing the coordinates to its location, energy level, heat value, and the node identifier. In this way, all of the nodes are updated with the location of their neighbors and related information. Sink then broadcast a packet and disseminates its location to all of the nodes on the body. The temperature threshold is a value where there is a chance of tissue damage. This becomes two-fold when the temperature value exceeds from the threshold value. The most important aspect in WBAN is to avoid tissue damage as the heat that is generated by the node's circuitry and antenna could cause damage to the human tissue [25]. The fact that the human body has a thermoregulatory mechanism to balance the heat around the body does not mitigate the issue of tissue damage when the heat received rate is larger than the thermoregulatory mechanism rate [25]. This issue can be resolved by avoiding those implantable nodes in the routing path that have been heavily utilized and may cause a risk of tissue damage. A metric, which is called Specific Absorption Rate (SAR), can be used to measure the rate at which energy is absorbed by tissues when exposed to an electromagnetic field. Experiments show that exposure to an SAR of 8 W/Kg in any gram of tissue in the head or torso for 15 min may have a significant risk of tissue damage [26]. Even with more modest heating, organs that are especially sensitive to any temperature increase due to a lack of blood flow to them are prone to thermal damage (e.g., lens cataracts [27]). Researchers have also expressed concern regarding the heating of the eye by examining the SAR and the temperature of the eye when exposed to a Wireless LAN [28] or Infrared [29] radiation. In [30], a detailed description about the thermal effects of bioimplants has been presented.
When considering the above discussion, the threshold of the parameter Temp used in Figure 2 is set according to the application requirement where the system is going to be used. It would be different when the sensors are places in the head instead of any other part of the body.

Routing
The proposed routing protocol is based on a multi-hop schema and it works by selecting a Data Forwarder (DF), who is responsible for the collection of packets from surrounding nodes and thereby forwarding them to sink for onward submission to a remote server.
A node with less temperature and distance from the sink along with higher residual energy is selected as DF. The temperature of DF is estimated, and is only allowed for transmission if it is below the specified threshold; otherwise, the respective DF is suspended for upcoming few rounds, until its temperature is revamped to normal. As a multi-hop routing protocol, a neighbor node can also be selected as a forwarder, especially, in the case where the distance of DF to sink is comparatively greater than a node's direct distance to sink. In such a scenario, the node itself will directly forward data to Sink. Neighbor node's heat estimation will be performed likewise for Hot-Spot detection and avoidance.
In order to find an increase in the temperature of human tissue upon exposure to RF communication, there is a need to calculate the SAR value of the tissue where the sensor is implanted. Equation (4) is used to calculate SAR. Temperature rise is estimated by using Equation (5) [10,31]: where, t is the time duration for which tissue is exposed to electromagnetic radiation and C is the specific heat capacity. The TAEO protocol also controls the energy utilization to increase the network lifetime and stability period. It first specifies a distance value. Afterward, it calculates the distance in between six non-critical nodes and to sink. During transmission, the distance between the two nodes involved in the communication is compared to the specified distance value. If it is below the defined distance limit, it means that the two nodes are closely placed. Therefore, it utilizes less transmission power. Otherwise, for distant nodes, the packet is transported with maximum energy.

Scheduling
Data submission to the DF node or to sink is TDMA based, in which a time slot is assigned to each node, during which it can submit the recorded information to the DF. If a node has no data to transmit during the assigned time slot, it declares itself as an idle node and moves to a standby state to save power.

Results
In this section, a comparative analysis is conducted that is based on analytical simulations of two state of the art protocols with the proposed protocol. The major performance parameters selected in this study are: 1.
Throughput-No of packets successfully received at the sink.

2.
Stability Period-Duration before the energy of the first node depletes completely and node dies.

3.
Network Lifetime-Total time duration till the last node's energy drains completely and all nodes in a network died.

4.
Thermal Dissipation-Heat energy generated by the internal sensor circuitry and due to electromagnetic signals. This needs to be controlled for a hazard-free deployment of WBAN.
The evaluation of TAEO routing protocol is carried in comparison to ATTEMPT (Adaptive Threshold-based Thermal-aware Energy-efficient Multi-hop ProTocol) [19] and SIMPLE (Stable increased-throughput multi-hop protocol for link efficiency in wireless body area networks) [32].

Network Architecture
Eight miniature sensors are implanted in-body and a sink node is placed near its waist. The sensors read data from the body under observation related to temperature, heart rate, glucose, etc. Table 2 shows a list of sensors with the specified location. Data from heart rate and glucose monitoring devices are specified as critical and they are directly transferred to the sink. Also, all other sensor nodes will follow the normal multi-hop transmission. Sink will receive information from the critical and non-critical sensors and transmits it to the remote server through the internet. All of the nodes are equivalent in terms of energy consumption and processing power. When considering multihop transmissions, body movements may bring changes in topology. Table 2 depicts the initialization state of the network. Additionally, Table 3 elaborates on the simulation parameters. It also shows the SAR values and corresponding temperature change estimates [10,33].  Figure 3 represents the stability period analysis and proposed protocol supersedes when considering the stability period in comparison with ATTEMPT and SIMPLE protocols. Effective DF selection in each round along with hotspot node avoidance involves major elements that make proposed routing protocol more efficient. When comparing the stability period of three protocols, the first node in the proposed routing protocol dies in 6000th round, showing 50% improvement as compared to ATTEMPT and 28% with respect to the SIMPLE protocol.

Stability Period
Right Palm 0.1 0.8 Figure 3 represents the stability period analysis and proposed protocol supersedes when considering the stability period in comparison with ATTEMPT and SIMPLE protocols. Effective DF selection in each round along with hotspot node avoidance involves major elements that make proposed routing protocol more efficient. When comparing the stability period of three protocols, the first node in the proposed routing protocol dies in 6000 th round, showing 50% improvement as compared to ATTEMPT and 28% with respect to the SIMPLE protocol.

Residual Energy
Improvement in stability period also depicts the optimum power consumption of nodes in each round, which is illustrated in Figure 4 under the caption of residual energy. The reason behind this improvement is the careful selection of DF and, more importantly, the use of multiple transmission

Residual Energy
Improvement in stability period also depicts the optimum power consumption of nodes in each round, which is illustrated in Figure 4 under the caption of residual energy. The reason behind this improvement is the careful selection of DF and, more importantly, the use of multiple transmission power levels with respect to distance between nodes. The results in Figure 4 depict that, due to better stability period, the nodes of TAEO consume less energy and significant improvement is observed. Sensor nodes with underlying ATTEMPT lost more than half of its energy until 2000th round, whereas the proposed routing protocol utilized a similar amount of energy in approximately 6000 rounds and SIMPLE scheme until 4000 rounds.

Throughput
Throughput is one of the major performance metrics in terms of routing protocols. Figure 5 shows that an average number of successful packets reached the sink. The optimized use of energy keeps the node alive for a longer period resulting in better throughput with respect to SIMPLE and ATTEMPT. Initially, until 6000 rounds, TAEO is not performing optimally in terms of throughput, as the proposed protocol tries to avoid heat generation due to rapid transmissions of nodes, however, after 6000 rounds, it performs better, as it was efficient in residual energy prolonging its network lifetime along with maximizing the number of packets reached at sink node, as can be seen in Figure 5.

Throughput
Throughput is one of the major performance metric in terms of routing protocols. Figure 5 shows that an average number of successful packets reached the sink. The optimized use of energy keeps the node alive for a longer period resulting in better throughput with respect to SIMPLE and ATTEMPT. Initially, until 6000 rounds, TAEO is not performing optimally in terms of throughput, as the proposed protocol tries to avoid heat generation due to rapid transmissions of nodes, however, after 6000 rounds, it performs better, as it was efficient in residual energy prolonging its network lifetime along with maximizing the number of packets reached at sink node, as can be seen in Figure 5.  Figure 6 shows the ratio of the heated and unheated nodes after 16,000 rounds. The proposed protocol results in 70% saving i.e. only 30% of the total number of nodes become heated. Avoiding those implantable nodes in the routing path that have been heavily utilized and become heated does this. This is in contrast with the existing protocols, in which case 100% of the nodes become heated.  Figure 6 shows the ratio of the heated and unheated nodes after 16,000 rounds. The proposed protocol results in 70% saving i.e. only 30% of the total number of nodes become heated. Avoiding those implantable nodes in the routing path that have been heavily utilized and become heated does this. This is in contrast with the existing protocols, in which case 100% of the nodes become heated. Figure 6 shows the ratio of the heated and unheated nodes after 16,000 rounds. The proposed protocol results in 70% saving i.e. only 30% of the total number of nodes become heated. Avoiding those implantable nodes in the routing path that have been heavily utilized and become heated does this. This is in contrast with the existing protocols, in which case 100% of the nodes become heated.

Conclusions
The proposed protocol, TAEO, is a thermal dissipation controlled and highly stable routing protocol for WBAN. The selection of DF is based on node's heat value and residual energy. It has shown significant improvements in stability period and overall lifetime of the network. The detection of a hot spot node is carried out by estimating the temperature rise of a node in each round. In the case that the temperature exceeds the specified threshold, it suspends the node for the coming few rounds until its temperature is restored to normal. Varying the levels of energy and decision for the

Conclusions
The proposed protocol, TAEO, is a thermal dissipation controlled and highly stable routing protocol for WBAN. The selection of DF is based on node's heat value and residual energy. It has shown significant improvements in stability period and overall lifetime of the network. The detection of a hot spot node is carried out by estimating the temperature rise of a node in each round. In the case that the temperature exceeds the specified threshold, it suspends the node for the coming few rounds until its temperature is restored to normal. Varying the levels of energy and decision for the amount of energy required for transmission based on the distance between two nodes pursuing transmission also optimizes power consumption by a node. Our simulation results show that the optimum utilization of energy for transmission has significantly enhanced the overall network lifetime and throughput of the network.