Voronoi-based Efficient Surrogate-assisted Evolutionary Algorithm for Very Expensive Problems

Very expensive problems are very common in practical system that one fitness evaluation costs several hours or even days. Surrogate assisted evolutionary algorithms (SAEAs) have been widely used to solve this crucial problem in the past decades. However, most studied SAEAs focus on solving problems with a budget of at least ten times of the dimension of problems which is unacceptable in many very expensive real-world problems. In this paper, we employ Voronoi diagram to boost the performance of SAEAs and propose a novel framework named Voronoi-based efficient surrogate assisted evolutionary algorithm (VESAEA) for very expensive problems, in which the optimization budget, in terms of fitness evaluations, is only 5 times of the problem's dimension. In the proposed framework, the Voronoi diagram divides the whole search space into several subspace and then the local search is operated in some potentially better subspace. Additionally, in order to trade off the exploration and exploitation, the framework involves a global search stage developed by combining leave-one-out cross-validation and radial basis function surrogate model. A performance selector is designed to switch the search dynamically and automatically between the global and local search stages. The empirical results on a variety of benchmark problems demonstrate that the proposed framework significantly outperforms several state-of-art algorithms with extremely limited fitness evaluations. Besides, the efficacy of Voronoi-diagram is furtherly analyzed, and the results show its potential to optimize very expensive problems.

INTRODUCTION
The evaluation of solutions to some real-world problems could be difficult or very expensive, in terms of computational cost or money. For instance, evaluating once a solution to the Navier-Stokes equations involves several hours of computational fluid dynamic simulation [1]. Therefore, it is often the case that only a limited amount of budget, in terms of the number of evaluations, is available for searching for a solution. How to search efficiently toward an optimal solution within a small number of evaluations is a crucial problem.
In the past decades, evolutionary algorithms (EAs) have been widely applied to many real-world problems [2]. However, EAs' ability of converging close to the global optimum relies on the availability of a sufficiently large number of Hao Tong, Changwu Huang, Jialin Liu  fitness function evaluations. Obviously, the common EAs is not suitable to solve very expensive problems due to the large number of fitness evaluations (FEs) required to obtain an acceptable solution. Hence, surrogate-assisted evolutionary algorithms (SAEAs), which sometimes sample form a cheap surrogate model instead of successively calling the actual complex evaluation process in conventional EAs, have been proposed to solve computationally expensive problems (CEPs) in mid-1980s [3].
SAEAs have been popular over the past years for its lower number of FEs required for convergence thanks to the use of surrogate models. Many effective models have been employed to reduce the number of FEs required. Examples include polynomial regression (PR) [4], Kriging model [5], support vector machine (SVM) [6], radial basis function (RBF) [7] and neural network (NN) [8]. Besides, the model management strategies play a significantly important role in guaranteeing the convergence of SAEA [9]. One of the most popular strategies is the individual-based model management [10], in which only a few individuals are evaluated by actual function in each generation. Generally, only the individual considered to be the best by the surrogate will be evaluated to exploit the search space [8], but it is likely to be trapped in a local optimum and easily cause premature convergence. On the other hand,information of uncertainty provided by Kriging model is employed to construct much better re-evaluation strategy, like Expected Improvements(EI), Lower Confidence Bound(LCB), Possibility of Improvements(PoI) [11], which are improve the accuracy of surrogate as well as the ability of exploring the search space.
Although SAEASs require fewer FEs to converge than common EAs and variations of SAESs have shown their strength in solving some real-world CEPs (e.g., engineering design [12], health services [13], interactive design [14], etc.), the FEs required is still problematic in some very expensive problems. Take the aircraft design as an example, the duration of accomplishing one crash simulation varies from 36 to 160 hours [15]. If the SAES starts to converge to a global optimal solution after 100 FEs, the total optimization time will be between 150 and 660 days. Despite the importance and demand of handling the budget issue, there is a lack of work considering very limited budget in the literature. For example, Lu et al. [16] applied differential evolution (DE) assisted by rank-SVM for expensive problems given 100D FEs as budget, where D denotes the problem dimension.
The Gaussian process assisted EA proposed by Liu et al. [5] consumed fewer, but still, 50D FEs. More recently, the particle swarm optimization (PSO) assisted by semi-supervised learning (SSL-PSO) [17] and the active learning based PSO [18] were both given only 11D FEs as optimization budget.
All of the existed works use a budget of at least 11D FEs probably due to the minimal number of FEs required to initialize the corresponding algorithm, otherwise these algorithms are not applicable any more. In this work, we consider the extremely expensive problems with only severely limited actual fitness evaluations available and propose a new optimization algorithm named Voronoi-based efficient surrogate-assisted evolutionary algorithm. This work provides the following novel contributions: • To the best of our knowledge, this work is the first to solve very expensive black-box optimization problem with only 5D FEs. With such a limited computational budget, the preference region and model accuracy of corresponding area are much important that we should devote the limited resource to potential optimal area. • We proposed an efficient algorithm for very expensive optimization problem. Leave-one-out cross validation is employed to detect the uncertain area where the current surrogate model is not able to describe much precisely. Meanwhile, the Voronoi diagram is used to partition the search space and force the algorithm to exploit a relatively better area, which makes the algorithm more efficient for very expensive problems with limited fitness evaluations. • The numerical experiment for the proposed framework manifests its superior performance in very expensive black-box problem compared with the state-of-art algorithms. And we also analysed the influence of Voronoi diagram to the framework through empirical results. The remainder of paper is structured as follows. The literature review for SAEAs and the motivation for this work is introduced in Section II. Section III describes the proposed framework and gives some further discussion on the new algorithm. Experiments are presented in IV for comparing the proposed algorithm with the state-of-art algorithms on a set of benchmark problems. Finally, a brief conclusion and the discussion of future work are drawn in Section V.

LITERATURE REVIEW
The conventional evolutionary algorithm is not ideal to tackle CEPs due to the huge fitness evaluation consumption. SAEAs, replacing the actual expensive fitness evaluation by cheap surrogate evaluation, are very effective for CEPS [19] which could categorized into two kinds of SAEAs in the literature, reducing the computational cost in different levels.
a) Category A: The popular framework of SAEAs is inherited from the canonical evolutionary algorithm, which also contains mutation, crossover and selection operator [9]. The surrogate model applied in the framework mainly plays two roles, the approximation and the preselection.
For the fitness approximation, surrogate models mainly influence the selection process that most individuals are evalu-ated by metamodels while remainder individuals are evaluated by the actual fitness function to guarantee the convergence of optimization. For instance, the classification-and regressionassisted differential evolution (CARDE) makes use of classification and regression methods to assist the selection process in differential evolution algorithm [20]. Besides, some algorithms may cluster the population into several clusters and only evaluated the representative individual of each cluster and the other individuals are approximated by the surrogate model [21].
On the other hand, SAEAs employ the surrogate model for preselection, which reduces the computational cost by improving the quality of individuals of the selection process [22]. During the mutation and crossover processes, the framework generates plenty of offspring and the surrogate model is used to preselect the potentially better individuals for selection operator. For example, the differential evolution algorithm employed the preselection strategy (GPEME) to solve the medium scale expensive problems, where only the most promising solution will be re-evaluated by the actual fitness function in every generation [5].
Although the above framework is very effective for expensive problems, the fitness evaluation consumption is still very huge for very expensive problems because the involved population-based algorithm has to evaluate a certain number of individuals to guarantee the convergence and diversity of the algorithm. For example, the CARDE costs 10000 FEs for 30 dimension problems and the GPEME still uses 1000 FEs for problems of 20 and 30 variables. The number of fitness evaluations required to initialize for this kind of SAEA might not be afforded in a very expensive problem. b) Category B: Different from the framework discussed above, another SAEA framework is inherited from the efficient global optimization (EGO) [23]. This category of framework focuses on sampling solutions in the whole search space to improve the accuracy of surrogate model or the quality of the re-evaluated solution, which aims to balance the exploitation and exploration in the optimization process. For example, the classical EGO algorithm uses EI criterion to re-evaluate solutions, which considers the uncertainty of the current surrogate model and the quality of the re-evaluated solution simultaneously.
Recently, researchers proposed different criteria to describe the model's uncertainty. For example, Wang et al. [18] employed the query by committee active learning to assist this kind of SAEA (CALSAPSO), where the solution with the biggest disagreement among various models is regarded as the most uncertain solution. And the local search focuses on the surrogate model constructed by the top 10% evaluated solutions to exploit the current best area.
Although the optimization ability of this category of SAEAs is significantly inferior to the first category of SAEAs, the fitness evaluation is dramatically reduced to obtained an acceptable solution. For example, the CALSAPSO only consumed 11D FEs to solve the expensive problem. Obviously, this kind of SAEAs is more appropriate for the expensive problem.
In this paper, we focus on very expensive problems that the number of fitness evaluations is only setting as 5D in this paper. Obviously, the SAEA based on EGO framework is more appropriate for this problem. However, CALSAPSO, as the most efficient algorithm in the literature is hard to handle CEPs with extremely limited fitness evaluations. The ensemble model to determine the most uncertain solution could only help to improve the accuracy of the surrogate model. As a sequence, the number of fitness evaluation for exploitation will be very small that the performance for very expensive problem will be hindered. Furtherly, the CALSAPSO collects the top 10% best evaluated solutions to build a surrogate model for local search, which only focuses on the region that the selected solutions determine. It has a big problem that the global optimum is likely to locate outside the local area and then the local search is not able to find a better solution for this case.
To address the limitation of CALSAPSO, we employ the leave-one-out cross-validation (LOOCV) and Voronoi diagram to embed in CALSAPSO's backbone, in which the LOOCV can detect the uncertain area of landscape and obtain a better solution at the same time and the Voronoi diagram can guarantee a better solution to be covered by some best Voronoi cells. As a sequence, the new framework is more efficient for very expensive problems. The details of the proposed framework will be introduced in the next section.

Overall framework
According to the previous discussion, we propose a novel model management strategy to assist evolutionary algorithms, named as Voronoi-based efficient surrogate-assisted evolutionary algorithm (VESAEA). The overall procedure of VESAEA is presented in Fig. 1.
We embed LOOCV error surrogate model and Voronoi diagram into the framework, which are much more efficient than the original framework for very expensive problems. The proposed algorithm also contains the two search stages as the global search and the local search. The global search is made up of two stages: one employs the LOOCV error to reevaluate the most valuable individual and another optimizes the surrogate model to select the potentially best individual. On the other hand, the local search applies Voronoi diagram assisted strategy to select the best sample in the local area. Two search processes are controlled by a performance selector which switches the search stage to another one if the current search has no improvement in the previous generation.
In the framework, particle swarm optimization [24] is employed as the optimization algorithm in the global search process. The surrogate used for both global and local search stages in the framework is the RBF model [25].
The proposed algorithm initializes a dataset by Latin hypercube sampling (LHS) [26], at first. Due to the limited fitness evaluations, we use part of fitness evaluations for initialization, which is set as 2D samples in this work. And then the algorithm operates the global search with evaluating two samples on the basis of LOOCV error and RBF surrogate model. The Voronoi-based local search will be applied if the global search makes no improvement during the optimization process according to the performance selector and vice versa. Finally, the algorithm will terminate if it runs out the total FEs setting as 5D in our experiments.
In the following subsections, the detail of two different stages will be described including global search with LOOCV and Voronoi for local search.

Global search with LOOCV a) Construct model with LOOCV error:
The LOOCV is able to measure the sensitivity of landscape and points with high LOOCV error usually located at the rugged area [27]. Therefore, re-evaluating samples with high LOOCV error will not only help to improve the model accuracy but also might discover the local optimal located in the rugged region.
We could easily calculate the LOOCV error for evaluated samples as presented in Eq. (1), whereŷ i is the approximate fitness value according to the surrogate model constructed excluding point (x, y i ).
However, it is impossible to obtain the LOOCV error for unobserved samples in search space. Suggested by [27], we construct a surrogate model by LOOCV error of evaluated samples to approximate the LOOCV error of every point in the design landscape. Assume the dataset: And by leave-one-out experiments, we could obtain the dataset of samples with LOOCV errors as: Sampling with maximal LOOCV error: After completing all leave-one-out cross validation experiments, the surrogate modelê LOO (x) will be built for LOOCV error. The RBF surrogate model with thin plate spline [25] is used both in the cross-validation experiments and LOOCV error model building. Once the model is constructed, the PSO optimizer will be applied to search for the point with the highest approximate LOOCV error in the whole landscape as presented in the Eq. (2) When PSO satisfies the terminal condition, the solution with the highest LOOCV error found by PSO will be re-evaluated by the fitness function and be saved into an archive. c) Construct surrogate model: The second stage in global search is to optimize the global surrogate model followed by individual re-evaluation in LOOCV error model sampling. The global surrogate modelf (x) is still built by the RBF model and optimized by the PSO algorithm. Therefore, the objective function changes into Eq. (3):

d) Select best approximate solution:
Once the PSO completed the optimization process, the best solution found is reevaluated using actual expensive fitness function and then the new sample will be added to the archive.

Algorithm 1: Pseudo Code of Local Search
Input: Samples: S = {xi|i = 1, 2, ..., N } 1 P rand ← Generate a large number of samples in the search space; 2 n ← Dimension of the problem; 3 |P rand | = |S| × n × 1000 4 Each existed point xi constructs one cell Ci; 5 for pr ∈ P rand do 6 pr is assigned to the closest Voronoi cell Ci.
Output: New sample: x * According to the performance selector, the search process will be switched into local search if the global search performs poorly in the current generation. We creatively applied Voronoi diagram approach to promote the efficiency and performance of local search. In mathematic, the Voronoi diagram is used to partition the space into several small regions as described in Section II. In this framework, we regard the evaluated sample of each Voronoi cell as the representative point and the Voronoi cells with better representative points are regarded as better cells. Then, the local search is operated in better cells. The procedure of Voronoi for local search is presented in Algorithm 1.
a) Partition space with Voronoi diagram: The landscape is firstly partitioned by samples in the sample set. Consider a sample set S = {x 1 , x 2 , . . . , x N }, each point constructs a Voronoi cell V i defined in (4): where, d(x, x ′ ) denotes the Euclidean distance between points x and x ′ [28]. Points of each Voronoi cell are closest to the corresponding evaluated sample of this cell. Thus, the boundary between the cells of any pair of adjacent points is the perpendicular bisector of these two points. Due to the irregular shape of Voronoi cell, it is hard to describe the boundaries with one specific equation. The Monte Carlo (MC) simulation is an alternative method to identify the boundaries approximately [29] which is presented in Lines 1-8 of Algorithm 1 where | · | denotes the size of one sample set. A large amount of samples are randomly sampled in the entire search space, then each sample is assigned to a Voronoi cell according to Eq. (4). b) Identify the better Voronoi cell: Then, the top 10% best cells are selected to form a pool of samples C top as presented in Line 9 in Algorithm 1. Assume the number of 10% best cells and total random samples are k, N P respectively. Then, the C top can be described as Eq. (5).
c) Select best solution in local area: Then, the framework will select the best sample in the set of C top according to the approximate modelf (x) that the objective function is  6). The new sample will also be re-evaluated with expensive fitness function after finishing local search and then added to the archive.
Take Fig. 2 as an example, P 1 , P 2 are two best samples so that C 1 , C 2 are two best Voronoi cells and S is the optimal solution. According to the local search, the new samples will be selected from C top = C 1 ∪ C 2 . As shown in Fig. 2, we could easily find the optimal solution is likely to locate in the few top best Voronoi cells.

Discussion
In order to obtain a better solution, it is very important to trade off the exploration and exploitation. In this framework, a performance selector is applied. The global search explores potentially better area in the whole search space and then the local search is applied to exploit the specific area to obtain a local optimum if global search has no improvement in current generation.
The global search firstly starts with sampling with respect to the LOOCV error to increase the model accuracy. After re-evaluating the sample with maximal approximate LOOCV error, the new surrogate model built by all samples in the dataset is more likely to describe the valuable rugged area of landscape where the local optimal might hidden. Then, reevaluating the best individual of new accurate surrogate model is easy to find a better solution. Therefore, the global search has the ability to not only explore the uncertain area, but also has the potential to obtain a higher quality solution.
By contrast, the local search considers exploiting the potential area. In the framework, the space partition makes local search more focus on the specific better area and meanwhile, the top 10% cells are selected to avoid trapping into a local optimum. Monte Carlo simulation is used to describe the Voronoi cells and it only requires to select the new solution among the generated random samples belonging to the selected cells, which avoids searching next sample by complex algorithms.

NUMERICAL EXPERIMENT AND ANALYSIS
In order to analyse the efficacy of VESAEA, we conduct a series of empirical studies in this section. We compared the proposed algorithm with several state-of-art algorithms on classical some benchmark problems at first. Then the empirical studies of Voronoi diagram's effect to the proposed algorithm are presented.

Experimental Setup
To assess the performance of SAEA, we select one efficient classical algorithm and three state-of-the-art algorithms, which have different characteristics on computationally expensive problems. A brief description of four algorithms is presented below.
• EGO-LCB: Lower confidence bound (LCB) criterion based efficient global optimization [23] is a classical algorithm but performs efficiently on expensive problems, especially on low-dimension problems. • GPEME [5]: DE assisted by Kriging model with individual-based evolution strategy. The LCB criterion is also employed as the re-evaluation strategy. • SSLAPSO [17]: PSO is enhanced by semi-supervised learning which is employed to make use of both evaluated and un-evaluated samples. Two RBF surrogate models are built by true evaluated and approximately evaluated samples respectively to determine new individuals' fitness. • CAL-SAPSO [18]: The committee-based active learning is applied to SAEA framework by using ensemble of PR, RBF and Kriging model. Three surrogate models are used simultaneously to determine the most uncertain samples and potentially best solutions. The comparative experiments are performed on 5 widely used benchmark problems of 5 different dimensions listed in Table II. The size of initial samples is set to 2D and all algorithms terminate after 5D real fitness evaluations. The surrogate models used in all algorithms are all imported from SURROGATES tool-box [30], which is based on a RBF toolbox [31].

Comparative Experiment on Benchmark Problems
The results obtained by five algorithms over 25 independent runs are shown in Table I  where the x-axis ranges from 2D to 5D because the first 2D fitness evaluations are used for initialization, which is same for all algorithms in each run for fair comparison. The average rank is calculated according to the best fitness on each test problem. Then, the Friedman statistical test and its postdoc test, Nemenyi test [32], are both carried out with a 0.05 significance level. The p-value by Friedman test equal to 6.193e − 12 lower than 0.05 significantly, revealing significant difference among five algorithms. Then the Nemenyi test result is presented as adjust p-value in Table I where VESAEA is the control method. It also shows the performance of VESAEA on benchmark problems is greatly different with ECG-LCB,  The Sphere problem is a uni-modal problem and the Griewank problem is a little similar to sphere problem except for many small peaks in the whole landscape. The performance of the proposed framework is also very similar in these two functions whose evolution curves are presented in Fig. 4. Compared with the other four algorithms, VESAEA obtains the best results both in convergence behaviour and the quality of final solution for D = 10, 15, 20, 30. By contrast, the EGO-LCB is slightly more efficient than others when the dimension D = 5. As a sequence, we could generally conclude the proposed model management strategy is efficient in the overall uni-modal whether with or without small peaks.
For Ackley function, which has a nearly flat outer region and a large hole at the centre, the VESAEA is slightly better than other four algorithms for D = 15, 20, 30 while the EGO-LCB performs best for D = 5, 10 as shown in the first column of Fig. 5. Also for D = 15, 20, 30, we could easily observe that CALSAPSO performs well at the beginning of search process but is surpassed by VESAEA in the latter stage of the search, which indicates that the new framework is capable to balance better the exploitation and exploration. Different with the above problems, Rosenbrock function is a multi-modal problem with a very narrow valley. VESAEA obtains the best results for problems of D = 5, 10, 15 and the convergence profile is presented in the second column of Fig. 5. From evolution processes, it is obvious that VESAEA outperforms other four methods totally for problems of D = 10, 15. And for the lowest dimension case, VESAEA is also not very efficient in the early search stage but still gets best result finally similar to former cases in Ackley problems. However for D = 20, 30, the CALSAPSO converges slightly better than VESAEA in the latter stage of the search and obtains higher-quality results.
For the most complicated test problem, Rastrigin function, the performance of VESAEA framework is better than EGO-LCB, GPEME and SSLAPSO and almost equal to CAL-SAPSO of D = 10, 15, 20, 30 as shown in Fig. 6. And when dimension equal to 5, the LCB-EGO still obtains the best solution compared with other four algorithms. Actually, the Rastrigin problem has many significant peaks that have strong capacity of trapping the optimization algorithm in one of them. And all five algorithms are hard to jump out of the local optimal in limited fitness evaluations.
From the above observations, we can see that the VESAEA is capable of solving the uni-modal problems even though with little noise in the search space. VESAEA obtains the best result in 16 out of 20 test cases of D = 10, 15, 20, 30 which indicates its capability in optimizing low-dimension of problems. But the classical framework, EGO-LCB, is always the best algorithm in very low-dimension problems, like D = 5. Besides, experimental results also indicate that it is hard for GPEME and SSLAPSO to find a high quality solution in such limited computational resources. However, the performance of VESAEA deteriorates in very complicated problems with many attractive local optimum, like Rastrigin functions. For this kind of problems, the global search hardly jumps out of the local optimums with limited fitness evaluations and on the other hand, the Voronoi based local search only focuses on the better area in the current generation. As a sequence, the algorithm is easily trapped in the local optimal especially for relatively high dimension cases.

The effect of Voronoi-based local search
In this paper, we creatively employed Voronoi diagram for local search in SAEA framework. In order to study the effect of Voronoi based local search furtherly, we analyzed the average contribution ratio of local search and global search stages over all 25 independent runs. The results are presented in Fig. 3 where the red dotted line represents bisector of contribution ratio.
In Fig. 3, there are 19 out of 25 test instances in which the local search contributes more than the global search, which indicates the potential performance of local search. We can find in Griewank and Sphere problems, the Voronoi-based local search accounts for a large proportion (over than 70%) of the whole search process, while in Rastrigin and Rosenbrock problems, both search stages make about the same contributions to the final result. Moreover, we calculate the relation the between dominated search stage and the optimization result as presented in Table III   To further assess the capability of Voronoi-based local search, we compare VESAEA and VESAEA without Voronoibased local search stage (VESAEA-woVLS) on 5 test problems of dimension d = 15 with 25 independent runs. The results are presented in Table IV and the convergence curves are presented in Fig. 7 where the initial sample size is 2D and the total fitness evaluation is 5D. From the Table IV and Fig. 7, it is very clear that VESAEA performs significantly better than VESAEA-woVLS in Sphere, Ackley and Griewank problems, probably because the better Voronoi cells usually cover the area containing global optimum instead of local optimum. And it has a similar performance to VESAEA-woVLS in Rosenbrock problems because the best Voronoi cells might all fall into the narrow valley and VESAEA can not benefit much from the Voronoi-based local search. But in Rastrigin problem, VESAEA-woVLS performs slightly better than VESAEA, probably because VESAEA-woVLS only focuses on the global search and easily jumps out of many attractive local minimums. In addition, we could find test problems in which VESAEA performs better than VESAEA-woVLS are consistent with problems in Fig. 3 in which the local search contributes more than the global search.
Overall, we could conclude that the Voronoi-based local search is able to improve the convergence of optimization, especially for uni-modal problems and multi-modal problems with many small peaks. And the Voronoi-based local search is not obviously outstanding in complicated multi-modal problems with many attractive local optimums.

CONCLUSION
A Voronoi-based efficient surrogate assisted evolutionary algorithm framework is proposed in this paper for very expensive problems with extremely limited computational resources.     The framework mainly has two search stages including leaveone-out cross validation based global search and Voronoi based local search. The global search stage employs LOOCV and RBF model to improve the accuracy of surrogate model and explore the search space. In local search stage, the Voronoi diagram, which divides the whole space into many cells, is applied to assist the exploitation in local area. We designed a performance selector which switches between two search stages by detecting whether current search stage has improvements or not to trade off the exploitation and exploration. The efficacy of proposed framework is examined by comparing with a few state-of-the-art SAEAs on 25 commonly used test problems. The results demonstrate that the proposed framework is powerful at solving uni-modal problems, even though there are several small peaks around the response surface. Moreover, we evaluates the effect of Voronoi-based local search and the results show that Voronoi diagram contributes a lot in finding a better solution on a very limited computational budget.
The ability of Voronoi for boosting optimization in expensive problems has been demonstrated in this work. In the future, the efficacy of Voronoi-based optimization algorithms will be tested on more complicated test functions. On the other hand, more strategies with Voronoi assisted SAEA for very expensive problems will be considered. Since Voronoi diagram divides the whole search space into many cells, it is highly desirable to investigate more on Voronoi diagram for large-scale problems.


extfieldsofstudy: Physics Engineering

text:
Design of OFDM Joint Synchronization Algorithm Based on Cyclic Prefix

By studying the Maximum Likelihood (ML) algorithm and applying it to Orthogonal Frequency Division Multiplexing (OFDM) system, we analyze its performance. OFDM system has obvious advantages over other wireless communication systems, but it also has disadvantages, such as being sensitive to synchronization errors. The ML algorithm can solve this shortcoming of OFDM system. ML algorithm uses the redundant information carried by the cyclic prefix in OFDM code element. At the same time, ML algorithm calculates its maximum likelihood estimation algorithm to complete the code element timing synchronization and carrier frequency synchronization. Compared with other synchronization algorithms, ML algorithm can complete initial capture and synchronous tracking in one step, so it is widely used in OFDM systems. In this paper, we build a traditional OFDM system and apply the ML algorithm to OFDM synchronization simulation. By changing the design of relevant parameters, we describe the ML algorithm in detail and obtain the same results as the theory.

1.Introduction
Since the beginning of the 21st century, with the gradual improvement of the degree of informatization, information exchange and resource sharing have become extremely convenient. People have put forward higher requirements for the effectiveness and reliability of communication technology. So the fourth generation mobile communication system based on multi-functional integration began to develop gradually [1].
OFDM is a multi-carrier orthogonal modulation technique with high spectrum utilization, strong ability to be undisturbed by other signals and multi-path fading ability caused by multipath propagation. The principle is that the high-speed serial data stream is decomposed and modulated, then sent to several parallel orthogonal sub-channels, so that the code element width of each sub-channel is larger than the channel delay expansion. However, since there are many orthogonal carriers in the orthogonal frequency division multiplexing system, it has higher orthogonality to the system than the traditional single carrier system. When OFDM signals are propagated in the system, they are affected by channel characteristics and other signals, which may result in mismatch between the sender and receiver, destroy the orthogonality between carriers, and cause interference between subcarrier (Inter-carrier interference, ICI),. Therefore, a good performance synchronization algorithm is particularly important for OFDM systems.
ML algorithm is a mature algorithm in OFDM synchronization algorithm, which is proposed by Van

OFDM mathematical modulation model
OFDM technology is a kind of multi-carrier modulation technology, and its difference from the general multi-carrier technology is that its carriers are mutually orthogonal.
The multicarrier modulated signal s(t) can be written as the following plural form: ∆ is the carrier frequency of the nth carrier. is the complex signal above the nth subcarrier. If it remains constant over a code-meta period , . Set the signal sampling frequency to 1/T，so: There are N sample values in one symbol period: τ NT. Without losing generality, set 0. So Comparing with Inverse Discrete Fourier Transform (IDFT) form (coefficient neglect), it can be seen that if is regarded as a frequency domain sampling signal, s kT is its corresponding time domain signal. If the following formula is established Then the above formula is equivalent to the IDFT transformation method. If the carrier time interval is selected as 1⁄ , the orthogonality of the signal subcarriers is maintained, and the process can also be defined by DFT. In other words, OFDM system adjustment part can be realized directly by digital signal processing, and doesn't need to be implemented by using complex algorithms. So the complexity of the system is reduced, and the high computing speed and high computation accuracy can also be satisfied [3][4].

OFDM Physical Framework
At the transmitting end of the OFDM system, firstly, the data stream to be transmitted is divided into multiple signals by a splitter, and each of them need to be scrambled, and channel forward error correction coding is added. Then, through the symbol interleaver, the multiplexed signals are combined on the same transmission line, and the parallel transmission signals are converted into serial transmission signals. The data signal is then modulated, aimed to form a transmitted frame. And what is embodied here is symbol timing synchronization. Then, the Inverse Fast Fourier Transform Algorithm (IFFT) process is used to calculate the signal in the frequency domain into the time domain. A guard interval is then inserted to accommodate symbol crosstalk. Then, through digital-to-analog conversion, the digital signal is converted into an analog signal, where the signal waveform begins to appear. Then, after quadrature modulation, the signal is amplified, and the transmitted signal power is increased, and then transmitted to the channel.
After the signal reaches the receiving end, it is amplified and then officially enters the demodulation process. In the first step, analog-to-digital conversion is performed, and the analog signal in the channel is converted into a digital signal for subsequent arithmetic processing. Then enter the frequency offset module. Due to the influence of the channel, the received signal at the receiving end is not completely orthogonal to the local carrier, so frequency compensation is needed. Here, three kinds of synchronization occur: symbol synchronization, frequency synchronization, and sampling clock. Synchronize. The sampling clock controls the sampling frequency, and then sends the sampling 3 frequency value to the crystal oscillator, which controls the frequency of the analog-to-digital conversion module. Frequency synchronization mainly uses a synchronization algorithm to calculate the amount of frequency offset. After frequency compensation, a Fast Fourier Transformation (FFT) operation is performed on the received signal to convert the time domain signal into a frequency domain signal. The latter phase correction, channel equalization, and demapping are all corresponding to the corresponding processes at the transmitting end, except that the transmitting end is a modulation process and the receiving end is a demodulation process. Finally, the corresponding information code stream is recovered by decoding and descrambling [5].

3.Introduction of ML Algorithm
Due to the limitation of the modulation mode of the OFDM system, the two synchronization processes need to be completed before the signal is demodulated. The first step is to find the boundary range of the OFDM signal and the best sampling time point, in order to minimize interference between the sub-carriers and the inter-symbol interference. The second step needs to be correctly calculated the frequency offset between carriers, in order to ensure the orthogonality between carriers. The ML algorithm inserts a cyclic prefix in front of the transmitted OFDM data stream, and calculates the correlation between cyclic prefixes at the receiving end to calculate the windowing position and frequency offset [6].
The following is the mathematical principle of OFDM algorithm. In the OFDM system, the baseband form of the transmitted signal at the transmitting end can be expressed as Because the actual received signal is affected by delay, frequency offset and gaussian white noise, the actual received signal should be: In the above formula, s k is the transmitter transmitting signal, and n k is the channel noise, and θ is the timing offset, and ∈ is the normalized carrier frequency deviation.
The ML algorithm considers both the symbol timing synchronization position and the carrier frequency deviation, so the maximization of the log likelihood function is completed in two steps: The maximum carrier offset can be based by the expression of Λ θ, ε , which can set to 0. When the cosine value is 1, the maximum carrier frequency offset of Λ θ, ε can be obtained. So the ML of ε is estimated as: In the above formula, n is an integer, taking into account that the general carrier frequency deviation should be in a relatively small range, so as to take the n=0. At this point, the logarithmic likelihood function is So , θ, ε's combined ML is estimated to be When we use MATLAB for simulation, because the ML algorithm performs a correlation operation on each function value of the cyclic prefix, the calculation amount is extremely large, and it takes a long time to obtain the experimental result. In actual operation, the amount of data of the signal will be larger, and the waiting time needs to be longer, so the ML algorithm is not suitable for the transmission of the burst signal.
According to graph a, it can be seen that the maximum value of the likelihood function is probably the position of the N=120, and according to the setting of the experimental data, the best window opening position is N=128. From this, it can be seen that the ML algorithm for the best window opening position calculation is still more accurate. Moreover, the peak of function value image is prominent, and it is easy to get the best window opening position, which reduces the difficulty of physical level realization. From the a diagram can also be found that the waveform has a periodic, every 1300 points on the peak, but also to meet the composition of the signal is the cycle prefix + useful signal.
The data set by the experiment is that the simulation offset value obtained by the frequency offset e=0.25, b graph's result is between -0.5-0.5, and it can be found that the frequency offset is the most similar to the setting value when the optimal window opening position can be obtained: the correct optimal window opening position affects the accuracy of the frequency offset value. Therefore, for the ML algorithm, it is very important to get the accurate window opening position.
In practical applications, adding a cyclic prefix in front of the signal reduces the effectiveness of the system, while the ML algorithm relies on the length of the cyclic prefix. When the length of the cyclic prefix is low, the calculation of the window opening position of the system is not very accurate, which affects the calculation of the frequency offset value, resulting in ICI and ISI, which greatly affects the reliability of the system.
During the simulation process, we change the value of the cyclic prefix while keeping other conditions unchanged. When the value of cyclic prefix is L=16, get the following two result graphs:


GET-AID: Visual Recognition of Human Rights Abuses via Global Emotional Traits

In the era of social media and big data, the use of visual evidence to document conflict and human rights abuse has become an important element for human rights organizations and advocates. In this paper, we address the task of detecting two types of human rights abuses in challenging, everyday photos: (1) child labour, and (2) displaced populations. We propose a novel model that is driven by a human-centric approach. Our hypothesis is that the emotional state of a person -- how positive or pleasant an emotion is, and the control level of the situation by the person -- are powerful cues for perceiving potential human rights violations. To exploit these cues, our model learns to predict global emotional traits over a given image based on the joint analysis of every detected person and the whole scene. By integrating these predictions with a data-driven convolutional neural network (CNN) classifier, our system efficiently infers potential human rights abuses in a clean, end-to-end system we call GET-AID (from Global Emotional Traits for Abuse IDentification). Extensive experiments are performed to verify our method on the recently introduced subset of Human Rights Archive (HRA) dataset (2 violation categories with the same number of positive and negative samples), where we show quantitatively compelling results. Compared with previous works and the sole use of a CNN classifier, this paper improves the coverage up to 23.73% for child labour and 57.21% for displaced populations. Our dataset, codes and trained models are available online at https://github.com/GKalliatakis/GET-AID.

I. INTRODUCTION
In the era of mobile phones with photo and video capabilities, social media and the Internet, citizen media and other publicly available footage can provide documentation of human rights violations and war crimes. However, the omnipresence of visual evidence may deluge those accountable for analysing it. Currently, information extraction from humanrights-related video and imagery requires manual labour by human rights analysts and advocates. Such analysis is utterly expensive and time consuming, while it is emotionally traumatic to focus solely on images of horrific events without context. This paper attempts to address the problem of predicting human rights abuses from a single image, and provides technical insight and solutions to address the above issues. Note that naive schemes based on object detection or scene recognition are doomed to fail in this binary classification problem as illustrated in Fig. 1. If we can exploit existing smartphone cameras, which are ubiquitous, it may be possible to turn human rights image analysis into a powerful and cost-effective computer vision application.
Recent work by Kalliatakis et al. [15] has shown that fine-tuned deep CNNs can address the human rights abuses classification problem to a certain extent. In this paper, we investigate the potency of representation learning methods in solving two independent, binary classification problems in the context of human rights image analysis: 1) child labour / no child labour 2) displaced populations / no displaced populations  In many instances, emotional traits can be the only notifying difference between an image that depicts human rights abuse and a non-violent situation.
As will be shown in the experimental section, we found that the integration of novel global emotional traits with a fine-tuned CNN reports an improvement in coverage up to 23.73% for child labour and 57.21% for displaced populations, over the sole use of a CNN classifier that is trained end-to-end using the given training data. Based on these new global emotional traits, our approach makes the following technical contributions: First, we describe the design and implementation of an end-to-end system, which is used to estimate the emotional states of people, adopting the continuous dimensions of the VAD Emotional State Model [23]. This model describes emotions using three numerical dimensions: Valence (V) to measure how positive or pleasant an emotion is; Arousal (A) to measure the agitation level of the person; and Dominance (D) that measures the control level of the situation by the person.
Our second technical contribution comprises a new mechanism capable of characterising an image based on the emotional states of all people in the scene, termed global emotional traits (GET). This mechanism exploits two of the continuous dimensions of the VAD emotional state model which are relevant to human rights image analysis. As will be explained in the following, global emotional traits are learned by jointly analysing each person and the entire scene similar to [17].
We present a human-centric, end-to-end model for recognising two types of human rights violations. Our central observation is that the proposed global emotional traits of an image, which expose the entire mood of a situation are highly informative for inferring potential human rights abuses, when used along with a standard image classification system. Finally, we describe HRA-Binary, a subset of the HRA dataset [15] as benchmark for this classification task. This is used to evaluate our GET-AID model. The remainder of the paper is organized as follows. In Section II we review the related work on object detection, emotion recognition and human rights abuses recognition. In Section III, we introduce our method, GET-AID. In Section IV, we describe the subset of the HRA dataset. Section V discusses our results on human rights abuse classification. We conclude this paper in Section VI.

II. RELATED WORK
This section gives an overview of the related work on human rights abuses recognition, which can be regarded as a category in image understanding. The background of object detection and emotion recognition are also investigated since they are important components in our human-centric approach.

A. HUMAN RIGHTS ABUSES UNDERSTANDING
Human rights abuses understanding plays a crucial role in human rights advocacy and accountability efforts. Automatic perception of potential human rights violations enables to discover content, that may otherwise be concealed by sheer volume of visual data. These automated systems are not producing evidence, but are instead narrowing down the volume of material that must be examined by human analysts who are making legitimate claims, that they then present in justice, accountability, or advocacy settings [2].
One recent example of the potential use of AI to advance human rights analysis can be found in the work of the Center for Human Rights Science (CHRS) at Carnegie Mellon University [25], where they have created computer vision methods to rapidly process and analyse large amount of video by detecting audio (explosions and gunshots), detecting and counting people and synchronising multiple videos from different sources. Another example is the Event Labeling through Analytic Media Processing (E-LAMP) computer vision-based video analysis system [3] which is capable of detecting objects, sounds, speech, text, and event types in a video collection.
Though the above methods have shown good performance in their respective applications, a significant concern associated with human rights-related technology, is that the ability to extract large video collections tends to be limited to institutions with large staff base or access to expensive, technologically advanced tools and techniques [16]. A different group of methods based on a single given image alongside the first ever image dataset for the purpose of human rights violations recognition was introduced in [14]. Recently, Kalliatakis et al. [15] proposed to recognise human rights violations from a single image by fine-tuning object-centric and scene-centric CNNs on a larger, verified-by-experts image dataset. That work also introduced a web-demo for recognising human rights violations, accessible through computer or mobile device browsers.

B. OBJECT DETECTION
Object detection, the process of determining the instance of the class to which an object belongs and estimating the location of the object by outputting the bounding box around the object, is one of the computer vision areas that has improved steadily in the past few years. It is possible to group object detectors using different aspects, but one of the most accepted categorisation is to split object detectors into two categories: one-stage detectors and two-stage detectors.
One-stage detectors: The OverFeat model [29] which applies a sliding window approach based on multi-scaling for jointly performing classification, detection and localization was one of the first modern one-stage object detectors based on deep networks. More recently YOLO [10], [22] and SSD [26], [27] have revived interest in one-stage methods, mainly because of their real time capabilities, although their accuracy trails that of two-stage methods. One of the main reasons being due to the class imbalance problem [20].
Two-stage detectors: the leading model in modern object detection is based on a two-stage approach which was established in [31]. The first stage generates a sparse set of candidate proposals that should contain all objects, and the second stage classifies the proposals into foreground classes or background. R-CNN, a notably successful family of methods [11], [12] enhanced the second-stage classifier to a convolutional network, resulting in large accuracy improvements. The speed of R-CNN has improved over the years by integrating region proposal networks (RPN) with the secondstage classifier into a single convolution network, known as the Faster R-CNN framework [28].
In this work, we adopt the one-stage approach of Reti-naNet framework [20] which handles class imbalance by reshaping the standard cross entropy loss to focus training on a sparse set of hard examples and down-weights the loss assigned to well-classified examples.

C. EMOTION RECOGNITION
Recognising people's emotional states from images is an active research area among the computer vision community. One established categorisation of emotion recognition methods in the literature is to split them into two categories based on how emotions are represented: discrete categories and continuous dimensions.
Discrete categories: most of the research in computer vision to recognise people's emotional states is explored by facial expression analysis [7], [8], where a large variety of methods have been developed to recognise the 6 basic emotions defined in [6]. Many of these methods are based on a set of specific localised movements of the face, called Action Units, in order to encode the facial expressions [5], [9]. More recently emotion recognition systems based on facial expressions use CNNs to recognise the Action Units [8].
Continuous dimensions: instead of recognising discrete emotion categories, this family of methods use the continuous dimensions of the VAD Emotional State Model [23], [24] to represent emotions. The VAD model uses a 3-dimensional approach to describe and measure the emotional experience of humans: Valence (V) describes affective states from highly negative (unpleasant) to highly positive (pleasant); Arousal (A) measures the intensity of affective states ranging from calm to excited or alert; and Dominance (D) represents the feeling of being controlled or influenced by external stimuli. In recent times the VAD model has been utilised for facial expression recognition [30].
In this paper, we adopted the tridimensional model of affective experience alongside a joint analysis of the person and the entire scene in order to recognise rich information about emotional states, similar to [17].

III. METHOD
We now describe our method for detecting two types of human rights abuses based on the global emotional traits of people within the image. Our goal is to label challenging everyday photos as either human-rights-abuse positive ('child labour' or 'displaced populations') or human-rights-abuse negative ('no child labour' or 'no displaced populations' respectively).
To detect the global emotional traits of an image, we need to accurately localise the box containing a human and the associated object of interaction (denoted by b h and b o , respectively), as well as identify the emotional states e of each human using the VAD model. Our proposed solution adopts the RetinaNet [20] object detection framework alongside an additional emotional traits branch that estimates the continuous dimensions of each detected person and then determines the global emotional traits of the given image.
Specifically, given a set of candidate boxes, RetinaNet outputs a set of object boxes and a class label for each box. While the object detector can predict multiple class labels, our model is concerned only with the 'person' class. The region of the image comprising the person whose feelings are to be estimated at b h is used alongside the entire image for simultaneously extracting their most relevant features. These features, are fused and used to perform continuous emotion recognition in VAD space. Our model extends typical image classification by assigning a triplet score S a img,GET to pairs of candidate human boxes b h and an abuse category a. To do so, we decompose the triplet score into three terms: We discuss each component next, followed by details for training and inference. Fig. 2 illustrates each component in our full framework.

A. MODEL COMPONENTS 1) Object detection
The object detection branch of our network, shown in Fig.  2(a), is identical to that of RetinaNet [20] single stage classifier, although the SSD [22] detector has also been tested. First, an image is forwarded through ResNet-50 [13], then in the subsequent pyramid layers, the more semantically important features are extracted and concatenated with the original features for improved bounding box regression. Although the combination of the proposal and classification components into a single network results in a more complicated training process and lower accuracy compared to their two-stage counterparts, RetinaNet manages to counterbalance that with higher detection speed by utilising a smarter loss function named focal loss.

2) Emotional traits recognition
The first role of the emotional traits branch is to assign an emotion classification score S e h,img to each human box b h and emotion e. Similar to [17], we use an end-to-end model with three main modules: two feature extractors and a fusion module. The first module takes the region of the image comprising the person whose emotional traits are to be estimated, b h , while the second module takes as input the entire image and extracts global features. This way the required contextual support is accommodated in our emotion recognition process. Finally, the third module takes as input the extracted image and body features and estimates the continuous dimensions in VAD space. Fig. 3 shows qualitative results of the method.
The second role of the emotional traits branch is to assign a two-dimensional emotion classification score S GET h,img which characterises the entire input image, based on two emotion classification scores S v h,img and S d h,img for each human. To the best of our knowledge, this particular score, called global emotional traits (GET), is the first attempt to establish a method for summarising the overall mood of a situation de- picted in a single image. We decompose the two-dimensional GET score into two terms: In the above, S v img is the encoding of the global valence score relative to human box b h and entire image img, that is: Similarly, S d img is the encoding of the global dominance score relative to human box b h and entire image img, that is: In Fig. 4 (a),(c) the three different emotional states over target objects location are estimated, while (b), (d) illustrate the global emotional traits proposed here. For the sake of completeness all three predicted numerical dimensions are depicted. However, only valence and dominance are considered to be relevant to human rights violations recognition tasks since the agitation level of a person, denoted by arousal, can be ambiguous for several situations. For example, both Fig. 4 (a) and (c) depict people with arousal values close to 5.5, but the activities captured are utterly different from a human rights image analysis perspective.

3) Abuse recognition
The first role of the abuse branch, shown in Fig. 2(c), is to assign an abuse classification score to the input image. Just like in the two-phase transfer learning scheme deployed previously for HRA-CNNs [15], we train an end-to-end model for binary classification of everyday photos as either humanrights-abuse positive ('child labour' or 'displaced populations') or human-rights-abuse negative ('no child labour' or 'no displaced populations').
In order to improve the discriminative power of our model, the second role of the abuse branch is to integrate S GET h,img in the recognition pipeline. Specifically, the raw image classification score S a img is readjusted based on the recognised global emotional traits. Each GET unit, that is deltas from the neutral state, is expressed as a numeric weight varying between 1 and 10, while the neutral states of GET are assigned between 4.5 and 5.5 based on the number of examples per each of the scores in the continuous dimensions reported in [17]. The GET feature of each input image can be written in the form of a 2-element vector: where D 1 and D 2 refer to the weights of valence and dominance, respectively. The adjustment that will be assigned VOLUME 4, 2016 to the raw probability, S a img is the weight of valence or dominance multiplied by a factor of 0.11 which has been experimentally set. When the input image depicts positive valence or positive dominance, the adjustment factor is subtracted from the positive human-rights-abuse probability and added to the negative human-rights-abuse probability. Similarly, when the input image depicts negative valence or negative dominance the adjustment factor is added to the negative human-rights-abuse probability and subtracted from the positive human-rights-abuse probability. This is formally written in Algorithm 1.
Finally, when no b h were detected from the object detection branch, (1) is reduced into plain image classification as follows:

B. TRAINING
We approach human rights abuses classification as a cascaded, multi-task learning problem. Due to different datasets, convergence times and loss imbalance, all three branches have been trained separately. For object detection we adopted an existing implementation of the RetinaNet object detector, pre-trained on the COCO dataset [21], with a ResNet-50 backbone.
For emotion recognition in continuous dimensions, we formulate this task as a regression problem using the Euclidean loss. The two feature extraction modules described in Section III-A2, are designed as truncated versions of various wellknown convolutional neural networks and initialised using pretrained models on two large-scale image classification datasets, ImageNet [18] and Places [32]. The truncated version of those CNNs removes the fully connected layer and outputs features from the last convolutional layer in order to maintain the localisation of different parts of the images which is significant for the task at hand. Features extracted from these two modules (red and blue boxes in Fig. 2(b)) are then combined by a fusion module. This module first uses a global average pooling layer to reduce the number of features from each network and then a fully connected layer, with an output of a 256-D vector, functions as a dimensionality reduction layer for the concatenated pooled features. Finally, we include a second fully connected layer with 3 neurons representing valence, arousal and dominance. The parameters of the three modules are learned jointly using stochastic gradient descent with momentum of 0.9. The batch size is set to 54 and we use dropout with a ratio of 0.5.
For human rights abuse classification, we formulate this task as a binary classification problem. We train an end-toend model for classifying everyday images as human-rightsabuse positive or human-rights-abuse negative, based on the context of the images, for two independent use cases, namely child labour and displaced populations. Following the twophase transfer learning scheme proposed in [15], we finetune various CNN models for the two-class classification task. First, we conduct feature extraction utilising only the convolutional base of the original networks in order to end up with more generic representations as well as retaining spatial information similar to emotion recognition pipeline. The second phase consists of unfreezing some of the top layers of the convolutional base and jointly training a newly added fully connected layer and these top layers.
All the CNNs presented here were trained using the Keras Python deep learning framework [4] over TensorFlow [1] on Nvidia GPU P100.

C. INFERENCE
Object Detection Branch: We first detect all objects (including the person class) in the input image. We apply a threshold on boxes with scores higher than 0.5, which is set conservatively to retain most objects. This yields a set of n boxes b with scores s h and s o . These boxes are used as input to the emotional trait branch.
Emotional Traits Branch: Next, we apply the emotional traits branch to all detected objects that were classified as human. We feed each human box b h alongside the entire input image img to the VAD emotion recognition model. For each b h , we predict valence s v and arousal s a scores, and then compute the global emotional traits S GET h,img that describes the entire input image img.
Abuse Branch: If no human box b h has been detected, for example when a plain beach without people or kitchen appliances was given as input image, the branch predicts the two abuse scores S v and S n v directly from the binary classifier. On the other hand, when one or more people have been detected, the branch weights the raw predictions from the binary classifier based on the computed global emotional traits of the input image according to Algorithm 1.

IV. DATASET AND METRICS
There are a limited number of image datasets for human rights violations recognition [14], [15]. The most relevant for this work is HRA (Human Rights Archive) [15]. In order to find the main test platform on which we could demonstrate the effectiveness of GET-AID and analyse its various components, we construct a new image dataset by maintaining the verified samples intact for the two categories with more images, child labour and displaced populations. The HRA-Binary dataset contains 1554 images of human rights abuses and the same number of no violation counterparts for training, as well as 200 images collected from the web for testing and validation. Note that each abuse category is treated as an independent use case for our experiments. The dataset is made publicly available for future research.
Following [15], we evaluate GET-AID with two metrics accuracy and coverage and compare its performance against the sole use of a CNN classifier trained on HRA-Binary. We also evaluate the continuous dimensions using error rates -the difference (in average) between the true value and the regressed value.

A. IMPLEMENTATION DETAILS
Our emotion recognition implementation is based on the emotion recognition in context (EMOTIC) model [17], with the difference that our model estimates only continuous dimensions in VAD space. We train the three main modules (Section III-B) on the EMOTIC database, which contains a total number of 18,316 images with 23,788 annotated people, using pre-trained CNN feature extraction modules. We treat this multiclass-multilabel problem as a regression problem by using a weighted Euclidean loss to compensate for the class imbalance of EMOTIC. Table 1 shows the results for the continuous dimensions using error rates. The best result is obtained by utilising model ensembling, which consists of pooling together the predictions of a set of different models in order to produce better predictions. We pool the predictions of classifiers (ensemble the classifiers) by conducting weighted average of their prediction at inference time. The weights are learned on the validation data -usually the better single classifiers are given a higher weight, while the worse single classifiers are given a lower weight. However, ensembling the classifiers results in prolonged inference times, which causes us to turn our focus onto single classifiers for the remainder of the experiments. Following the two-phase transfer learning scheme proposed in [15], we fine-tune our human-rights-abuse classification models, Fig. 2(c), for 50 iterations on the HRA-Binary trainval set with a learning rate of 0.0001 using the stochastic gradient descent (SGD) [19] optimizer for cross-entropy minimization. These vanilla models will be examined against GET-AID. Here, vanilla means pure image classification using solely fine-tuning without any alteration.

B. BASELINE
To enable a fair comparison between vanilla CNNs and GET-AID, we use the same backbone combinations for all modules described in Fig. 2. We report comparisons in both accuracy and coverage metrics for fine-tuning up to three convolutional layers. The per-network results for child labour and displaced populations are shown in Table 2 and Table 3 respectively. The implementation of vanilla CNNs is solid: it has up to 65% accuracy on the child labour classification and up to 80% accuracy on the displaced populations classification, 29.82 and 44.82 points higher 1 than the 35.18 reported in the single-label, multiclass classification problem of [15]. Regarding coverage, vanilla CNNs achieve up to 64% for child labour classification and up to 67% child displaced populations classification, which is in par with the 64% maximum coverage reported in [15]. We believe that this accuracy gap is mainly due to the fact that [15] deals with a multiclass classification problem, whereas in this work we classify inputs into two mutually exclusive classes. This comparison showed that it is possible to trade coverage with accuracy in the context of human rights image analysis. One can always obtain high accuracy by refusing to process a number of examples, but this reduces the coverage of the system. Nevertheless, vanilla CNNs provide a strong baseline to which we will compare our method.
Our method, GET-AID, has a mean coverage of 64.75% for child labour and 46.11% for displaced populations on the  HRA-Binary test set. This is an absolute gain of 12.42 and 16.78 points over the strong baselines of 52.33% and 29.33% respectively. This is a relative improvement of 23.73% and 57.21% respectively. In relation to accuracy, GET-AID has a mean accuracy of 54% for child labour and 61.25% for displaced populations, which is an absolute drop of 4 and 5.5 points over the strong baselines of 58% and 66.75% accordingly. This indicates a relative loss of only 6.89% and 8.23% respectively. We believe that this negligible drop in accuracy is mainly due to the fact that the HRA-Binary test set is not solely made up of images with people in their context, it also contains images of generic objects and scenes, where only the sole classifier's prediction is taken into account.

C. QUALITATIVE RESULTS
We show our human rights abuse detection results in Fig.  5. Each subplot illustrates two predictions alongside their probability scores. The top of the two predictions is given by GET-AID, while the bottom one is given by the respective vanilla CNN sole classifier. Our method can successfully classify human right abuses by overturning the initial-false prediction of the vanilla CNN in Fig. 5

VI. CONCLUSION
We have presented a human-centric approach for classifying two types of human rights abuses. This two-class human rights abuse labelling problem is not trivial, given the highlevel image interpretation required. Understanding what a person is experiencing from his frame of reference is closely related with situations where human rights are being violated. Thus, the key to our computational framework are people's emotional traits, which resonate well with our own common sense in judging potential human right abuses. We introduce global emotional traits of the image which are responsible for weighting the classifiers prediction during inference. We benchmark performance of our GET-AID model against sole CNN classifiers. Our experimental results showed that this is an effective strategy, which we believe has good potential beyond human rights abuses classification. We hope this paper will spark interest and subsequent work along this line of research. All our code and data are publicly available.


Investigation of Nonlinear Multi-connected Systems of Automatic Control with Delay

The problem of controlling objects with time delay is quite complicated. The presence of a delay in the control loop leads to an increase in the phase shift, which can provoke instability of the system. The paper proposes a study of a nonlinear multi-connected system of automatic control (MSAC) of an object with a delay by frequency methods, based on the method of harmonic linearization and a system description of the characteristics of a multi-connected automatic control system. A nonlinear multi-connected automatic control system with delay in direct channels and cross-channels is considered. The nonlinear multi-connected automatic control system with delay consists of a set of identical separate subsystems and mutual connections between subsystems are single. With the help of frequency methods, the parameters of periodic motions (PM) in nonlinear homogeneous multi-connected automatic control systems with delay (amplitude and frequency of self-oscillations) are determined and their stability is estimated according to the proposed criteria. The paper presents examples that demonstrate the use of the proposed techniques to assess the stability of a nonlinear multi-connected system of automatic control of the object with delay both in direct communication channels and cross and finding the parameters (amplitude and frequency) of periodic movements. The results of the study were confirmed by simulation using MATLAB SIMULINK package. Keywords—multi-connected systems of automatic control, delay, nonlinear element, periodic motion, frequency method, stability.

I. INTRODUCTION
In many modern technical objects of control (OC) can observe the phenomenon of delay, consisting in the fact that with the beginning of the signal change at the input of the OC, signal at the output of the object begins to change only after a certain interval of time. For example of control objects with delays can be jet engines in transient modes, belt conveyors, rolling mills, drying and combustion processes, on a state -processes with a recycle, in particular, processes in grinding machines or processes in chemical reactors, at the output -control objects with more inertial measurement sensors than the object itself [1,14].
Generally, the influence of delay on the management processes is negative. Therefore, methods of studying control systems for the above objects, not considering the delay factor, are ineffective. The problem of designing similar control systems is even more complicated if the control object is multi-connected and nonlinear.
We know a large number of works, devoted to investigate of automatic control systems (ACS) with delay on control with one input and one output . A small number of scientific papers are devoted to multidimensional (multi-connected) control systems with delays [2].
In article proposes a study of a multi-connected nonlinear system of automatic control of the object with delays (control and output) by frequency methods, based on the method of harmonic linearization and system description of the characteristics of the MSAC.

II. RESULTS OF INVESTIGATION
We will propose in the future that this class of multiconnected systems consists of a set of identical separate and mutual connections between single subsystems (Fig.1). For systems with nonlinearities in direct channels of separate subsystems as a local characteristic of the harmonically linearized i-th subsystem, you can consider its transfer function Фi(s,а) in control mode [3]:    (2) where h i (i= n , 2 ) -generalized numerical characteristics of multidimensional elements of communication between separate subsystems, calculated by the formulas proposed in [4,5].
We introduce an algebraic equation of connection with respect to some variable х=Ф[q(a),q / (a),s], obtained from (2) : By frequency criterion [5]: the condition for the presence of periodic movements (PM) in this system is the passage of the local amplitude-phase frequency response (APF) Ф(a,j) harmonically linearized subsystem through one value ot the roots of the set {x i }, which play the role of critical points [6,7].
If you change the 0<a< deformed form of the hodograph APF Ф(a,j). In general case, the critical point is the own value x * i = i -j i , which go through the APF Ф(а,j). Then equality is true: Consider the homogeneous nonlinear systems with delay in the cross-ties. Then, in the characteristic equation (2) for a complete ACS consisting of n subsystems, the characteristic of the connection in the general form between k subsystems has the form: and local characteristic of the harmonically linearized i-th . The presence of PM in a nonlinear ACS with a delay in cross-linking the fulfillment of the condition: So, on the basis of equality (4), (5) it is possible to using frequency methods, to determine the parameters of periodic motion in nonlinear one-type MSAC and to evaluate their stability.

MOVEMENTS
Determine parameters, amplitude a* and frequency  i * , periodic movements. We distinguish the linear and nonlinear characteristics of the open subsystem from equation (4) [8]: j) in the equation (6): To solve the equation (7), we select the linear parts of the system and use two scalar equations: or we can wrte: 2. From equation (7) we distinguish the linear and nonlinear parts of the system:  4. We define the parameters a* and  * periodic movements at the intersection of the characteristics W з (j) and ) ( * ja Z н ; 5. To evaluate the stability of periodic movements by the frequency criterion [5]. Periodic movements in a nonlinear homogeneous MCACS will be stable if the amplitude a* by a increases (decreases) at the point corresponding to the intersection of hodographs W з (j) and ) ( * ja Z н , APF W з (j) does not cover a new point a=a*+a, otherwise, PM will be unstable [9][10][11].

Example 1.
We investigate a two-connected system (Fig.  1) for the presence of self-oscillations in the presence of a nonlinear element of the "saturation" type ( Fig. 2) and delays in subsystems.

Fig. 2. The nonlinear element of the type "saturation"
The coefficients of harmonic linearization for the characterization of F(z) of the type "saturation" is equal to: Consider the solution of this problem in stages.  . 3). 4. Figure 3 shows that PM is present in the system. At point 1 the hodographs intersect W з (j) and ) ( * ja Z н at delay τ=5 with parameters PM a=11.8, ω=0.23 ( fig. 4), at point 2 when the delay τ=0 with the parameters PM a=4.9, ω=0.5. (fig. 5). 5. We estimate the stability of periodic motions by the frequency criterion [5].
Periodic motion in the system is stable, because with increasing amplitude a* a in points 1 and 2 corresponding to the intersection of hodographs W з (jω) W(jω) locus Z н *(ja), APF W з (jω) W(jω) do not cover the new point a=a*+a.
The results confirm the transients (Fig. 4, 5), obtained by modeling the original system. To determine the parameters of PM and analyze their stability in a nonlinear ACS with delay in cross-links, consider the next steps: 2. From equation (7) we select the linear and nonlinear parts of the system: 4. We define the parameters a* and  * periodic movements at the intersection of the characteristics W л (j) and ) ( * ja Z н ; 5. To evaluate the stability of periodic movements by the frequency criterion [5].

Example 2.
Investigate the two-connected system ( Fig. 1) with nonlinear elements of the "saturation" type in subsystems (Fig. 2, example 1) and the delay in cross-linking for the presence of self-oscillations.
Transfer function of a two-dimensional object with delay:  . 6). 6. Define the parameters a* and  * periodic movements at the intersection of the characteristics W л (j) and Figure 6 shows that PM present in the systems, and cross the hodographs at delay τ=1 (Fig. 7). At delay τ=0 (Fig. 6) no PM, because hodographs do not cross.

Fig. 6. The intersection of hodographs Wл(jω) and Zн*(ja) without delay
Parameters a=4.9, ω=0.5, =0. 7. Assessing the stability of periodic movements on the frequency criterion it can be concluded, that in the presence of delay in the cross-links PM stable.
The results confirm the transition processes (Fig. 8, 9), obtained by modeling the original system. In the study of systems with delay is obtained of intersection of hodographs W л (j) and fig. 7). The amplitude and frequency are approximately the same as the results of the simulation ( fig. 7,9). Thus, the paper proposes an approach based on frequency methods and the harmonic linearization method, which allows to estimate the parameters and stability of periodic motions in the same type nonlinear MSAC with nonlinearities and delay in direct channels and crosslinks [12,13].

V. CONCLUSION
As a result of the study, a nonlinear multi-connected system of automatic control of the object with delay (control and output) by frequency methods, based on the harmonic linearization method and the system description of the MSAC characteristics was studied. The ACS with a nonlinear element and delay in direct channels and crosslinks is considered, provided that this ACS consists of a set of identical separate subsystems, the mutual relations between the subsystems are single. Used frequency-domain methods for the parameters of the PM to nonlinear similar MSAC, and assessed their stability. The results are confirmed by simulation in Matlab Simulink package.