s2fieldsofstudy:
Computer Science
Engineering

text:
Building extraction based on improved message passing network

In this paper, an improved building edge extraction method based on message passing networks (MPNNs) is proposed. In the field of high-resolution remote sensing image processing and analysis, the accuracy of building edge extraction is studied from three aspects: data set construction, network training and post-processing of edge probability map. Based on the WHU building data set, the proposed method can effectively extract the building edge, and can maintain high extraction accuracy. PPG Net, L-CNN and UNet++ algorithm is used to compare with the results of this paper. Through the analysis of experiments, the results show that the method proposed in this paper has advantages over PPG Net, L-CNN and UNet++ algorithm in the accuracy, recall and F1 score of region extraction.

Introduction
Building edge extraction is an important research branch in the field of three-dimensional reconstruction. It combines the latest research results of image processing, pattern recognition and other disciplines. It mainly analysis the high-resolution remote sensing image to realize the building edge extraction.
In recent years, convolutional neural network has shown good performance in computer vision target classification, image segmentation and other aspects. Many scholars have convolutional neural network to remote sensing field and gradually improved it.
First of all, the VGGNet [1] proposed by Simonyan et al. The feature map generated by the convolution module into a fixed-length feature vector, which is used for image-level classification tasks. But it cannot be directly applied to the extraction of buildings from remote sensing images. On this basis, Badrinarayanan et al. proposed a code-decoding structure, SegNet, for semantic segmentation. The up-sampling part of the decoder integrates the indexes in the maximum pooling of the corresponding encoder, which is conducive to feature recovery.
Later, Ronneberger et al. proposed a deep convolutional neural network (UNet) architecture for medical image segmentation in the paper. The same code-decoding structure was used. The encoding part was used to obtain the feature information of different scales, and the decoding part gradually restored the image resolution to achieve per-pixel classification. On the basis of UNet, Zhou et al. proposed a better medical image deep supervised segmentation network UNet++ [2], which improved the extraction efficiency.
The above research has improved the accuracy of building extraction from remote sensing images, but there are still some shortcomings. To further improve the effect of building edge extraction, this paper combined convolutional Message Passing network and extended Residual network, and proposed a building extraction method based on Message Passing network called DR-MPNNs (Dilated Residual Networks --Message Passing Neural Networks). In the second part, the first three convolution layers of the extended Residual Network DRN-C-26 are used as the basic network to extract features and get the feature map in the last layer; In the third part, MLP is used as the coding network to encode the message, encode it with the feature extracted in the second part, and then update it as the feature vector; 2) Edge verification module In the fourth part, the CNN is used as the decoder to align the feature channels, and then the candidate regions are classified by the maximum pooling layer, and the classification information is mapped to the candidate regions by the full connection layer, and the final confidence score is output.

Feature candidate region generation based on improved faster-RCNN
In this paper, faster-RCNN backbone network is replaced from vgg16 to rest net50 to generate candidate regions. The specific process is as follows: 1) Input RGB image; 2) VGG-16, whose backbone is replaced by RestNet-50, is used to generate a certain number of candidate regions with corners and edges, and each corner is regarded as an 8 × 8 candidate region.
As can be seen in Figure 2, in the improved model, the backbone network residual network block can naturally replace the original fast RCNN [5] and combine with the RPN network based on the candidate region. At the same time, the parameters of the whole network are calculated and shared based on the initial input image in the training process, which improves the overall detection performance of the network. The upper part of the graph is the backbone network rest net of feature

Feature extraction of deep residual network with extended convolution
Based on the deep residual network, this network model constructs the feature extraction module by introducing the extended convolution. In the process of feature extraction of building remote sensing image, the large receptive field with extended convolution can be used to further extract the image detail features, which greatly improves the learning efficiency and learning quality of residual network feature learning, and improves the learning efficiency the feature extraction quality of building remote sensing image is analysed. In this paper, the first three layers of DRN-C-26 model are used as feature learning module [6], and a network structure suitable for building extraction is designed to extract feature information.

Feature fusion coding based on message passing network
In DR-MPNNs, the update function in the standard message passing network[7] is replaced by MLP, so that the standard form of feature vector update is to encode the message and mix it with the current feature.
f represents the feature v vector related to the node, ( ) N v represents the set of adjacent nodes, and ";" represents the feature series connection.
In DR-MPNNs, CNN is used to encode messages. Edge pixels need to be clipped for feature fusion operation at decoder end. So, before inputting data into decoder, we use bilinear difference method to zoom the picture to 256 256 4   and then input data into the model.
By max pooling, all information is kept in the message. The features are merged across all adjacent nodes to encode the message, and then the feature vector is updated by the encoder

Edge verification module
The edge verification module uses CNN structure as the decoder, samples the output of the feature extraction module, and finally outputs the corresponding confidence score of the feature node, which is used to judge whether the node is a real building edge.
In the decoding process, the low-resolution feature map is deconvoluted; Then, it decodes by feature fusion; Finally, a convolution operation is used at the end of the decoder to convert the number of channels to 1, and then the ReLU activation function is used to output the confidence scores of candidate edges and corners.

Edge loss function
The task that indicates the authenticity of the predicted edge belongs to the classification task, which is divided into two categories. If you want to judge whether the feature of a predicted edge or corner is true or not, you can find out whether the feature of a predicted edge or corner is true. With the classifier, whether the output is a real edge is represented by or 1, where 1 means yes and 0 means no.
The loss function of the training phase adopts the two-classification cross entropy loss function, and the calculation formula is as follows: Where, H it is expressed as real value and Ĥ as predicted value.  used to increase weight.

Experimental environment and data pre-processing
The experiment is carried out under 64-bit Ubuntu operating system, and the network is constructed and trained by using the python framework. The hardware configuration is Intel Xeon (R) platinum 8160,250g memory, and the GPU is four NVIDIA P100. The improved res net parameters are pre trained on ImageNet, and the pre trained parameters are used to initialize the network model. The training uses the Space Net data set and the WHU building data set.
In the process of training, the batch size is set to 1, and the initial learning rate is set to 1. When the test loss is not reduced in four cycles, the training rate is reduced to 0.8. When the test loss does not decrease in 20 cycles, the training process will be terminated.

The results were compared and analysed
In the training process, due to the limitation of GPU memory, this paper only uses 815 buildings for training. The time of one, two and three iterations is about 26, 34 and 40 hours. Due to the limitation of computer hardware, only three iterations are carried out. Figure 3 show the comparison of the results of this algorithm on the test data set. It can be seen from the extraction results of WHU building data set in Fig. 3 that this method can extract the geometric structure of the roof in the building remote sensing image very well. After three iterations, this method can extract the buildings in the image more accurately. From the building extraction results of the left one, it shows that there is almost no missing edge and corner, the output structure is close to the real annotation structure; But in the extraction results in Figure 4, we can see that the recognition accuracy of the algorithm in this paper is slightly reduced due to the relatively complex building structure. Although the corners of the buildings in the image are extracted, the edges connected with them in the upper right corner of the image are not completely extracted. However, from the extraction results of most of the building data in the data set, the extraction results of this method are still good, and the correct geometric structure of the building can be extracted after three iterations

Contrast experiment
The results of this paper are compared with those of PPG Net [8], L-CNN and UNet++, and the results are shown in Figure 5. Figure 5 compare It can be seen from Figure 5 that the extraction results of PPG Net and the algorithm in this paper on the simple structure building (c) are more accurate and basically consistent with the real structure; L-CNN and our algorithm are consistent in the extraction of building (b); In the building extraction task shown in figure (d), the effect of UNet++ is the best of all the comparison algorithms. However, on the building image with simple structure, UNet++ produces some wrong nodes and too many redundant building edges. To sum up, in the visual comparison results, compared with other algorithms, the reconstruction effect of this paper is better than the extraction effect of previous algorithms, and the extraction of circular building structure is slightly lacking. The algorithm in this paper is compared with the contrast algorithm according to the accuracy, recall and F1 index of corner, edge and region, and the results are shown in Table 1 The data in Table 1 show that this paper achieves the best high-order (region) index in the no priori solution. For the angle and edge parameters, the effect of this algorithm is not always the best. Especially, on the edge data, the performance of L-CNN is slightly better than that of this algorithm, which is reflected in the recall rate and F1 value; In other results, this algorithm is better than the comparison algorithm.

4.Conclusions
Building target extraction has always been a hot and difficult problem in the field of remote sensing signal processing and pattern recognition. In this paper, aiming at the problems of low precision of large building extraction, imprecise building contour extraction and underutilization of high-resolution features in automatic building extraction of remote sensing image, a coding and decoding network based on message passing network is proposed. The convolution module for feature extraction introduces extended residual packet convolution to alleviate the problem of gradient disappearance of deep network. The multi-layer perceptron is used to connect the sub networks of different scales in parallel and continuously fuse the context information of the adjacent sub networks. The highresolution features of the input image are maintained in the whole process, which ensures the effectiveness of the subsequent features. Experiments show that, compared with the existing full convolution neural network PPG Net, L-CNN and UNet++, the model has a significant improvement in the effect of building extraction.


RWN: Robust Watermarking Network for Image Cropping Localization

Image cropping can be maliciously used to manipulate the layout of an image and alter the underlying meaning. Previous image crop detection schemes only predicts whether an image has been cropped, ignoring which part of the image is cropped. This paper presents a novel robust watermarking network (RWN) for image crop localization. We train an anti-crop processor (ACP) that embeds a watermark into a target image. The visually indistinguishable protected image is then posted on the social network instead of the original image. At the recipient's side, ACP extracts the watermark from the attacked image, and we conduct feature matching on the original and extracted watermark to locate the position of the crop in the original image plane. We further extend our scheme to detect tampering attack on the attacked image. Besides, we explore a simple yet efficient method (JPEG-Mixup) to improve the generalization of JPEG robustness. According to our comprehensive experiments, RWN is the first to provide high-accuracy and robust image crop localization. Besides, the accuracy of tamper detection is comparable with many state-of-the-art passive-based methods.

I. INTRODUCTION
Image manipulation leads to severe security threats, where misleading photojournalism, copyright violation or fabricating stories can be a means for some writers or politicians to potentially influence public opinion. Researchers have developed a number of schemes to detect various kinds of digital attacks, e.g., tampering [1]- [3], DeepFake [4] and cropping [5]- [8]. Among them, cropping is an extremely cheap and effective way to manipulate the layout of an image and alter the underlying meaning. Existing crop detection algorithms [5], [6] mainly focus on predicting whether an image is cropped. They are represented by detecting the exposing evidences of asymmetrical image cropping, e.g., the shift of the image center [5], and the inconsistency of JPEG blocking artifacts This work is supported by National Natural Science Foundation of China under Grant U20B2051, U1936214. Qichao Ying and Xiaoxiao Hu contribute equally. Corresponding author: Zhenxing Qian (zxqian@fudan.edu.cn). We are grateful to the anonymous reviewers for their constructive and insightful suggestions on improving this paper. [6], [7]. Despite the effectiveness of these works, different cropping behaviors in the real world usually lead to varied intentions. Crop detection is only a binary classification from this perspective, unable to distinguish benign crop behaviors from those malicious attacks such as discarding a visible watermark or removing a person. Van et al. [8] proposes an image crop localization scheme to study how image cropping causes chromatic and vignetting aberration. But the tiny traces only exists in high-definition images and can be easily destroyed by image processing, which makes the scheme hard to be applicable. Experiments show that the scheme cannot be applied on highly-compressed or low-quality images. Therefore, image cropping localization remains a big issue.
Watermarking [9]- [11] aims at hiding information imperceptibly into the host data for covert purposes. The technology focuses on robustness against possible digital attacks, such as image compression, noise adding, filtering, etc. Watermarking has been widely used in copyright protection and content authentication of images in multimedia. Recently, many novel watermarking schemes for image protection is proposed. Khachaturov el al. [12] proposes a watermarking-like adversarial method that prevents inpainting systems to perform normally on a protected image. In [13], the authors propose an adversarial attack method to cause errors in super-resolution model, including making the SR image has undesired style, be incorrectly classified in classification task, or generates wrong words in image caption task. The myth behind these schemes is that once the secret information successfully travel through the Online Social Network (OSN), the recipient can decode the information to administrate a number of tasks.
This paper explores the potential of robust watermarking on image cropping localization. We propose a robust watermarking network (RWN) for image crop localization by protecting the original image through watermarking. If the protected image is cropped, RWN is expected to locate the cropped region at the recipient's side. Watermarking is utilized to hide imperceptible traces for cropping localization which is resilient against common attacks, namely, JPEG compression, etc. We further introduce a tamper detector to deal with tampering attack. Therefore, compared to previous works [5], [8], the tough goal of finding natural-yet-fragile traces for image cropping can be circumvented by maintaining the overall image quality after watermark embedding.
We train an anti-crop processor (ACP) based on invertible neural network [14] to embed a watermark into a original image. The watermark is shared with the recipient. ACP produces a visually indistinguishable protected image, which is then posted on the social network instead of the unprotected version. On receiving the attacked version of the protected image, we use ACP again to extract the watermark, and conduct an efficient and classical feature matching algorithms (SURF [15]) between the original and extracted watermark to determine where the crop was positioned in the original image plane. We further use the tamper detector to predict the tamper mask, and the features within the predicted tamper mask is discarded to prevent mismatching. Besides, we explore a simple yet efficient method (JPEG-Mixup) to improve the generalization of JPEG robustness. We test our scheme by introducing man-made hybrid attacks. The results demonstrate that our scheme can accurately locate the cropped region. We also show the effectiveness of tamper detection by comparison with some state-of-the-art image forgery detection schemes [1]- [3].
The highlights of this paper are three-folded. 1) This paper presents the first high-accuracy robust image crop localization scheme. We innovatively use normalizing flows to build invertible function for image forensic problems. 2) With the embedded watermark, RWN can also conduct high-accuracy tamper detection, which is comparable with the state-of-the-art works. 3) We explore a simple yet efficient method (JPEG-Mixup) for improved JPEG simulation. Experiments show that JPEG-Mixup can remarkably improve the generalization of RWN on robustness against JPEG compression.
II. RELATED WORKS Crop Detection and Localization. Cropping is a simple yet powerful way to maliciously alter the message of an image. The existing crop detection algorithms [5]- [7] mainly focus on predicting whether an image is cropped. For example, Fanfani et al. [5] exploits the camera principal point insensitive to image processing operations. Yerushalmy et al. [6] detects whether there are vanishing points and lines on structured image content. But cropping different areas in a same image will definitely result in different semantic changes. Therefore, we need to predict the position of the crop. In [7], the block artifact grids (BAGs) are extracted blindly with a new extraction algorithm, and then abnormal BAGs can be detected with a marking procedure as a trail of cropping detection. For crop localization, Van et al. [8] investigates the impact that cropping has on the image distribution, and predicts the absolute location of image patches. However, [8] requires the images to be untouched and uncompressed so that the tiny traces are preserved. To address the issue of practical usability, this paper presents RWN for robust image crop localization that does not restrict image format or quality. Tamper Localization Great efforts have been made on combating daily image forgeries. Kown et al. [2] propose to model quantized DCT coefficient distribution to trace compression artifacts. Mantra-Net [1] uses fully convolutional networks for feature extraction and further uses long short-term memory (LSTM) cells for pixel-wise anomaly detection. In MVSS-Net [3], a system with multi-view feature learning and multiscale supervision is developed to jointly exploit the noise view and the boundary artifact to learn manipulation detection features. In this paper, we study the effectiveness of using watermarking to aid image forensics where we forgo the idea of finding universal traces. The shortcoming of these methods is that they only detect limited types of attacks and cannot generalize their performances well on images with a chain of combined attacks. We robustly hide traces into the original image instead of discovering a universal trace for a much easier forgery localization. Robust Watermarking. The uprising of deep networks have given birth to a series of novel watermarking methods with enhanced robustness [10], [16], [17]. Shin et al. [16] is the first in including a differentiable approximation to JPEG in the watermarking model. Later, Zhu et al. [10] proposes a powerful and comprehensive network that is robust against a variety kinds of attacks. From then on, subsequent watermarking works [17] design more sophisticated and accurate algorithm to simulate image post-processing attacks, especially the JPEG compression. However, these methods uses fixed quantization table for data compression. In contrast, the quantization table in real-world JPEG images can be customized and flexibly controlled by the quality factor as well as the image content. As a result, the neural networks can over-fit and lack realworld robustness. We explore a simple yet efficient method for improved JPEG simulation.

III. METHOD
A. Approach Overview Pipeline Design. The proposed RWN consists of five stages, namely, watermark embedding, image redistribution, tamper detection, watermark extraction and crop localization. We embed a watermark W into an original image I. The protected image I M is generated and we upload it onto the social cloud instead of the unprotected original image. The attacker generates the attacked image I A by freely adding three kinds of attacks (benign attacks, cropping, tampering) on I M . On the recipient's side, the tamper detector T predicts the tamper maskM on the attacked image to see which parts of the image are tampered, and we also extract the watermark asŴ from the received image. Afterwards, we rectify the extracted watermark byŴ =Ŵ · (1 −M) to discard the tampered contents. Finally, with the original watermark W as reference, we uses a feature matching algorithm to locate the position of the crop in the original image plane. An image is determined as cropped if there is valid matching result. Modeling. We regard the embedding and extraction as the inverse problem, even if the protected image is cropped and attacked. The formulation of the inverse problem is: subject to E(Î) = I G , E(Ŵ) = W G and E(G) =Ĝ. Here, R andR are the additional output and input of the network to keep the channel numbers as four. A denotes the function of image post-processing that attacks the hidden watermark. We letR = 0.5·1 where 1 is a matrix full of one. The ground-truth cropped watermark W G and the ground-truth cropped original image I G can be generated by sharing the crop mask used by A. Besides the invertibility, we regulate that E(I M ) = I for the imperceptibility of the embedding. Fig. 1 shows the pipeline of our scheme. We employ an Anti-Cropping Processor (ACP) P that jointly learns the paired function of both watermark embedding and extraction. We implement A using a differentiable attack layer that simulates the attacker's behavior. The rest of the components include a tamper detector T , a discriminator D and a feature matcher M, which is the SURF algorithm.

B. Network Implementation
Considering the efficiency of the invertible U-Net proposed in [18], we build our ACP on top of this architecture. The network consists of six invertible blocks each of which contains a Haar wavelet transformation and a double-side affine coupling. The network ends with a conditional split layer. The number of the input and output channels are four. The sizes of R andR are of the same as W. R andR are not required to be the same.
In the attacking layer A, we first convert the marked images into the corresponding 8-bit RGB-formatted image. We adopt the Straight-Through Estimator [19] on calculating the gradients. Then, we build differentiable methods to simulate the hybrid cropping attacks, which include benign attacks, cropping and tampering attack. The benign attacks are represented by JPEG compression, scaling, etc., which do not alter the semantic meaning of the image. We take the implementation from [10] except that we build our own JPEG simulator. For cropping, we randomly crop a portion of the protected image I M . For tampering attack, we first randomly select random areas using a binary matrix M inside I M , and generate the tampered image by where I irr refers to the source of the tamper. Although there are already many scheme which include a carefully-designed JPEG simulator, e.g., JPEG-SS [17], JPEG-Mask [10], MBRS [20], the real-world JPEG robustness of these schemes is still limited. We believe it mainly attribute to that the networks are over-fitted to a fixed compression mode. In this paper, we propose to apply the Mix-Up strategy [21] over these implementations so that the generated results are more flexible. We randomly sample a quality factor QF 1 that follows QF 1 ∈ [10, 100], together with a previous differentiable implementation of JPEG simulator J 1 , where J 1 ∈{JPEG-SS, JPEG-Mask, MBRS}. We correspondingly generate a pseudo JPEG image I 1 jpg by I 1 jpg = J 1 (I M , QF 1 ). In the same fashion, we generate k − 1 more pseudo-JPEG images, denoted as I 2 jpg , ..., I k jpg . Finally, the output of JPEG-Mixup is produced by mixing the pseudo-JPEG images with arbitrary contributing rates l .
where l = 1. The proposed mix-up based JPEG simulation (JPEG-Mixup) is fairly simple yet effective. Compared to previous schemes, the interpolation technique can generates more diversed pseudo-JPEG images for RWN. In the experiments, we show the performance gain in JPEG robustness using JPEG-Mixup in comparison with the other simulators.
Finally, we build the tamper detector T upon U-Net [22], a traditional image segmentation network, and implement the discriminator D using Patch-GAN [23]. Watermark Selection. We choose gray-scaled natural images with rich texture as valid watermark. Images with too much repetitive patterns will lead to multiple matching solutions for the SURF detector. Also, considering that the recipient can only blindly extract a partial watermark, we use those common images available on the Internet that can be easily downloaded by the recipient using image retrieval systems by referring to the extracted watermark. Fig. 2 shows some examples of recommended/not recommended watermarks.

C. Objective Loss Function and Training Details
For ACP, the first part of the loss is reconstruction loss L rec for E(Ŵ) = W G , E(Î) = I G and E(I M ) = I. E(·) is the expectation operator. L rec = F(I, I M )+F(W G ,Ŵ)+F(I G ,Î), where F is the 1 distance function. The second part is the nullification loss for E(R) =R. L ran = F(R,R). The third part is the adversarial loss, which further control the introduced distortion by fooling the discriminator. We accept the least squared adversarial loss (LS-GAN) [24]. The total loss for ACP is L G = L rec + α · L dis + β · L ran , where α and β are hyper-parameters. For the tamper detector, we minimize the binary cross entropy (BCE) loss between the predicted tamper maskM and the ground-truth mask M. L T = BCE(M,M).
We resize the images to the size of 256 × 256. The hyperparameters are set as α = 1, β = 8. We set k as 3, which is the generated pseudo-JPEG images used in JPEG-Mixup. The batch size is set as 16, and we empirically find that using Batch Normalization (BN) in our scheme outperforms using Instance Normalization (IN). We use Adam optimizer [25] with the default parameters. The learning rate is 1 × 10 −4 .

A. Experiment Setup
We train the scheme on the COCO training/test set [26] with automatically generated attacks. The dataset contains 117266 training samples and 40670 testing samples. The scheme is tested with human-participated attacks, where we have invited some volunteers to crop the provided marked images and perform further image post-processing attack on them. The crop rate is roughly δ ∈ [0.5, 1). During testing, we use real JPEG coder to compress the protected images. Evaluation Metric. We employ peak signal-to-noise ratio (PSNR), structural similarity (SSIM) [27] to evaluate the image quality, and F1 score [28] to measure the accuracy of crop/tamper localization. Benchmark. We compare RWN with Van et al. [8] for crop localization. In addition, we compare RWN with Mantra-Net [1], MVSS-Net [3] and CAT-Net [2] for tamper localization.

B. Real-World Performance of Crop Localization
Quality of the Protected Images. In Fig. 3, we randomly sample different pairs of images as the original and watermark. From the figures, generally, little detail of the watermark can be found. Though the magnified difference is visible, it does not matter if the difference is visible, only that the marked image is perceptually close to the original image. We have conducted more embedding experiments over 1000 images from the test set, and the average PSNR between the protected images and the original images is 36.23dB, and the average SSIM [27] is 0.983. Accuracy of the Crop Localization. Fig. 3 further shows the results of watermark extraction, feature matching and crop localization. In row (a), we only crop the protected image. We see that the crop mask is accurately predicted. As a result, even without the prior knowledge of the original image, we know the relative position of the attacked image in the original image plane. We have conducted more experiments over 1000 images under different crop rate and different kinds of benign attacks. Here for space limit, we take QF =70 as an example. The average performances are reported in Table I. Higher IoU (Intersection over Union) and SSIM [27] indicates more accurate localization result. The average IoUs are above 0.8 where the protected images do not undergo further attacks except cropping. The scheme is proven to be agnostic to the crop size in that the performance does not degrade significantly with larger crop rate.  Robustness. We test the robustness of our scheme by conducting image post-processing attack on the marked image. In the last two groups, tampering attack is introduced. In Table I the original images for [1]- [3]. We observe that although the images are tampered by a variety of random attacks, we succeed in localizing the tampered areas. In Table II, we provide the average results over 1000 images. The F1 score of RWN is 0.773 on uncompressed images and 0.736 on JPEG attacked images (QF = 80). The performance of Mantra-Net on JPEG images is much worse than that on plain-text images. We believe the reason is that less statistical trace is preserved in the compressed version. In contrast, the embedded watermark signal serves as the alternative trace for tamper detection, which is designed to resist benign attacks. The performance on JPEG images does not drop too much. Therefore, robust watermarking can successfully aid tamper detection by hiding crafted traces similar to [12], [13].

D. Ablation Study
Influence of Discriminator. In Test 3, we do not use the discriminator D and train the pipeline with only the reconstruction and the BCE loss. Fig. 5 shows the detailed comparison results. First, the baseline results outperform the encoder-decoder network design. Second, while MBRS [20] can provide decent robustness, the extraction performance is even better using our JPEG simulator. Specifically, the average SSIM betweenŴ and W G using MBRS [20] is 0.797 compared to 0.878 reported in Table.I. The Mix-Up strategy prevents the networks from being over-fitted to any single JPEG simulator, which helps the scheme significantly improve its real-world robustness. Third, without the discriminator, the extraction result is also worse than the baseline.

V. CONCLUSION
This paper presents a novel robust watermarking network for image crop localization. We use an invertible pipeline for watermark embedding and extraction, where the watermark is shared with the recipient. ACP produces a visually indistinguishable protected image, which is then posted on the social network instead of the unprotected version. On receiving the attacked version of the protected image, we use ACP again to extract the watermark, and conduct SURF feature matching algorithms to determine where the crop was positioned in the original image plane. We also extend our scheme to detect tampering attack on the attacked image. Experiments verify that RWN is effective in real-world application. Finally, we prove the necessity of the progressive recovery as well as the network design.


extfieldsofstudy:
Computer Science
Economics
Mathematics

text:
Sparse High-Order Portfolios via Proximal DCA and SCA

In this paper, we aim at solving the cardinality constrained high-order portfolio optimization, i.e., mean-variance-skewness-kurtosis model with cardinality constraint (MVSKC). Optimization for the MVSKC model is of great difficulty in two parts. One is that the objective function is non-convex, the other is the combinational nature of the cardinality constraint, leading to non-convexity as well dis-continuity. Based on the observation that cardinality constraint has the difference-of-convex (DC) property, we transform the cardinality constraint into a penalty term and then propose three algorithms including the proximal difference of convex algorithm (pDCA), pDCA with extrapolation (pDCAe) and the successive convex approximation (SCA) to handle the resulting penalized MVSK (PMVSK) formulation. Moreover, theoretical convergence results of these algorithms are established respectively. Numerical experiments on the real datasets demonstrate the superiority of our proposed methods in obtaining high utility and sparse solutions as well as efficiency in terms of time usage.

INTRODUCTION
Portfolio management is a fundamental and challenging task for investors. One significant progress was made by Markowitz, who developed the mean-variance (MV) framework [1]. In the MV framework, the investors' purpose is to maximize their expected profit (i.e., mean return rate, or first moment) and minimize the corresponding risk (i.e., variance of portfolio, or second moment). Due to transaction costs, budget constraints, or even mental costs, investors may pick only a small number of assets out of all possible candidates, thus resulting in the sparse portfolios [2]. Some optimization methods have been developed for the MV framework with cardinality constraint [3][4][5].
The MV framework is based on the assumption that the returns follow a Gaussian distribution or investors have a quadratic utility [6]. However, in real financial markets, the Gaussian distribution assumption is seldom satisfied because asset returns are usually asymmetric and heavy-tailed [7,8].
To overcome this drawback, several studies took the third and fourth moments (i.e., skewness and kurtosis) into consideration, as they can better capture the asymmetry and heavy-tailed properties of a distribution [9][10][11]. They then proposed the mean-variance-skewness-kurtosis (MVSK) framework, which aims to maximize the mean and skewness while minimizing the variance and kurtosis. Assuming that investors have constant relative risk aversion (CRRA) [12] preferences, the MVSK model is the fourth-order Taylor expansion of the expected utility. Although the MVSK framework is more accurate than the MV framework, it requires solving a nonconvex optimization problem, which leads to computational challenges.
Recently, with the development in the optimization field, some algorithms were proposed to solve the MVSK-related problems. A difference-of-convex algorithm (DCA) was developed in [13] for the MVSK framework. Further improvement based on the difference-of-convex-sums-of-squares (DC-SOS) was proposed in [14]. Quite recently, to approximate the non-convex objective function more tightly, two algorithms based on the majorization-minimization (MM) and successive convex approximation (SCA) were proposed in [15]. By contrast, the cardinality constrained MVSK (MVSKC) framework has not received much attention yet. The work [16] considered a bi-objective optimization problem based on the trade-off between expected utility and cardinality. It is shown that there are gains in terms of outof-sample certainty equivalent and Sharpe ratio for certain cardinality levels. However, they applied a derivative-free solver based on direct multi-search [17], which may be cumbersome and sensitive to the initial point.
Our contributions. In this paper, based on the available structures of the MVSKC model, we consider integrating the cardinality constraint into the objective function via a penalty technique, which leads to the penalized MVSK (PMVSK) problem. Further, we present three approaches for solving the PMVSK problem, namely proximal difference-of-convex algorithm (pDCA), pDCA with extrapolation (pDCAe), and the successive convex approximation (SCA). Theoretically, we establish the convergence of pDCA and SCA to a stationary point of the problem at hand. Numerically, we demonstrate through experiments on a real dataset that our methods are able to obtain sparse solutions with lower objective function values. It is worth mentioning that SCA outperforms pDCA and pDCAe in terms of running time, which indicates that it is more favorable for large-scale problems.

PROBLEM FORMULATION
Suppose that the returns of N assets are given byr ∈ R N and the portfolio weights are given by w ∈ R N . The expected return of this portfolio, i.e., the first moment, is φ 1 (w) = E(w r) = w µ, where µ = E(r) is the mean return vector. We denote r = r − µ as the centered returns. The q-th central moment of the portfolio return is E[(w r) q ], from which we have the following: • The second moment, a.k.a. variance, of the portfolio return is is the co-skewness matrix. • The fourth moment, a.k.a. kurtosis, of the portfolio return is is the co-kurtosis matrix. Based on the above definitions, the MVSKC problem [16] is given as follows: min (1) where λ 1 , λ 2 , λ 3 , λ 4 > 0 are parameters to balance the four moments of the portfolio return, w 0 is defined as the number of non-zero elements of w, k < N is an integer controlling the number of assets to be selected, and α > 0 is to bound each element of w.
It is easy to verify that f is non-convex and twice continuously differentiable. The gradient and Hessian of each term in f are given in the following lemma.

Lemma 2. Under the constraints in
When it comes to the cardinality constraint, inspired by its DC property [18] w is the largest-k norm (i.e., the sum of the k largest elemens in absolute value), we transform (1) into the following PMVSK problem by taking the cardinality constraint as a penalty term: Here, ρ > 0 is the penalty coefficient.

ALGORITHM DESIGN
In this section, we propose three algorithms-pDCA, pD-CAe, and SCA-to solve the PMVSK problem (3).

pDCA
The main idea of pDCA is to successively construct a global upper bound of the objective function in (3) by linearizing the concave part. In the j-th iteration, we solve the subproblem where s j is a subgradient of w [k] at w j and can be computed efficiently through the following two steps: 1) sort the elements of |w j | in decreasing order, i.e., By introducing a new variable u ∈ R N , problem (4) can be cast as a convex quadratic programming (QP) problem: which can be efficiently solved by quadprog in MATLAB. In the rest of the paper, we will always utilize this technique to cast the 1 norm into linear inequality constraints. The complete description of pDCA is summarized in Algorithm 1.

pDCAe
Despite the common use of pDCA in many applications, it can be slow in practice [19]. To accelerate pDCA without increasing too much the computational cost, we adopt an extrapolation technique similar to that in FISTA [20] when approximating the concave part of the objective function. In the j-th iteration, we solve the subproblem where s j is a subgradient of w [k] at w j . The difference between (4) and (6) is that the latter linearly approximates the smooth concave part at y j , which is a carefully designed extrapolation point of w j−1 and w j . Note that a direct application of the extrapolation technique proposed in [19] for solving (3) can only lead to pDCA due to the nonconvexity of f . We summarize the pDCAe in Algorithm 2.

SCA
In this subsection, instead of utilizing the DC decomposition as in pDCA and pDCAe, we construct a strongly convex function that is not necessarily a global upper bound of f ncvx (w) but can lead to a tighter approximation. This is a Set y j = w j + β j (w j − w j−1 ).

3:
Perform the backtracking line search (8) to obtain the stepsize γ j .

7:
break 8: end if 9: end for kind of successive convex approximation strategy, and we refer to it as SCA for brevity.
Specifically, we havẽ and τ w > 0 guarantees thatf ncvx (w, w j ) is strongly convex. In addition to constructing a strongly convex local upper bound for f ncvx (w), we linearize the concave part −ρ w [k] . In the j-th iteration, we solve the following convex QP problem to getŵ j+1 : min Note that since problem (3) is non-convex and non-smooth, the traditional choice of stepsize [22] does not work. Hence, we perform a line search to guarantee the convergence of SCA. Specifically, given scalars 0 < c < 1 and 0 < β < 1, the stepsize γ j is set to be γ j = β mj , where m j is the smallest nonnegative integer m satisfying It is guaranteed that the stepsize γ j determined by (8) is nonzero and satisfies f p (w j+1 ) < f p (w j ) [23, Proposition 2]. The complete SCA algorithm is given in Algorithm 3.

THEORETICAL ANALYSIS
In this section, we provide convergence guarantees for our proposed pDCA and SCA in the following theorems. Theorem 1. [19, Theorem 4.1, Proposition 4.1] Let {w j } be the sequence generated by Algorithm 1. Then the following statements hold: • lim j→∞ w j+1 − w j = 0.
• Every limit point of {w j } is a stationary point of (3).
where Ω is the set of limit points of {w j }. Theorem 2. [23, Theorem 3] Let {w j } be the sequence generated by Algorithm 3. Then, every limit point of {w j } is a stationary point of (3).
We first show the convergence curves w.r.t. the iteration number and CPU time. Fig. 1 shows the objective value curves of pDCA, pDCAe, and SCA with ξ = 10 and the same random initialization. Fig. 2 presents the corresponding results when ξ = 5. Here, we only show the first 150 and 40 iterations of the algorithms for ξ = 10 and ξ = 5, respectively. We can observe from Fig. 1 (a) and Fig. 2 (a) that SCA converges fastest among the three algorithms, and pDCAe converges faster than pDCA. Since the QP subproblems (5)- (7), which constitute the main computational burden of the corresponding algorithms, require roughly the same cost, fewer iterations indicate less running time. This is verified in Fig. 1  (b) and Fig. 2 (b), which show that SCA takes the least time among the three methods.
Next, we compare our proposed algorithms with several baselines. One commonly used strategy, which we denote as relaxed MVSKC (RMVSKC), is to first relax the cardinality constraint to 1 norm constraint [15] (it is worth emphasizing that this kind of relaxation is not able to get a sparse solution) and then project the resulting solutionw * to satisfy the original constraints of MVSKC (1). The projection step amounts to solving To this end, we introduce a binary variable u to rewrite the cardinality constraint, leading to −αu ≤ w ≤ αu, and 1 u ≤ k. However, the resulting mixed-integer quadratic programming problem is challenging even with Gurobi (e.g., it takes more than 5 hours using Gurobi withw * ∈ R 50 ). Hence, we use the genetic algorithm (GA) instead. Utilizing the same transformation technique, we can also solve the MVSKC (1) via GA. From Table 1, we can see that pDCA, pDCAe, and SCA are superior in terms of the objective values obtained. In addition, it is achieved with much less time when using SCA. Table 1. f value and time usage of different methods with ξ = 10 and ξ = 5. Each method is tested five times using different initializations and the results are then averaged. 6. CONCLUSION In this paper, we considered the problem of high-order portfolio optimization with cardinality constraint. We proposed to recast the cardinality constraint into a penalized term and then developed three methods, namely pDCA, pDCAe, and SCA. Convergence results were established. Extensive experiments showed that our methods got lower objective values and are more efficient than the baselines.


Brain Computer Interface Drone

Brain-Computer Interface has emerged from dazzling experiments of cognitive scientists and researchers who dig deep into the conscious of the human brain where neuroscience, signal processing, machine learning, physical sciences are blended together and neuroprosthesis, neuro spellers, bionic eyes, prosthetic arms, prosthetic legs are created which made the disabled to walk, a mute to express and talk, a blind to see the beautiful world, a deaf to hear, etc. My main aim is to analyze the frequency domain signal of the brain signals of 5 subjects at their respective mental states using an EEG and show how to control a DJI Tello drone using Insight EEG then present the results and interpretation of band power graph, FFT graph and time-domain signals graph of mental commands during the live control of the drone.

Introduction
The brain computer interface (BCI) technology makes the possible manipulation of embedded systems using signals generated by brainwaves. A characteristic of the BCI system can easily capture brain signals generated by neural activities, it can also recognize differently firing neural activity patterns, and these signals can transform them into useful commands [1]. These commands can be utilized to control the machines or the devices. BCIs are most commonly applied in prosthetic limbs for paralyzed patients, exoskeletons, robotics, autonomous vehicles, virtual keyboard and computer games [2]. The BCI system can be classified as invasive, non-invasive (these are classified based on the location of placement of EEG biosensors). Noninvasive BCIs are based on electroencephalography (EEG) to record the brain activities using a series of biosensors disposed on the scalp will be able to measure the potential generated by the electrical activity of thousands to billions of cortical neurons inside our brain [3]. Our study is focused on noninvasive BCI using an Electroencephalogram. The neocortex is a convoluted surface which resides at the top of the brain. It is about ⅛ of 1 inch thick. It has 30 billion neurons arranged in 6 layers. Each neuron makes around 10,000 synapses with other neurons, which results in around 300 trillion connections in the total [4]. The most common type of neuron in the cortex is the pyramidal neuron, populations of which are arranged in columns oriented perpendicular to the cortical surface. The surface of the cortex is convoluted, with Fissures sulci, Ridges gyri. The neocortex exhibits functional specialization. Each area of the cortex is specialized for a particular function. The occipital areas near the back of the head specialize in basic visual processing [5]. The parietal areas towards the top of the head specialize in spatial reasoning and motion processing [6]. Visual and auditory recognition occurs in the temporal areas (towards the sides of the head) while frontal areas are involved in planning and higher cognitive functions. Inputs to a cortical area mainly come into the middle layers, Outputs of the cortical area leave from the upper and lower layers [7]. Based on these input-output patterns, the cortex roughly acts as an organized network of sensory, motor areas. Coming to the EEG, it is a device that extracts, organizes, and filters the electric signals which exist due to the neural firings (action potential) of the brain it is used for various diagnosing purposes, it is a popular non-invasive technique for recording the neuronal firing using electrodes placed on the scalp. The currents originating deep in the brain due to the firing of the neurons are not detected by EEG because the voltage fields will fall off with the square of the distance from the source [8]. The time domain signal displays the signals from different electrodes in a graph known as electroencephalography. EEG will reflect the summation of postsynaptic potentials occurring due to firing of thousands of neurons which are oriented radially to the scalp but not due to tangential electrodes. The spatial resolution of EEG is poor in a square centimeter range because of the impedance caused due to the presence of skull, scalp, CSF, meanings [9]. These layers' act as volume conductors and low pass filters to smear the original signals, whereas coming to the temporal resolution is good at the range of milliseconds [9]. This time domain signal from EEG is then converted into frequency domain signal using different transforms in the signal processing such as discrete Fourier transform, fast Fourier transform, etc. The amplified frequencies (according to the fast Fourier transform) which are extracted from brain by electroencephalogram into 4 ranges they are theta (θ) which ranges from 4 Hz to 8 Hz, alpha (α) which ranges from 8 to 12, beta (β) ranges from 12 to 25 Hz and finally gamma (γ) ranges minimum from 25 Hz to maximum of 45 to 75 Hz [10]. After performing many experiments on many patients specifically to observe the type of waves and the amplified frequencies (when will they occur, in what state of patient these waves can be observed) they have presented a generalized form of relation between their frequency ranges and normal human functions. When a person is ready or about to perform tasks or if he/she is in an alert state then more percentage of α frequency waves are generally observed and if a person is task oriented or if he is in a busy state or anxiously thinking or actively concentrating then high percentages of β frequency waves are generally observed, if a person is performing high motor functions, or if the person is switching the activities during multitasking then high percentage of γ frequency waves are observed mostly in the frontal lobe of the human brain. After performing several tests, I was able to predict that in my meditation state a high percentage of θ frequency waves were observed even in a sleeping state where the mind is in a relaxed condition there are high percentages of θ frequency waves. The Emotiv Insight is an EEG Brain wear device which is composed of five sensors that are projected to acquire and measure the key activity from the entire functional areas of the cortex. The device can provide raw EEG Signals, Mental Commands (conscious thoughts), Facial Expressions -Facial mimicry and Measurements of performance of the brain. The principal key characteristics of this scientific design is the dynamic brain-computer interface interactions with more degrees of freedom for controlling physical and virtual objects. The device accurately identifies mental states and emotion such as Engagement, Focus, Excitement, Meditation, Relaxation, Stress [11]. There is a possibility to build brain activity models in real-time based on spatial resolution. A deeper perspective on specific patterns of an individual's brain activity. The very important problem in EEG processing is low signal to the noise ratio since there are many layers between the neural cortex and the scalp and also due to the artifacts which result with great amplitude, the solution to minimize the noise is we can use filtering techniques and noise reduction techniques to remove the noise from the raw EEG data and extract the brain activity signals [12]. Since the EEG signal is non-stationary signal we use classifiers which are trained on user data (which is limited, this is also another main problem) we can generalize those results poorly to the already trained data on the same individual (different for different individuals because of physiological differences, this also limits the use of EEG applications) at different times. The accuracy might increase as we increase the number of training sessions but generalizing for subjects, i.e., to handle inter-subject variability processing pipelines with domainspecific approaches are used in order to clean, extract relevant features and classify (Riemannian geometry based classifiers, adaptive classifiers) EEG data. The subset of Machine Learning which is Deep Learning is used mainly to extract the features, Recently CNNs (convolutional neural networks) are used to simultaneously extract the feature and the classifier in order to achieve end to end supervised feature learning. Hence the devices use CNNs and recurrent neural networks of 3 to 10 layers in total [13].

EEG
In most of the Neurosurgery hospitals or the hospitals where the diagnosis of the human brain takes place, they mostly use EEG which have very high Temporal and spatial resolutions, devices also occupy huge amounts of space. But in the study of Brain Computer Interface wherein we are required to develop the applications to control the external environment in such a way that the patient or subject or the user must not be facing the adaptability issues to significant extent [14]. If we observe the EEG devices which are used in the Hospitals, they generally require minimum of 2 to 3 hours for just equipping the device or placing the electrode in proper locations over the scalp, where in the contact optimizing fluid needs to be applied all over the head and the electrodes needs to be placed which is a tedious and complex process. But recently the company Emotiv has come up with a very sophisticated and easily adaptable device named as INSIGHT. This Insight device is equipped with very efficient specifications, where in which It has 3 axis gyroscope and 3 axis Magnetometer which are very helpful to remove the artifacts due to the head movements which is absent in the case of hospital EEGs (therefore the patients is instructed not to move their head/to avoid the motor movements). There are 5 important electrodes installed in the device which are made of semi-dry polymer. The electrode locations are 2 in the frontal region, 2 in the temporal region and another is at the central peritoneal region. The nomenclature of these electrodes are given according to the international 10-20 system. Where the frontal left electrode is named as AF3, frontal right electrode is named as AF4, temporal left electrode is named as AT7, temporal right electrode is named as AT8 and peritoneal central electrode is named as Pz plus DRL reference mastoid electrode on the left, the channels with built in digital 5th order sinc filter, bandwidth of 0.5-43 Hz with digital notch filters at 50 Hz and 60 Hz, 2.4 Hz wireless connectivity, 8400uV is the dynamic range (input referred), sequential sampling, 128 samples per second is the sampling rate, 14bit motion resolution, the electrodes are semi dry polymers, with 14bits 1 LSB = 0.51muV (16 bit ADC, 2 bits instrumental noise floor discarded) EEG Resolution. Main principle of EEG devices is the differential amplifier which has 2 inputs Input 1 and Input 2. From both the inputs the information is fed, the information can be mathematical or any signal it gives the output resultant signal which is the relative deference chunk of signal. Mounting the device over the subject's scalp is a crucial procedure, where saline glycerol solution is applied to the semi dry electrodes in order to maintain the optimum contact quality with the scalp of the head. The electrodes were being placed in active locations according to the International 10-20 system protocol and the optimal 100% contactivity is ensured. The subject is instructed to not to move to avoid motion artifacts and stay focused on mental commands. Connection between the insight and the laptop is achieved by following the EmotivApp protocol, connection can be established either using Insight dongle or through the Bluetooth connection. Authentic Interpretation of EEG requires a very high amount of training and experience in analyzing and predicting the Graphical data. The most important set of rules which needs to be followed while analyzing the EEG data is the type of montages used, the time domain of the subject state compared to the EEG data at that current time in order to check that there are no external noises or movements made by the subject. There are different types of montages such as longitudinal bipolar Montage, longitudinaltransverse bipolar montage, circumferential bipolar montage, temporal bipolar montage, Cz referential Montage, Ipsilateral ear referential Montage in order to analyze the patient's cognitive state in different ways in order to predict the correct result. In the EmotivePro app in MacBook Air laptop to record raw EEG data for the experiments I have used an Ipsilateral ear referential montage where one input of the differential amplifier will be a DRL reference mastoid electrode and the other input will be any one of the active electrodes. I set up the channel spacing to 400 μV minimum amplitude to À100 μV, maximum amplitude to +100 μV, with a high pass filter.

EEG pattern in eye blinks
As we know that the cornea is positively charged and the retina is negatively charged, when there is a movement of the eyes towards upwards due to the bell's phenomenon Figure 1, when eyes are closed this will result in the abnormal signals in the frontal electrodes [15]. There is an upward peak then downward peak when eye is blinked once, I blinked the eye continuously therefore crist and troughs continuously occurred only at the AF3, AF4 frontal electrodes in the Figure 2, this is because of the bells phenomenon when the eyes are closed the cornea moves up due to which there is a change is potential observed near the frontal electrodes, there is an upward peak at the first when the eyes are closed which implies that there is a relative positive potential of AF3/AF4 with the reference electrode and when eyelids opened there is a downward peak which implies that there is a relative negative potential of AF3/AF4 with the reference electrode.

Meditation
To cross check a theoretical aspect of the brain frequencies also in order to validate and prove that the device is authentic, we performed meditation mental tasks for diverse subjects and analyze their signal frequencies of their respective time domain signals using the Fast Fourier Transform (FFT) in the MATLAB.

Test dataset acquisition and observation
The test is being conducted in an anechoic chamber as we can observe at the background of the subjects in Table, in order to minimize the external noise and to isolate the experiment. The subject's voluntary consent has been taken prior for performing this particular test experiment. Also, the subjects are strictly advised to be in a relaxed mental condition for 10 minutes before the test perusal and before 30 seconds of the performance of the test, the subject's eye movements (closed state and opened state) also analyzed by the EmotivPro application in order to remove the eye blink artifacts. All subjects have performed the test successfully as instructed.
As the subject's experiments begin, their mental state is being captured in the camera and the state at which the subjects are present is being noted and presented in the mental state column of Table 1. After ensuring the optimum contact quality, the EEG data following common reference montage is analyzed in the live, with EmotivPro license, it is possible to record the current EEG data of its respective electrodes.
After performing the tests on the subjects, the raw EEG data which is recorded during experimentation is stored in the EmotivPro application cloud. This recorded data is exported to the client server system in the .csv format, Figure 3 these files can be assessed through the links provided in the above Table 1 for the respective subject test.
This .csv file contains the recorded potentials of all 5 channels (AF3, AF4, T7, T8, Pz) at their respective timestamp. This data is copied and imported into the workspace of MATLAB. Database toolbox is used to read, write, import and export the data of .csv files Figure 4. Digital signal processing toolbox is used for converting time domain signal to frequency domain signal.
The main principle of fast Fourier transform is it converts the time domain signal into frequency domain signal. As the raw data is injected into the FFT in matlab, it analyzes the frequencies of the time domain signals. The xfft vs. absolute part of the FFT gives us the frequency domain signal. In this case, there is a large amount of noise observed in the frequency domain graph for the test dataset. Hence, in order to remove the noise and the high peak which is near to zero, the smoothing filters were applied independently and a band pass filter is also applied between 4 to 45 Hz, as the result of the experiment lies within that particular frequency range. 1. The length of the signal is assigned to variable = nfft, 2. In order to have a good resolution of the signal the nearest 2 power value of the length nfft is assigned to the variable nfft2.
3. The fast Fourier transform filter is applied and assigned to the variable fft.
4. In the absolute part of the fft, we can observe a symmetric curve with frequency x axis and amplitude magnitude of the y-axis. 5. In order to remove the symmetry about the line parallel to the y-axis and only consider the initial half part, we only consider half of the nfft2 length and that half portion is assigned to the variable = fff.
6. For the x component of the fft signal is assigned to the variable xfft, where input values of the sample frequency with half of the length of the data is assigned to xfft.
7. The graphical plot is made with xfft as x-axis and fff as y-axis for that particular time period. Savitzky Golay filter, moving average filter, smoothing filter were applied and analyzed independently to all of the test files. After applying filters to the signals, their respective frequencies are extracted, recorded and saved and the smoothing filter is considered to be the best fit filter at a window of 100 neighboring samples for our test datasets.

Test results
The smoothing filter, band pass filter along with FFT in combination resulted in best output for the frequency domain analysis and the results are observed, noted and represented graphically in Table 2 as there is time domain graph at the top and frequency domain graph at the bottom in the experimental analysis column in Table 2.
As most of the subjects are performing the meditative test they should experience a very high amount of theta waves they ranged between 5.1 to 7.6 Hz i.e., there will be a maxima peak within the range of 5.1 to 7.6 Hz in the frequency domain signal and also when the subject performs a demanding motor task he experienced a high amount of gamma waves with maxima peak at 35.9 Hz (Table 3).
Hence, the test results prove that the subject's mental state falls within specific range when a specific mental task is performed. This ends the validation part of the Emotiv Insight EEG device.

File Name Experimental analysis
InsightTest Onesimus10

Common EEG patterns
After connecting the device with the EmotivApp, the EmotiveBCI application is used to train mental commands. After training, the live mode is switched ON to control a cube with our Imagery thoughts. Next those live mental commands should be extracted from the application to integrate with the drone. This is achieved through cortex API documentation, for the ease I have edited the code in python language. Then simultaneously those commands should be integrated to a dji tello drone. This is achieved through dji tello API documentation. Finally a link between the drone and Insight brainware is achieved.

Mental command training
Giving mental command to the EmotivBCI application will result in the movement of the object in the desired direction. Initially numerous training needs to be done to the device in the initial stage to get good desired output. For example during an object movement test when the subject is thinking to move the cube towards the right direction we can observe this in the Figures 5 and 6 represented below that there is a movement of the cube towards the right direction. Each of the Neutral, moving left, lift, drop mental commands were trained for 10 times.

Extraction of mental commands and assigning them to the drone
The python code is built in Atom software editor in Dell Inspiron laptop. Jason, websocket, ssl, time, win32, requests, pyautogui, socket, keyboard, threading are the libraries used in the code. The control of the drone with the computer is achieved using DJI Tello drone protocol Figure 7, where the drone is connected to the local wifi and the laptop should also be connected to the same wifi using a TP-Link USB Wi-Fi Adapter which is used for PC(TL-WN725N) it's an N150 Wireless Network Adapter for laptop, UDP protocol is used to make an interface between the computer system and dji tello drone, it should be explicit and bind to a local port on our computer where tello can send messages. The functions that listen for messages from tello will be printed on the screen. Connection between the insight and the laptop is done according to the Emotiv App protocol, connection can be established either using Insight dongle or through the Bluetooth connection. After connecting the device with the EmotivApp, the EmotiveBCI application is used to train mental commands. After training, the live mode is switched ON to control a cube with our Imagery thoughts. Next those live mental commands should be extracted from the application to integrate with the drone. This is achieved through cortex API documentation. Then simultaneously those commands should be integrated to a dji tello drone. This is achieved through dji tello API documentation. Finally a link between the drone and Insight brainware is achieved. Coming to the Insight, InsightHandler class is used then the web connection is acquired, In order to get the approval form the Emotiv app we have to generate client id and secret then we have to give the approval in the emotive app. After approving then authorisation happens to generate the token this token will create the session and loads the profile from the cortex app, then we are going to start and stream the mental commands data in our terminal, these commands will be integrated with the our keyboard clicks, these clicks will intern control the drone as a controller, so basically as we are thinking of a mental command which will be controlling our computers keyboard this will intern control the drone.

Result
After several attempts the drone control was modified and whenever the power meter of a mental command crosses 50% then that specific keyboard key will be  clicked. For example if the subject is in a neutral state and suppose he is thinking of a lift command and simultaneously if the power meter of that lift mental command is at 50% or crosses 50% then that command will be executed and the computer keyboard key which is assigned to the push will be clicked this will intern transmits the thrust command from the laptop and drone will receive this command and lifts off the floor. In the terminal window and live mode in the Mental command window controlling a cube are represented in the Figures 8-10. The cube is in the     neutral position therefore we can observe neutral in the terminal window in Figure 8. When the person is thinking of lifting the cube, the cube changes from its neutral position and moves upwards therefore we can observe change from neutral to lift in the terminal window, Figure 9. The cube is in the neutral position therefore we can observe neutral in the terminal window, Figure 10.
The drone is in the resting state when the mental command of the subject is neutral, Figure 11. The drone takes off when the mental command of the subject has changed from the neutral to lift, Figure 12.

Conclusions
EEG patterns such as motor movements, eyes movements, meditation, sleeping tests are recorded and analyzed. A mathematical model is developed in the MATLAB using concepts such as signal processing in order to analyze the theoretical data of the brain frequencies at different mental states. The validation of the device by analysis of raw signal has been performed using signal processing methods such as fast Fourier transform simultaneously applying filters to extract the signal of interest by removing noise. These EEG patterns are analyzed on 5 different subjects and cross validated the data with the theoretical brain frequencies data. After these above experiments, Interface between DJI Tello Drone and Emotiv Insight BCI headset was achieved and the Drone was controlled with 2 mental commands moving up and moving down from the neutral state.


Latency Aware Distributed ADMM over Networks

Methods based on the alternating direction method of multipliers (ADMM) has attracted academic attention because of its excellent convergence performance and potential application scenarios in many machine learning or optimization fields. However, classical distributed ADMM algorithm assumed ideal network communication, which do not consider the impact of network delay on computing performance. In this paper, based on the strategy of selecting bridges with lowest network latency and appropriate iterative process, we propose a latency aware distributed ADMM algorithm to alleviate the impact of network delay. The classical algorithm and proposed algorithm are tested and compared in real network scenarios. Experiments show that the proposed algorithm reduces the running time and improves the computing performance. Especially in networks with large delay, the effect is more obvious.

Introduction
Methods based on the alternating direction method of multipliers (ADMM) is a simple method to solve the problem of decomposable convex optimization, since Boyd proves that it is suitable for solving large-scale distributed optimization problem [1], this method has attracted academic research, and is widely used in various fields in recent years, including distributed machine learning, image processing, communications network optimization, Power system scheduling, etc. [2][3][4][5] The basic form of ADMM algorithm is as follows: ( ) In the above equation, ∈ , ∈ , ∈ × , ∈ × , ∈ , the objective function can be separated into several convex functions ( ) and ( ). There are two common architectures for parallel computing. One is the classical C/S architecture, it is a star-like network composed of a master node and several worker nodes. The other architecture is peer-to-peer. In this architecture, there is no master node, and each node has a piece of data, forming a decentralized network. Each node exchanges calculation results with its neighbors, and then achieves global convergence through continuous iteration. The ADMM algorithms implemented by the above two methods are Star-ADMM [6] and decentralized ADMM [7][8][9] respectively. Centralized networks are the earliest and most widely used networks, but also has very obvious flaws, the bottleneck lies in the center node traffic jams because all nodes need to communicate. Decentralized networks have similar total computational complexity, but the busiest nodes have much lower traffic than centralized ones. Therefore, the algorithms with decentralized networks is faster than centralized blocks in the case of low bandwidth or high latency [10]. In order to accelerate distributed ADMM, some scholars have proposed many different schemes [11][12], and we are very interested in a semi-centralized algorithm proposed in [7] [9]. This method is to select a group of nodes in the network as the master node, and their neighbor nodes as the worker node, this group of master nodes is called bridge nodes, the worker node can only communicate and exchange updates with the specified bridge node. We call it B-ADMM. B-ADMM greatly improves the convergence of the algorithm in the presence of noise, and greatly reduces the communication traffic of the master node in the centralized algorithm. It has excellent performance in the distributed ADMM solution. Of course, The acceleration of decentralized network largely depends on the choice of bridge nodes, but the existing research does not consider the network latency when selecting the bridge node.
Our research is based on B-ADMM and takes network latency into consideration in the selection strategy of bridge nodes and we propose a latency aware distributed ADMM algorithm (Greedy-Bridge-ADMM). Our proposed algorithm can select appropriate bridge nodes according to the latency of a specific network, and we allow the bridge nodes to communicate only with the work nodes. We use a virtual network simulation platform [13] to implement this algorithm and carry out a lot of research. The results show that the proposed algorithm has a faster convergence rate than the traditional algorithms when the network connectivity is poor or the network connection latency and variance cannot be ignored.
In the follow parts of this paper, we analyse the formulation of ADMM problem and propose our algorithm in section 2. In section 3, we compared our algorithm with traditional algorithm under different network conditions in real situation through virtual simulation platform. In section 4, we conclude the paper.

Problem formulation
We target at solving some optimization or machine learning problem over a computer network based on distributed ADMM algorithm. For example, consider solving the following problem over a network.
Let a graph G = (V, E) denoted the network, where V= {1..., n} is the set of nodes and E= {1..., m} is the set of edges. The Star-ADMM algorithm (the classical distributed ADMM algorithm) for solving (2) is presented in Algorithm 1 as follow.
Algorithm 1 Star-ADMM algorithm (the classical distributed ADMM algorithm) Form the graph G into a star network with a master node and n-1 worker nodes. for The master initializes and sends 0 z to worker i.
The worker i calculate the following formula.  The master calculates the following formula. ( The master sends From the iterative process of algorithm 1, we can see that each iteration of the algorithm has to wait for the feedback of all workers. Once the communication delay of nodes in the network is prone to fluctuation, deterioration or abnormality, the algorithm will be greatly limited.

Proposed algorithm
The basic idea of the proposed algorithm is to split the master node in the Star-ADMM algorithm into multiple bridges. And each bridge has some nodes as its own workers. The proposed algorithm plans these bridges and workers as little delay as possible, and makes all bridges in the computing cluster converge to the final calculation result through some precise iterative calculation. Specifically, the algorithm first selects a worker, and then checks whether it and its every neighbor have a common bridge. If it does not exist, a public bridge is selected according to the principle of minimum network delay. This goes back and forth until each worker and its every neighbor have a common bridge. Then, in the iterative process, each bridge and its workers perform iterative calculation. Because the bridges and workers in the whole computing cluster are cross connected with each other, the optimization or machine learning problem finally converges.
We call the proposed algorithm Greedy-Bridge-ADMM algorithm because the algorithm performs greedy search according to delay when selecting bridges and their workers. The pseudocode of the Greedy-Bridge-ADMM algorithm is shown in Algorithm 2. The symbols used in the algorithm are defined as follows.

Algorithm 2 Greedy-Bridge-ADMM algorithm (the proposed distributed ADMM algorithm) for
The worker i calculate the following formula.
The worker i calculate the following formula.
to their workers. end for end for k = k+1 end while

Algorithm analysis
As mentioned earlier, the proposed algorithm does not use the master in the Star-ADMM algorithm as a single computing center node, but selects multiple bridges and their workers in the network for calculation respectively. On the one hand, it reduces the computing pressure of the master. On the other hand, more importantly, compared with the master and workers in the Star-ADMM algorithm, the network delays between each bridge and its workers are smaller, which makes the whole computing process not limited to some links with poor delay. In the proposed algorithm, each pair of neighbors ensures that there is at least one common bridge, and the calculation results of each small calculation cluster composed of bridge and its workers are fully integrated in the step of

Experiments
Experiments run on a full connectivity cluster of 16 nodes on GENI [13]. Experiments setting retain part of the links and their delays to simulate various real network scenarios. The implemented LASSO problem is defined as follow.  In Figure 1, we compare the real convergence time of the Star-ADMM algorithm and Greedy-Bridge-ADMM algorithm in typical random network. In the figure, the abscissa is the running time(s) and the ordinate is the accuracy. As can be seen from Figure 1, the Greedy-Bridge-ADMM algorithm converges faster and can converge to a reasonable value in about 40 seconds, while the Star-ADMM algorithm needs to converge to a reasonable value in about 80 seconds. The final accuracy of the Greedy-Bridge-ADMM algorithm is slightly worse than that of the Star-ADMM algorithm, but within the acceptable range.  Figure 2, we compare the performance of the Star-ADMM algorithm and the Greedy-Bridge-ADMM algorithm in different network scenarios. Specifically, in Figure 2(a), the performance of the algorithm under different network connectivity is described. The abscissa in the figure is the connection probability between any two nodes, and the ordinate is the running time. It can be seen that the running time of the Star-ADMM algorithm and Greedy-Bridge-ADMM algorithm decreases with the increase of connection probability. However, the running time of the Greedy-Bridge-ADMM algorithm is always lower than that of the Star-ADMM algorithm, so the performance is better. At the same time, in Figure  2(b), lines describe the performance of the algorithm under different scenarios of network delay variance. The abscissa in the figure is the probability of the edge with large delay in the network, and the ordinate is the running time of the algorithm. It can be seen that the running time of the Star-ADMM algorithm and Greedy-Bridge-ADMM algorithm increases with the increase of large delay edge. However, the running time of the Greedy-Bridge-ADMM algorithm is always lower than that of the Star-ADMM algorithm, so the performance is better. Moreover, with the increase of large delay edges, the advantages of the Greedy-Bridge-ADMM algorithm are more obvious.

Conclusions
Distributed ADMM algorithm has been studied more and more widely because of its excellent convergence performance and potential application in many fields like the emerging machine learning, data analytics, and largescale network system applications. However, the existing research is still lacking, especially ignoring the real network situation such as the latency.
In this paper, we propose an algorithm to construct ADMM networks based on network latency. It selects the bridges according to the principle of minimum network latency on the premise that every worker node and its neighbor have the same bridge node. We call the proposed algorithm Greedy-Bridge-ADMM algorithm. We implemented and evaluated the algorithm in an emulated network. The results show that proposed algorithm achieves real convergence time acceleration with little precision sacrifice, and demonstrate better performance than classical ADMM in different network scenarios, especially in cases of large delay.