extfieldsofstudy:
Computer Science
Mathematics

text:
secml: A Python Library for Secure and Explainable Machine Learning

We present \texttt{secml}, an open-source Python library for secure and explainable machine learning. It implements the most popular attacks against machine learning, including test-time evasion attacks to generate adversarial examples against deep neural networks and training-time poisoning attacks against support vector machines and many other algorithms. These attacks enable evaluating the security of learning algorithms and the corresponding defenses under both white-box and black-box threat models. To this end, \texttt{secml} provides built-in functions to compute security evaluation curves, showing how quickly classification performance decreases against increasing adversarial perturbations of the input data. \texttt{secml} also includes explainability methods to help understand why adversarial attacks succeed against a given model, by visualizing the most influential features and training prototypes contributing to each decision. It is distributed under the Apache License 2.0 and hosted at \url{https://github.com/pralab/secml}.

Introduction
Machine learning is vulnerable to well-crafted attacks. At test time, the attacker can stage evasion attacks (i.e., adversarial examples) [1,2,3,4,5], sponge examples [6], model stealing [7], and membership inference [8] attacks to violate system integrity, availability, or even its confidentiality. Similarly, at training time, the attacker may target either system integrity or availability via poisoning [9,10] and backdoor attacks [11]. The most studied attacks, namely evasion and poisoning, firstly explored by Biggio et al. [9,2], are formalized as constrained optimization problems, solved through gradient-based or gradient-free algorithms [10,12], depending on whether the attacker has white-or black-box access to the target system. Many libraries implement the former, however, they do not allow developers to assess machine learning models' security easily. Hence, we present secml, an opensource Python library that serves as a complete tool for evaluating and assessing the performance and robustness of machine-learning models. To this end, secml implements: (i) a methodology for the empirical security evaluation of machine-learning algorithms under different evasion and poisoning attack scenarios; and (ii) explainable methods to help understand why and how these attacks are successful. With respect to other popular libraries that implement attacks almost solely against Deep Neural Networks (DNNs) [13,14,15], secml also implements training-time poisoning attacks and computationally-efficient test-time evasion attacks against many different algorithms, including support vector machines (SVMs) and random forests (RFs). It also incorporates both the feature-based and prototypebased explanation methods proposed by [16,17,18].

Software Description
secml has a modular architecture oriented to code reuse. We have defined abstract interfaces for all components, including loss functions, regularizers, optimizers, classifiers, and attacks. By separating the definition of the optimization problem from the algorithm used to solve it, one can easily define novel attacks or classifiers (in terms of constrained optimization problems) and then use different optimizers to obtain a solution. This is a great advantage with respect to other libraries like CleverHans [13], Foolbox [14,19], and ART [15] as, we can switch from white-box to black-box attacks by just changing the optimizer (from a gradient-based to a gradient-free solver), without re-defining the entire optimization problem.
secml integrates different components via welldesigned wrapper classes. We integrate several attack implementations from CleverHans and Foolbox, by ex- tending them to also track the values of the loss function and the intermediate points optimized during the attack iterations, as well as the number of function and gradient evaluations. This is useful to debug and compare different attacks, e.g., by checking their convergence to a local optimum and properly tuning their hyperparameters (e.g., step size and number of iterations). secml supports DNNs via a dedicated PyTorch wrapper, which can be extended to include other popular deep-learning frameworks, like TensorFlow and Keras, and it natively supports scikit-learn classifiers as well. This allows us to run attacks that are implemented in CleverHans and Foolbox to universally run on both PyTorch and scikitlearn models.
Main packages. The library supports different threat models (Fig. 1a), and it is organized in different packages (Fig. 1b). The adv package implements different adversarial attacks and provides the functionalities to perform security evaluations. It includes target and untargeted evasion attacks provided by CleverHans and Foolbox as well as our implementations of evasion and poisoning attacks [10]. These attacks target the feature space of a model, regardless of its domain of application (e.g. attacking Android malware detection 1 ). Hence, developers can encode ad-hoc manipulations that correspond to perturbations in the input space of a targeted domain (e.g. Windows programs [20]), and then use the algorithms included inside secml to optimize the attack. All the included attacks are listed in Fig. 1b. The ml package imports classifiers from scikit-learn and DNNs from PyTorch. We have extended scikitlearn classifiers with the gradients required to run eva-sion and poisoning attacks, which have been implemented analytically. Our library also supports chaining different modules (e.g., scalers and classifiers) and can automatically compute the corresponding end-toend gradient via the chain rule. The explanation package implements the feature-and prototype-based explanation methods [16,17,18]. The optim package provides an implementation of the projected gradient descent (PGD) algorithm, and a more efficient version of it that runs a bisect line search along the gradient direction (PGD-LS) to reduce the number of gradient evaluations [21]. Finally, data provides data loaders for popular datasets, integrating those provided by scikit-learn and PyTorch; array provides a higherlevel interface for both dense (numpy) and sparse (scipy) arrays, enabling the efficient execution of attacks on sparse data representations; figure implements some advanced plotting functions based on matplotlib (e.g., to visualize and debug attacks); and utils provides functionalities for logging and parallel code execution. We also provide a model zoo, available on GitHub as well, 2 that contains pre-trained models to rapidly test newly implemented attacks and utilities.
We recap the main functionalities of secml in Table 1, where we also compare it with other relevant libraries. Notably, our library is the sole that provides attack loss inspection plots to choose the appropriate attacks' hyperparameters, and security evaluation plots, to ease the complexity of assessing the robustness of machine learning models. Moreover, the features offered by secml are not only related to attacking machine learning models, but they are gathered to elevate secml as a complete tool for attacking, inspecting, and  assessing the performances of machine learning models.
Testing and documentation. We have run extensive tests on macOS X, Ubuntu 16.04, Debian 9 and 10, through the GitHub Actions infrastracture, 3 with also experimental support on Windows 10 platforms. The user documentation is available at https://secml. readthedocs.io/en/v0.15/, along with a basic developer guide detailing how to extend the ml package with other classifiers and deep-learning frameworks. The complete set of unit tests is available in our repository. A comprehensive view of the functionalities available in secml is included in tutorials available as Jupyter notebooks.

Impact
We now offer two examples extracted from secml to showcase its impact: evasion attacks against DNNs, and a poisoning attack against an SVM.
Evasion attacks. We show here how to use secml to run different evasion attacks against ResNet-18, a DNN pretrained on ImageNet, available from torchvision. This example demonstrates how secml enables running CleverHans attacks (implemented in Tensor-Flow) against PyTorch models. Our goal is perturbing the image of a race car to be misclassified as a tiger, using the 2 -norm targeted Carlini-Wagner (CW) attack (from CleverHans), the 2 PGD attack implemented in secml, and PGD-patch, where the attacker can only change the pixels corresponding to the license plate [22].
We run all the attacks for 50 iterations, we set the confidence parameter of the CW attack κ = 10 6 to generate high-confidence misclassifications, and c = 0.4, yielding an 2 perturbation size = 1.87. We bound PGD to create an adversarial image with the same perturbation size. For PGD-patch, we do not bound the perturbation size for the pixels that can be modified. The resulting adversarial images are shown in Fig. 2. For PGD-patch, we also highlight the most relevant pixels used by the DNN to classify this image as a tiger, using the integrated gradients explanation method. The most relevant pixels are found around the perturbed region containing the license plate, unveiling the presence of potential adversarial manipulations.
We also visualize the performances of the attack in Fig. 3. The leftmost plot shows how the attack losses (scaled linearly in [0, 1] to enable comparison) iterationwise, while the rightmost plot shows how the confidence assigned to class race car (dashed line) decreases in favor of the confidence assigned to class tiger (solid line) for each attack, across different iterations. By inspecting them, we can understand if these attacks have been correctly configured. For instance, by looking at the loss curves on the left, we can understand if the attacks reached convergence or not, thus facilitating tuning of either the step size or the number of iterations. Also, by looking at the plot on the right, it is clear that all the attacks are successful since the confidence of the target class exceeds the score of the original one.
Poisoning attacks. We also show the effect of a poisoning attack provided by secml applied to an SVM classifier implemented in scikit-learn. The experimental setting and code are available in one of our tutorials on GitHub 4 . Results of the successful attack are represented in Fig. 4., highlighting the flexibility of secml in applying different strategies to third-party models as well, without the need of customizing them on a particular framework.

Conclusions and Future Work
The secml project was born more than five years ago and we open-sourced it in August 2019. Thanks to an emerging community of users and developers from our GitHub repository, we firmly believe that secml will soon become a reference tool to evaluate the security of machine-learning algorithms. We are constantly working to enrich it with new functionalities, by adding novel defenses, wrappers for other third-party libraries, and more pretrained models to the secml zoo. Poisoned classifier (test set) Figure 4: Poisoning attacks against an SVM implemented with scikit-learn. The poisoning data (denoted with in the right plot) induce the model to learn a worse decision boundary.


Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion

Learning joint embedding space for various modalities is of vital importance for multimodal fusion. Mainstream modality fusion approaches fail to achieve this goal, leaving a modality gap which heavily affects cross-modal fusion. In this paper, we propose a novel adversarial encoder-decoder-classifier framework to learn a modality-invariant embedding space. Since the distributions of various modalities vary in nature, to reduce the modality gap, we translate the distributions of source modalities into that of target modality via their respective encoders using adversarial training. Furthermore, we exert additional constraints on embedding space by introducing reconstruction loss and classification loss. Then we fuse the encoded representations using hierarchical graph neural network which explicitly explores unimodal, bimodal and trimodal interactions in multi-stage. Our method achieves state-of-the-art performance on multiple datasets. Visualization of the learned embeddings suggests that the joint embedding space learned by our method is discriminative.

Introduction
Multimodal machine learning, where multiple data sources are available, always fares better compared with the situation where only single data source is utilized (Baltrušaitis, Ahuja, and Morency 2019). For example, sentiment analysis of text has been widely explored for a long time (Liu and Zhang 2012). But recent research shows that information from text is not sufficient for mining opinions of human (Poria et al. 2017a), especially under the situation where sarcasm or ambiguity is included. Nevertheless, if machine is able to understand a speaker's facial expression and how spoken language is uttered, it would be much easier to figure out the real sentiment. In this paper, we focus on analyzing human multimodal language from videos, where acoustic, language and visual modalities are included Zadeh et al. 2018c). We can easily extend our work to other machine learning tasks as long as multiple modalities are presented, such as audio-visual speech recognition (Su, Lan, and Liu 2018) and cross-modal retrieval (Xing et al. 2018).
How to fuse representations of various modalities has always been an open research. A key issue in multimodal fusion lies in the heterogeneous data distributions from different modalities (Baltrušaitis, Ahuja, and Morency 2019), leading to the difficulty in mining complementary information across modalities which is critical for a comprehensive interpretation of multimodal information. The majority of previous works do not devote efforts to learn joint embedding space for various modalities to match multimodal distributions. Instead, they apply a subnetwork to each modality and then conduct fusion immediately (Zadeh et al. 2017;Poria et al. 2017b;Liu et al. 2018). However, due to the heterogeneity across divergent modalities (Lin et al. 2015), the transformed representations learnt by these approaches can still follow unknown yet complex distributions.
Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) have achieved significant progress, which can explicitly map one distribution to another prior distribution using adversarial training (Makhzani et al. 2016). Based on its unique characteristics, adversarial training is suitable for modality distribution translation. Motivated by it, we propose an adversarial encoder framework to match transformed distributions of all modalities and learn a modalityinvariant embedding space. Specifically, source modalities' encoders, transforming the unimodal raw features into embedding space, try to fool the discriminator into distinguishing their encoded representations as those from target modality, while discriminator aims to classify the encoded representations from target modality as true but others as false. We also define one decoder for each modality that seeks to reconstruct the raw features to prevent unimodal information from being lost. Moreover, a classifier is built to classify the encoded representations into true category, which ensures that the embedding space is discriminative for learning task.
Furthermore, many prior methods fail to conduct fusion in a hierarchical way and are unable to explicitly model the interaction between each subset of multiple modalities (Poria et al. 2017b;Liu et al. 2018;Zadeh et al. 2018b). In contrast, we interpret multimodal fusion as a hierarchical interaction learning procedure where firstly bimodal interactions are generated based on unimodal dynamics, and then trimodal dynamics are generated based on bimodal and unimodal dynamics. Drawing inspiration from recent success of graph-style neural network (Garcia and Bruna 2018;Zadeh et al. 2018c), we propose a hierarchical fusion network, i.e., graph fusion network, which is highly interpretable in terms of the fusion procedure. Our graph fusion network consists of three layers containing unimodal, bimodal and trimodal dynamics respectively. The vertices in lower layer deliver their information to the higher layers where the information is fused to form multimodal information for the higher layers. By this means, we can explore cross-modal interactions hierarchically while still maintain the original interactions.
In brief, the main contributions are listed below: • We propose Adversarial Representation Graph Fusion framework (ARGF) for multimodal fusion. We address the importance of matching distributions before fusion and introduce adversarial training to learn a discriminative joint embedding space for various modalities, which can significantly narrow modality gap by matching multimodal distributions. Moreover, we define decoders and classifier to retain unimodal information and enhance discrimination of the embedding space respectively. • We propose a hierarchical graph fusion network which can explicitly model unimodal, bimodal and trimodal dynamics successively and is highly interpretable due to explicit meanings of vertices. It is flexible in fusion structure due to the changeable weights of edges and the ability to assign different importance to various interactions. • ARGF achieves state-of-the-art performance on three multimodal learning datasets. We also provide visualizations for embedding space and graph fusion to prove their characteristics and contributions.

Related Work
Earlier works on multimodal fusion focus on early fusion and late fusion. Early fusion approaches extract features of various modalities and fuse features at input level by simple concatenation (Wollmer et al. 2013;Poria et al. 2016), but they cannot fully explore intra-modality dynamics as stated in (Zadeh et al. 2017). In contrast, late fusion first makes decision according to each modality and then fuses the decisions by weighted averaging (Nojavanasghari et al. 2016;Kampman et al. 2018), but it fails to model cross-modal interactions for features cannot interact with each other. Recently, local fusion has become the mainstream strategy which can model time-dependent cross-modal interactions (Gu et al. 2018;Zadeh et al. 2018b;Wang et al. 2019). For instance, (Zadeh et al. 2018a) propose Memory Fusion Network that fuses memories of LSTMs over time, which is extended by (Zadeh et al. 2018c) using a Dynamic Fusion Graph (DFG) to fuse features. Our graph fusion network is different from DFG mainly in that: 1) we use inner product as part of edges' weight to estimate similarity between interactions; 2) in addition to fusing bimodal and unimodal dynamics, we also fuse each two bimodal dynamics to obtain more complete trimodal representations; 3) we determine the importance of vertices respectively to obtain each layer's output firstly rather than directly adding weighted information from all vertices. Tensor fusion has also become a new trend. Tensor Fusion Network (Zadeh et al. 2017) adopts an outer product from multimodal embeddings to conduct fusion, followed by Liu et al. that tries to improve efficiency and reduces redundant information. More recently, Mai, Hu, and Xing propose a 'Divide, Conquer and Combine' strategy to conduct local tensor and global fusion, which is extended in (Mai, Xing, and Hu 2019) using a more efficient feature segmentation method and an elaborately-designed BM-LSTM. In addition, some approaches use factorization methods to learn multimodal embedding Tsai et al. 2019b). But methods above do not explicitly explore joint embedding space before fusion. Consequently, modality gap still seriously affects the effect of fusion.
For modality translation methods, Multimodal Transformer (MulT) (Tsai et al. 2019a) transforms source modality to target modality using directional pairwise cross-modal attention. Moreover, Multimodal Cyclic Translation Network (MCTN) (Pham et al. 2019) learn joint representations via encoder-decoder framework by translating encoder's input (source modality) into decoder's output (target modality). MCTN uses encoder's output as joint representations of source and target modalities without exploring information from target modality, aiming to solve modality loss problem. In contrast, we address the problem of distribution translation and learn joint embedding space by translating the distributions of source modalities to target modality via adversarial training. Graph fusion is then conducted using encoded representations of both target and source modalities.
GANs (Goodfellow et al. 2014) have been successfully adopted in lots of learning tasks (Xing et al. 2018;Lin et al. 2015). GANs have attracted significant interest in matching different distributions. Specifically, Adversarial Autoencoder (Makhzani et al. 2016) aims to match aggregated posterior of hidden vector with a prior distribution. (Peng and Qi 2019) seek to learn common representations for correlating heterogeneous data of various modalities, while we learn a modality-invariant embedding space for multimodal fusion by distribution translation. Besides, we add more constraints to transformed representations as well as separate the embedding space learning and cross-modal interaction learning procedures.

Model Architecture
An overview of ARGF is given in Fig. 1. ARGF is comprised of two stages: a joint embedding space learning stage and a graph fusion stage. In the first stage, we learn an embedding space for all modalities via adversarial framework. In the second stage, we utilize the representations output by encoders to conduct graph fusion (see Fig. 2).

Joint Embedding Space Learning
In this section we provide an effective approach for translating distributions of source modalities to that of the target modality so as to learn a modality-invariant embedding space. Matching distributions in embedding space is crucial for modality fusion because the divergent statistical properties among modalities might seriously prevent cross-modal dynamics from being effectively explored. Drawing inspiration from GANs (Goodfellow et al. 2014), we regularize the network by adversarial training. We also exert additional constraints like reconstruction loss and classification loss to optimize the learnt embedding space.
We start with notation: assume we have three modalities as input: acoustic x a ∈ R da , language x l ∈ R d l and visual x v ∈ R dv with d a being the dimensionality of x a and so on. Assume that x l is the target modality and other modalities are known as source modalities, and p(x l ) represents prior data distribution of language modality. Similar to (Makhzani et al. 2016), we define the transformed distributions of these three modalities as: where q(x e l |x l , θ l ) is known as the encoder function of language modality with θ l being the parameters, and p θ l (x e l ) is the transformed language distribution in learnt embedding space restricted by θ l (given specific input x l , x a and x v , the encoded representations are x e l , x e a and x e v respectively). Language encoder is a deep neural network that is denoted as E l : R d l → R k for simplicity, where k is the dimensionality of encoded representations, so are other en-coders (all encoded representations share the same dimensionality k).
We hope that through optimizing θ l , θ v and θ a , we can explicitly map transformed distributions p θa (x e a ) and p θv (x e v ) to p θ l (x e l ). However, the distributions of different modalities are very complex and they vary in nature which are extremely difficult to be matched by simple encoder networks. Therefore, we utilize adversarial training to add constraints to transformed distributions. Specifically, a discriminator D is defined which aims to classify p θ l (x e l ) as true but p θa (x e a ) and p θv (x e v ) as false, while the generators (which are encoders E a and E v ) seek to fool discriminator D into classifying p θv (x e v ) and p θa (x e a ) as true. The generators and discriminator compete with each other as a min-max game to learn joint embedding space. The loss function here can be divided into two parts: fake adversarial loss £ f al and true adversarial loss £ tal , as shown below: E a and E v try to fool D which results in £ f al while D aims to determine the distribution of target modality as true but others as false, leading to £ tal . In practice, we define £ f al and £ tal as: (4) where D(x e a ) denotes the predicted distribution value of x e a ranging from 0 to 1, and w is the learnt weight for adversarial loss. If discriminator can not tell the target modality from all modalities (in the case where D(x e a ) ≈ D(x e v ) ≈ D(x e l )), then the distributions of various modalities are successfully mapped into a modality-invariant embedding space. Adversarial training strategy puts restrictions on the statistical properties of the encoded representations. By adversarial training, modality gap can be narrowed effectively so that representations from various modalities can be directly fused.
Transforming distributions might lead to the loss of unimodal information needed for mining complementary information between modalities. Therefore, to retain modalityspecific information in learnt embedding space, we define decoders as: where q(x l |x e l , θ dl ) is the language decoder with θ dl being the parameters, and p θ dl (x l ) is the distributions of reconstructed representations for language modality. Given spe- To do so, we define a reconstruction loss as: By minimizing £ rl , encoded representations can retain the unimodal information for further fusion. Furthermore, to render the learnt embedding space more discriminative with respect to learning task, we also define a classifier which takes as input the encoded representations of each modality. The classifier is defined as: where C ∈ R 3k → R N denotes the classifier with N being the number of categories to be classified, andỹ m is the predicted label based on encoded representations of modality m. To minimize predicted error, we define a classification loss as: where y is the true one-hot label. Classification loss enables the encoded representations to carry the needed label information and thus the embedding space is discriminative for learning tasks.
In conclusion, the total loss of this section is: where λ is a hyper-parameter that determines the importance of loss functions whose value is determined by grid search. During gradient update, firstly, we apply £ f al and £ rl to update encoders and decoders; secondly, we use £ tal to update the discriminator to improve its discriminative power with respect to fake/true distributions; lastly, £ cl is utilized to update encoders and classifier to improve discrimination ability of joint embedding space.

Graph Fusion Network
We explore cross-modality interactions by fusing encoded representations of all modalities in this section. Assuming that multimodal fusion is in multi-stage  and considering the need to preserve all n-modal interactions, we introduce graph fusion network (GFN), a hierarchical neural network, to model unimodal, bimodal and trimodal interactions successively. As shown in Fig. 2, the network consists of unimodal, bimodal and trimodal dynamic learning layers. GFN regards each interaction as a vertex and the similarities between interacted vertices as well as the interacted vertices' importance as weights of edges, which is highly interpretable and flexible in terms of fusion structure.
The first layer is the unimodal dynamic learning layer consisting of unimodal vertices of three modalities whose information vectors are denoted as V l , V a and V v , corresponding to the encoded representations x e l , x e a and x e v respectively. In first layer, we apply a Modality Attention Network M AN ∈ R k → R 1 to process each vertex and determine the importance of these unimodal interactions since not all modalities contribute equally. Then we obtain final unimodal dynamics by calculating the weighted average of information from all unimodal vertices, as shown below: where U is the final unimodal vector and α m is the weight of modality m. In practice, M AN consists of a sigmoidactivated dense layer parameterized by Θ M AN . In the second layer, i.e., bimodal dynamic learning layer, each two unimodal vertices are fused using a multi-layer neural fusion network: M LP ∈ R 2k → R k to obtain each bimodal vertex: where ⊕ denotes vector concatenation, V m1m2 is the information vector of m 1 m 2 vertex and Θ M LP is the parameter matrix of M LP . In practice, M LP is composed of two dense layers activated by Leaky ReLU and tanh respectively. As for the weights of those edges linking the first and second layer, we firstly estimate the similarity of each two uniformed unimodal information vectors of first layer using inner product. We argue that the more similar two information vectors are, the less important their bimodal interaction would be. This is premised on the assumption that provided two information vectors are close to each other, then little complementary information lies between them and their information has been well explored in the first layer. The similarity of two information vectors is defined as: where V m1 represents the softmax-normalized vector of V m1 (the normalization ensures that the computed similarity is between 0 to 1), T means vector transpose operation and S m1m2 is a scalar that denotes the similarity of m 1 and m 2 vertices. The weight of edge that links m 1 vertex in first layer and m 1 m 2 vertex in second layer is defined as Sm 1 m 2 +0.5 , which varies proportionally with α m1 but grows in inverse proportion to S m1m2 , so are the other weights of edges. Then, the weight for m 1 m 2 vertex is defined as: where α m1m2 is a scalar that represents the weight of m 1 m 2 vertex (the value 0.5 on the denominator is applied to control the relative importance of similarity and vertex weights to α m1m2 ), and Eq. (14) is a softmax-uniformed operation of weight vector in second layer. The output of the second layer is the weighted average of information in bimodal vertices: where B is the combined bimodal dynamic. Each two bimodal vertices are further fused using the same M LP network that generate bimodal dynamics (but the parameters are not shared) to obtain trimodal vertices in the third layer, i.e., trimodal dynamic learning layer. In addition, as shown in Fig. 2, each specific bimodal vertex is also fused with the unimodal vertex that do not contribute to the formation of this bimodal vertex in previous fusion stage, resulting in three other trimodal vertices. Therefore, there are six trimodal vertices in total. We apply the same weight computing method for edges that link to the third layers and the same importance measure method for trimodal vertices as applied in second layer. Then we add up the weighted information from each trimodal vertex to obtain final trimodal information T .
The final output of the hierarchical graph fusion network is the concatenation of trimodal, bimodal and unimodal dynamics, defined as: Ω = U ⊕ B ⊕ T . Finally, a decision neural network is utilized to infer the final decision: where M ∈ R N (N is the number of classes). Dec : R 3k → R N is the decision inference module containing a normalization layer followed by three dense layers activated by tanh, tanh and sof tmax respectively.

Experiments
Datasets CMU-MOSI (Zadeh et al. 2016) is a widely-used dataset that includes a collection of 93 opinion videos from different speakers which have been divided into 2199 segments in total. We report the binary (positive and negative) sentiment accuracy and F1 score on this dataset. We use 1141 segments for training, 306 segments for validation and 752 segments for testing. CMU-MOSEI (Zadeh et al. 2018c), consisting of 2928 videos (20802 segments in total), is the largest multimodal language analysis dataset so far. We report positive, negative and neutral sentiment classification accuracy and F1 score on this dataset. We use 13168 segments for training, 3020 segments for validation and 4614 segments for testing. IEMOCAP (Busso et al. 2008) is an emotion recognition dataset that contains 151 videos and the videos have been segmented into 7433 segments. The dataset contains 9 emotional labels. We analyze the anger, happiness, sadness and neutral emotions so as to compare with previous approaches. The training set consists of 4674 segments, while the validation and test set has 1136 and 1623 segments, respectively.

Experimental details
We develop our model on Pytorch. The implemental details of encoder, decoder, classifier and discriminator are shown in Fig. 3. Specifically, the number inside the dense layer is the output dimensionality, k is the dimensionality of encoded representations, n is the number of classes and d denotes the dimensionality of input feature vector (d is 50 for CMU-MOSI and CMU-MOSEI datasets and 100 for IEMO-CAP datatset). We apply Mean Square Error as loss function for graph fusion network with Adam (Kingma and Ba 2015) being the optimizer. The framework is trained end to end. The hyper-parameters such as batch size, learning rate and k are chosen by grid search to optimize the performance. In feature extraction stage, for CMU-MOSI and IEMO-CAP, we follow the setting of (Poria et al. 2017b). A text-CNN consisting of word2vec embedding (Mikolov et al. 2013) followed by CNNs (Karpathy et al. 2014), openS-MILE (Eyben 2010) and 3D-CNN (Ji, Yang, and Yu 2013) are applied for language, acoustic and visual feature extraction respectively. For CMU-MOSEI, the features are extracted using CMU-MultimodalSDK 1 . GloVe word embeddings (Pennington, Socher, and Manning 2014), Facet 2 and COVAREP (Degottex et al. 2014) are applied for extracting language, visual and acoustic features respectively (please refer to  for more details).
After extraction, similar to HFFN (Mai, Hu, and Xing 2019), we develop a Unimodal Context Learning Network (UCLN): R u×dj → R u×d , which is composed of a bidirectional LSTM layer followed by a dense layer, for each separate modality. Here, u denotes the number of segments that constitute a video and d j is the dimensionality of raw feature vector for j th modality. Through UCLN, feature vectors of all modalities are mapped into the same dimensionality d

Results and Discussions
Comparison with Baselines: As presented in Table 1, ARGF shows improvement over typical approaches and outperforms the SOTA method LMFN by about 0.5% on CMU-MOSI dataset. Moreover, compared to the tensor fusion methods TFN and LMF, ARGF achieves improvement by about 3% and 6% respectively. We argue that it is probably partly because these approaches ignore exploring modalityinvariant embedding space, while we adopt adversarial training to learn joint embedding space before fusion. These results demonstrate the superiority of ARGF. We also report ARGF's performance on the more challenging datasets IEMOCAP and CMU-MOSEI to evaluate ARGF's robustness. For IEMOCAP, from Table 2 we can infer that ARGF achieves the best performance and significantly outperforms SOTA methods by about 2% in the most important index (average accuracy). For CMU-MOSEI, as shown in Table 3, the average accuracy and F1 score of ARGF are still the highest among all methods, showing excellent performance.
In addition, all the models' performance on 'Negative' emotion are weaker than that on other emotions by a large margin. We speculate that one of the reasons is that the number of samples belonging to 'Negative' class is much fewer than the ones of other emotions. Consequently the model tends to predict otherwise which is less 'risky'. Complexity Analysis: We use the amount of trainable parameters as a proxy for the space complexity. As shown in Table 4, our model has 1,270,770 trainable parameters (k is set to 50), which is approximately 29.93% and 91.83% of the number of parameters of TFN and BC-LSTM, respectively. To explore the time complexity of ARGF and make a comparison against baselines, we compute FLOPs for each method during testing. We empirically find that ARGF needs 2,017,645 FLOPs in testing, whereas the number of FLOPs in TFN and BC-LSTM are 8,491,844 and 1,322,044, respectively. The reason for fewer trainable parameters and moderate number of FLOPs is that despite having multiple components in the architecture, our model entails low dimensions for unimodal and multimodal representations. In addition, every component in our architecture has only several layers, which guarantees a reasonable computational load.
Choice of Target Modality: As presented in Table 1, 2 and 3, the results demonstrate that language modality is the best choice for CMU-MOSI, while acoustic and visual arrives in the second and third place respectively by a slight margin. Actually, all the modalities perform very closely across the three datasets. There are no hard and fast rules to choose an optimal target modality since their performances are rather similar. It is reasonable because the target modality only serves as a distribution provider so that the source modalities can map their transformed distributions into that of the target modality. During the distribution mapping procedure, all the modalities' information is well preserved by the use of decoders and classifier. Therefore, theoretically there is no difference what the target modality is.
Ablation Studies: From Table 5 we can see that the removal of any component in ARGF leads to a decline in performance. Specifically, the removal of decoders leads to the most significant decline of more than 1.5%, closely followed by the removal of adversarial training. In addition, experiments without classifier achieve relatively acceptable results, with a slight decrease of around 0.4%, but classifier is critical for learning a discriminative embedding space as demonstrated in the visualization part. The ablation studies show that these three components are crucial factors contributing to the competitive results of ARGF.
Comparison between fusion strategies: To demonstrate that our fusion method is indeed effective, we conduct a contrast experiment to compare the performance between other fusion strategies with our GFN. We can infer from Table 6 that GFN brings significant improvement on performance compared to other fusion methods, demonstrating its superiority. Specifically, DFG (Zadeh et al. 2018c)      parable results in two datasets, but our GFN outperforms it by over 2% in IEMOCAP and 1% in CMU-MOSI. We argue that it is because we explicitly model unimodal, bimodal and trimodal dynamics and obtain trimodal representations in a more comprehensive way (as also revealed in the following visualization part). From above analysis, we can draw the conclusion that the interpretable, comprehensive and hierarchical fusion strategy is a crucial factor that lead to the marked improvement of ARGF. In addition, all the fusion strategies fitted in our framework obtain relatively good results compared with the original frameworks. For instance, our version of Tensor Fusion (Zadeh et al. 2017) achieves the accuracy of 59.46% in IEMOCAP, which is much higher than TFN's accuracy (54.28%, see Table 2). To some extent, these comparisons prove that learning joint embedding space before feature fusion is indeed important and effective. Visualization of Embedding Space: We provide a visualization for distributions of sentiments in the embedding space where the the left sub-figure on Fig. 5 illustrates the embedding space learnt by ARGF while the right sub-figure    Table 6: Accuracy of various fusion structures. The diagrams of some fusion methods for comparison are given in Fig. 4. Please refer to the corresponding papers (Zadeh et al. 2017;Liu et al. 2018;Zadeh et al. 2018c) for the diagrams of Tensor Fusion, LMF and DFG respectively. Figure 6: Visualization for Graph Fusion in CMU-MOSI sentiment analysis task. Each row represents one test sample and each column denotes weight for one vertex.
presents the embedding space learnt without classifier. The visualization is obtained by transforming the concatenated features of all modalities output by encoders. We use t-SNE algorithm to transform high dimensional concatenated feature vectors into 2-dimension feature points. We can infer from Fig. 5 that in the embedding space learnt by ARGF, the dots tend to form two separate clusters which mainly belong to positive and negative sentiment respectively. The distance between two clusters is large but the dots belonging to same cluster are tied closely, which demonstrates the discrimination of our embedding space. Nevertheless, there are some dots that are extremely difficult to be correctly classified found in the wrong clusters, which drives the need for advanced fusion strategies to explore cross-modal dynamics.
In contrast, it is clear that the embedding space learnt without classifier cannot distinguish positive and negative sentiments effectively, which highlights the necessity of classifier in learning a discriminative embedding space.
Visualization for Weights of Vertices in Graph Fusion: This visualization is conducted to prove graph fusion's interpretability. As shown in Fig. 6, for unimodal interactions (the first three columns), obviously language modality is the most predictive for the majority of samples, which is reasonable since language is the most important clue for sentiment analysis. But its importance varies greatly across samples, indicating the advantage of analyzing sentiment in a multimodal form for other modalities can play a dominant role whenever language modality is trivial for a specific sample. For bimodal interactions (from 4 th to 6 th columns), weights for al vertex (α al ) and vl vertex (α vl ) are very close, followed closely by α av , possibly because language modality plays a huge role in bimodal fusion. For trimodal interactions, to our surprise, vertices obtained by fusing a bimodal vertex and a unimodal vertex hardly make any difference, but vertices obtained by fusing two bimodal vertices dominate trimodal information. It proves the necessity to explore interactions for each two bimodal vertices, which are not modeled by DFG (Zadeh et al. 2018c).

Conclusions
We propose a novel multimodal framework to learn a discriminative joint embedding space and then perform graph fusion. By introducing adversarial training to match distributions, modality gap can be significantly narrowed and the representations can be directly fused. With the aid of GFN, we can explore unimodal, bimodal and trimodal dynamics successively and dynamically change the fusion structure.


extfieldsofstudy:
Computer Science
Medicine

text:
An Innovative Method for Preserving Privacy in Internet of Things

Preservation of privacy of users’ personal data has always been a critical issue to deal with. This issue in the Internet of Things (IoT), which facilitates millions of applications, has become even more challenging. Currently, several approaches and methods are available to safeguard privacy but each of them suffers from one or more anomalies. In particular, Trusted Third-Party approach relies on the trust of a third-party server, Cooperation needs the trust of other peers, Obfuscation is known to return inaccurate results, and Dummy generates too much overhead. Moreover, these and most of the other well-known approaches deal only with specific types of applications linked to the location-based services. In this paper, we present two new methods, namely: Blind Third Party (BTP) and Blind Peers (BLP), and combine them to form a new one to be known as the Blind Approach (BLA). With the help of simulation results we shall demonstrate the effectiveness and superiority of BLA over the other available methods. The simulation results also exhibit that BLA is free from all the existing problems of the other approaches. However, BLA causes a slight increase in the average (response) time, which we consider to be a minor issue. We shall also discuss the capability and superiority of the Blind Approach in the cases of E-health, Smart Transportation, and Smart Home systems.

Introduction
Privacy is about the right of people to determine how, where, when, and who can use their data. Preserving privacy is about preventing misuse of personal data, which normally includes identity and the associated information about location, health, social status, financial condition, contact details, etc. Personal information is often sensitive [1] and unsolicited use of it may result in serious and sometimes life-threatening consequences. In Internet of Things (IoT) applications, users provide personal data to different devices and services, where privacy becomes extremely vulnerable [2][3][4]. In particular, location-based service (LBS), which uses personal data of users, is becoming increasingly popular [5]. Preservation of users' identity (and hence the personal information) from the service providers (SPs) has attracted the attention of many researchers. Privacy and security are different concepts which we have compared and depicted in Figure 1.

Background
The IoT [6] is now contributing to solving many issues and problems arising in business, industry, science and socio-economic areas to positively and significantly improve the quality of our lives [7,8].
IoT is a network of many devices dominated by Wireless Sensor Networks (WSNs), Radio Frequency Identification Tags (RFID Tags), and Smart Devices [9][10][11]. These devices can sense and identify everything in one's body or the surrounding environment to provide the required data for several innovative services and applications such as smart cities, smart home, E-health systems, E-learning, E-business, Location-based Services (LBS), crowd management, etc. as depicted in Figure 2. The objects (things) of IoT are growing at a very fast rate and are expected to grow to several billions by 2020 [11][12][13]. Therefore, we can expect an even larger number of applications and more benefits from these technologies. The objects of IoT sense massive data of users and send it to the SP, which is stored in clouds, resulting in significant privacy issues, which must be addressed properly and adequately [14]. As the number of IoT devices increases, so does the vulnerability to privacy breaches. Effective and concerted measures to ensure privacy is paramount to boost user confidence in dealing with their data in IoT applications or systems [15,16]. Preservation of privacy of user identity and associated data is about protecting it not only from outsider intruders but also from insider breaches, dominated by SPs, involving profiling, tracing, collecting, or unauthorized usage [17][18][19]. Many researchers are engaged in elucidating and exploring different ways and methods to ensure privacy. The available methods and approaches for preserving privacy of data in IoT applications, without any exception, suffer from one or more shortcomings, and hence the problem of ensuring the privacy of data is far from being resolved. A survey of available methods to ensure privacy in IoT applications [20] reveals that many of them are limited to LBS-enabled applications [21][22][23]. Moreover, these approaches are reliant on the third trusted party (TTP), which can't be taken for granted [24]. The SP or TTP can indeed be the internal attackers or partner to a host of external attackers [25]. Furthermore, the users are forced to compromise on the accuracy of their retrieved results [26]. A few methods have also dealt with the privacy of E-Health data, mostly involving either discrete queries or security rather than privacy [27]. Invariably, these methods force the users to absolutely trust the SP, which can be regarded as having next to no privacy protection.
Data in IoT applications passes through four (main) phases, each with several layers [28,29], as depicted in Figure 3. These four phases are Sensing and Collecting, Transferring, Processing, and Storing. Any protection scheme or method must ensure privacy in all these phases. The figure also shows three types of computing models, namely: cloud, fog, and core fog. It is worth mentioning that fog nodes are densely distributed at the edge of the network. Fog is used to enhance the speed of the response and capability of IoT applications, where the central cloud may be sluggish in response because of bottlenecks. Moreover, the usage of fog can reduce the dependence on trust with the SP [2]. Core fog is no different to fog except that it has more resources, and sits in between cloud and fog for better management. The mobile data during and after the transition, as shown in Figure 4, has five parts, namely: Identity, Location, Type, Time, and Data/Query, the most sensitive of which is determined by the nature of the ensuing application. To propose a protection scheme or technique, it would be logical to focus on the sensitive parts rather than all of them to promote a considerable trade-off between privacy and performance [30,31]. Ensuring the security of application data from outsider attacks can be dealt effectively through cooperation between the user and the trusted SP by encryption [32][33][34]. However, protection of privacy of the user deals with both the outsider intruders and insider attackers, one of which could be the SP itself. To safeguard the privacy of personal data from the SP is a vexed issue because most of the available methods resort to adding noise to data, which adversely impacts accuracy of the results, rendering it to be unacceptable in some domains like health.

Statement of Problems and Our Contribution
The problem of preserving privacy has become critical in IoT applications involving personal data of users. Existing methods for the protection of privacy are not reliable for one reason or another. The most prominent of the existing methods, namely: Trusted Third Party, Cooperation, Obfuscation, Dummy, Data Encryption, Private Information Retrieval (PIR) and Caching approaches, to be referred as the 'group of seven' approaches, suffer from various kinds of anomalies. For example, Trusted Third Party is reliant on a third-party server, Cooperation is built on the trust of other peers, Obfuscation is prone to produce inaccurate results, and Dummy has a problem of generating strong dummies in addition to resulting in too much overhead such as that in PIR. Also, as mentioned earlier, the group of seven approaches deal only with the LBS. Thus, there is no known method which can be relied upon for the protection of privacy in IoT.
In this article, we propose a new method to be known as the Blind Approach, abbreviated as B L A, to protect the user's identity and associated personal data in the IoT applications. The Blind Approach works by way of effectively misleading SP about the real identity and query of the user, and unlike the group of seven approaches, neither relies on the trust of any party nor compromises the accuracy of the results, with only a small overhead. The B L A is built by combining two methods, namely: Blind Third Party (BTP) and Blind Peer (B L P), and also presents an adaptive scenario between BTP and B L P, to be referred to as Integrated Blind Parties (IBPs). Blind Approach offers a solution to all the existing problems found in the in the previously mentioned group of seven approaches. In this article, we have achieved the following:

1.
A detailed description of how B L A preserves the privacy in IoT applications 2.
A discussion of the relevance of BTP, B L S, and IBPs, and usefulness and effectiveness of (a) BTP in overcoming the drawbacks in Trusted Third-Party (TTP) approach (b) B L P in addressing the problem of trust among peers in the Cooperation approach (c) IBPs to creating dynamism between BTP, B L S

3.
A detailed analysis of attacks to demonstrate the advantages of in Blind Approach in safeguarding privacy without affecting the accuracy of results or data 4.
Applications of B L A for preserving privacy of users in E-health, Transportation, and Smart Home systems, as well as underwater sensor networks, without having to trust any third party.

Literature Review
In this section, we shall discuss all significant approaches for preserving privacy in IoT. We have grouped them in four categories based on the trust issue [22] as shown in the Figure 5. An additional group-based partially on trust or on semi-trust can also be added into this classification, whereas the fourth group, namely: 'Without Trust', can also be classified as 'Trust Free'. Researchers over the years have invented several approaches to and methods for each group. Here we shall outline main ideas together with the drawbacks in each of them. In keeping up with the scope of this article, we shall focus on the group of seven approaches, members of which suffer from cooperation between peers, trust in the third party (TP), too much overhead, or adverse effects on the result provided by the service.
As shown in Figure 5, we have selected 11 approaches, grouped in four categories. Here we provide a brief description of each of them.

Date Encryption
This method sends user data to SP as encrypted strings. Other methods dealing with data apply Data Mining or Statistical functions. In this approach, processing and filtering of data is undertaken before sending it to a service provider (SP). Data mining and statistical models are examples of data analytics methods used in this approach. A drawback of this approach is that it can be used only in applications where the sensitive data is not involved [35]. The cases where encryption is used for protection of user's data from attacker are dealt with the trusted SP.

Access Control
When using this approach, the user would receive a notification whenever SP wanted to use their data. Although the users would have full access and control of their data but there is no surety that SP wouldn't violate the privacy of the user [22,[36][37][38].

Law and Policy
It is a measure which requires the user to be aware of their privacy as well as the policy of SP [38,39].

Clocking Area and K-Anonymity
In these methods, a broker or mediator server, between users and SP is used to protect users' privacy in a specific area from SP by hiding their identities (Anonymizer), creating a homogeneous cluster for users, storing some data in cache for answering future queries without connecting to SP, or storing historical information of users to predicate their transaction and then obfuscate it. Details of this method can be found in [20,22,24,40]. Although the users wouldn't be required to trust SP, they instead would have to trust TP.

Cooperation
User is required to cooperate with another peer or several peers to protect privacy. As a result, peers are required to swap their queries and answers, storing some answers in cache, transmitting the same query, or creating shared clacking area. How the peers can be trusted is still an open problem [41].

Caching
User must rely on the cache, where many of the previous highly requested queries' answers are stored to offset the number of connections with the SP. This approach can be integrated with other approaches to enhance the privacy level. However, determination of a suitable position for cache and the issue of trust are still wide-open problems [42].

Anonymity
A user sends the query with nickname or using hashing code instead of his identity. But it is not effective and easy to be hacked by attacker. To enhance this approach, users are required to change their nickname each time or whenever they enter a new area to prevent their tracing by SP. This is known as Mix-zone, which isn't very effective to protect the privacy of users because SP can easily trace the location and reconnect with a new nickname [43].

Obfuscation
In this case, the users adulterate their data and by adding some noise before sending it to SP to protect themselves. As a result, this method would affect the accuracy of results and the performance wouldn't be optimum [44]. Other aspects of this approach are related to the TTP approach, as described above.

Dummies
A user sends a group of queries (K) to the SP instead of just one, (K-1) of those will be dummies and the remaining one will be the actual query. That is done to prevent the SP from profiling the user information. The drawback of this approach lies in the fact that generation of effective dummies isn't a complex task, and the process adversely impacts the performance as well. For details, refer to [45].

Private Information Retrieval (PIR)
User retrieves a big part of data from the SP about a specific area/block, and store it safely for future queries. Moreover, this data can be encrypted or distributed to many servers to increase the level of privacy. This process severely affects the performance and creates an accumulated backlog of data for users to store and handle it [46].
There is also another method called Hybrid which uses more than one technique to protect the privacy. Examples are caching with dummies, encryption with obfuscation, and so on [47,48].
From the preceding discussion, it is evident that the real problem in all of these approaches is that of the trust in another party which can be the SP, TP, or Peer, which are capable of breaching privacy and so the user would remain vulnerable in every situation. Attempts to relieve users have resulted in more problems to them and, in addition, adversely effected the accuracy of results [22,35].
To address these issues completely, our approach, proposed in this article, addresses three scenarios, namely: Blind Third Party (BTP), Blind Peer B L P, and Integrated Blind Parties (IBPs). In our approach, we have successfully eliminated need for trust with any of the above-mentioned parties, without impacting on the precision of results. Moreover, this approach can be used in different applications of IoT, not just the location-based services applications.

Proposed System
Inspiration for the proposed approach came from trying to answer the question: Why do we have to trust the third party to protect the user from the SP? Of course, in order to avoid trusting SP, we shift the problem of trust from one party (SP) to another (TP/Server), which sometimes is easier to deal with. In dealing with TP/Server in B L A, we define a new protocol for dealing with SP.
As for the protection from TP/Server, we propose encryption with two different types of keys, namely: Public/Private Keys and Session Keys to hide the identity of the user from SP and that of the data from the mediator (TP/Server). These measures enable users to reap all the benefits flowing from the usage of TP/Server without creating any avenue for breach of data. Therefore, TP/Server can be regarded as blind and that is why we call it Blind TP/Pear or BTP/B L P. Steps of the query transaction in B L A are summarized in Figure 6. Our data model is depicted in Figure 7. It should be noted that we change general format of IoT query by adding to its session key as a new part.  Here we provide three scenarios of the proposed approach in detail.

First Scenario-Blind Third Party (BTP)
This scenario provides a solution to an earlier drawback of exposing data in the TTP approach above (E). In the proposed BTP approach, the user would avail all the benefits of using the TP without having to reveal any data. For that reason, we have renamed TP to BTP. This scenario makes the traditional TTP approach usable by removing its anomaly (trust in TP). The user can use this scenario to expedite the response even in the absence of other peers in the same area or with low resources.
This scenario is depicted in Figure 8, and the method is presented in Figure 9.

Advantages of BTP Approach
• The approach provides full protection for user privacy from SP as it can't track the user because BTP would relay the queries of all users in the area/cell.

•
There would be full protection for the data of the user from all outer attackers because the data would be encrypted in both directions. • BTP itself can't penetrate into the privacy of users who rely on it.

•
The user would connect only with the BTP and hence there wouldn't be any overload on the user. • BTP's role is only to change the identities of users, and hence would be moderately loaded.

•
There wouldn't be additional load on the link, as the number of connections with the SP would stay the same.

•
The BTP approach establishes an integration between TTP and Encryption approaches.

•
The BTP approach is also capable of supporting IoT applications in which sensors frequently output data which is saved in the SP, and which involve steady stream of queries (not just the discrete ones), and generation of fake IDs which are repeatedly used for the same user availing similar services. An example of such a case is discussed in Section 8.

Disadvantages of BTP Approach
• There is a possibility of collusion between the BTP and SP to breach user's privacy; however, as the users normally deal with credible service providers, this would be highly improbable. Nevertheless, this disadvantage can be considered to be the main drawback of the BTP approach.

•
Encryption may cause overload on some user's devices; however the proposed approach tries to relax this anomaly by using symmetric keys for encryption of the larger amount of data (i.e., the result of the query) and not relying on asymmetric keys in either direction.

•
Using encryption would result in more power consumption by users' devices

Second Scenario-Blind Peer Approach
This scenario addresses the main drawback of the previous scenario in which BTP may cooperate with SP to violate the privacy of the users. In the B L P approach, the new idea is to rely on cooperation with many peers instead of dealing with just the same TP. Therefore, a user's query would be exchanged with another peer in the same area, and subsequently be encrypted by PK of SP (SP PK ), leaving the other peer with only the option to forward the query to the SP, who would decrypt and resolve the query. The SP would then encrypt the answer by the Session Key (S Key ) and send it to the peer, while being unable to read the answer, would forward it to the main user. The same scenario would be repeated with peers' other queries as depicted in Figure 10. The steps in Figure 11 elaborate this method.  While in B L P, the user seeks more privacy vs. response time, because in the previous scenario the BTP can deal with the SP to disclose the users' privacy, even though this would be a rare possibility in real life. But searching for other peers in the hope of resolving the query, can affect the response time. Therefore, this scenario would depend on the cooperation between peers and swapping mechanism (P2PCache) but only after integrating it with Blind Concept and applying B L A protocols, can it avoid its disadvantages.

Advantages of Blind Peers
• All advantages of BTP and P2PCache Approach [47] are retained and all disadvantages are removed. In particular, B L P removes the possibility of collusion between BTP and SP, because each time the user would deal with a different peer.

•
There would not be any additional load on any peer and there would be no bottleneck in the TP.

Disadvantages of Blind Peers
• B L P Approach would have same disadvantages as in BTP, except for the possibility of collusion between BTP and SP.

•
Occasionally, a suitable peer for cooperation may not be available.

Third Scenario-Integrated Blind Parties (IBPs)-Adaptive Scenario
This IBPs approach doubles the level of privacy by way of integration between the BTP and B L P, and removes the shortcomings of the group of seven approaches. In this approach, the user can rely only on the B L P if a peer isn't available in the area. Moreover, the user can swap the query with another peer without encryption in case of scarcity of resources. In that case the peer can apply the BTP scenario instead of the user doing it. This scenario can also be adapted in the group of seven approaches [47]. Figure 12 depicts steps of IBPs. We could also use Cache with Bloom filter (to be discussed in Algorithm 2) to enhance the performance in the same way as is proposed in the DCA method [48].

Algorithm of IBPs
We provide two algorithms of IBPs. First algorithm of IBPs (Algorithm 1) is provided in Figure 13 and second (Algorithm 2) is provided in Figure 14.

Analysis of Attacks in Blind Approach
Here a discussion on the resilience of the three kinds of the Blind approaches, and the kind of attacks [22,24,27,48] they can deal with, follows.

Resilience of BTP, Blind Peers and IBPs
The proposed techniques in the Blind Approach can deal with many types of attacks/skills of attackers as follows.

Semantic Context Attack
In cases where the attacker has additional information such as user's age, profession, it may not be too difficult to detect the owner of the query. In BTP and B L P, queries are routed by the BTP or B L P, therefore there wouldn't be any connection between the user and SP, and hence the user would be immune to attacks by the SP.

Trajectory Attack
SP can track the user's positions to trace their path of their movement, then SP would detect new position of the user even if the user adds some obfuscation to it.

Historical (Temporal) Attack
A Historical attack can take place if the attacker has historical information. This cannot happen in cases of BTP and B L P as the SP wouldn't have the credentials to extract historical information about users.

Inversion Attack
An Inversion Attack can take place if the attacker can extract information about users from the algorithms used. However, in the cases of BTP and B L P, the only available information would be the peers routing the queries to the SP, and hence information about the users wouldn't be detectable.
In addition to the above analysis, B L Ps offer a superior and new level of resistance to physical attacks on peer devices. To illustrate this, consider that a user exchanges data with a peer p. Suppose that some malicious code manages to erase the memory of P before U gets the answer back. Unable to determine the identity, P would be unable to send the answer back to U. In this case P can simply broadcast the answer to all peers in the cell, enabling U to decrypt the answer, which would be disappointing for the attacker as they would fail to enforce Denial of Service (DoS) in the Blind Approach.

Enhanced Protection in BTP, Blind Peers and IBPs Approaches
The BTP, B L P and IBPs techniques also achieve a great deal of enhancement of the level of protection and are designed to prevent an attacker from breaking in depending on some skills. A discussion on this follows [22,48].

Closeness (Uniformity)
Closeness is a property which is achieved when the areas covered by two different BTPs are similar to each other. This can be realized by having only one BTP in each cell.

Diversity
For Diversity to take place, queries coming from any cell must be different. In the case of B L P, this will be achieved because each peer would send many queries belonging to several other peers. In the case of BTP, this wouldn't arise because BTP sends queries instead of user information.

Homogeneity
Homogeneity refers to a situation where the points of interests (POIs) or queries of users should be in a different field or major. Thus, Homogeneity is similar to Diversity.

Knowledge about Query Map
If the attacker has knowledge about the query map, it would cause shrinking of the user area; however, it wouldn't affect the privacy in BTP or B L P because of the congestion of users as the remaining area would increase and so would the level of privacy.

Analysis of Privacy Factors and Matrices
There are five groups of factors to evaluate and compare privacy approaches [22,24,42,47,48].

K-Anonymity
K-Anonymity refers to the number of real queries of a user that attacker has. It is formulated as where K refers to the number of false queries or dummies, and K ≥ 0.

Entropy
Entropy is the percentage of accurate information as compared to the whole information that the attacker has about the victim. It is a general and accurate metric and other metrics rely on it. The larger the entropy (E), the greater the privacy level.
where P i denotes the probability of real query from k queries.

Ubiquity
Ubiquity is related to Entropy; however, it refers to the degree of anonymous data that can't be attributed to the user without others. A larger value means more privacy level.

Error Estimation
Error Estimation is also related to Entropy; however, it refers to the percentage of error estimation from the attacker about the user according to the information it has.

Need to Trust
We classified the need to trust to five different levels which are: Trust with SP, Trust with TP, Trust with Peer, and Simi-Trust with SP/TP. For trust in all these levels, the user sends part of the data or encrypted (or distorted) data, which is the best choice according to privacy overview, because trusting any party amounts to providing personal information.

Performance
This factor affects the response time, cost of computation, and power consumption. It is also related to many metrics, which are: • Number of queries that are sent each time to SP.

•
Size of transferred data in both the directions, whether the result is for one query, multi-queries, or large part of server data which covers a specific area/cell.

•
Using encryption and the type of encryption algorithm, when (a) the asymmetric requests take more time compared to symmetric ones, and (b) symmetric algorithm affect the level of security and the cost of computation affecting time and performance.

•
The need for post-processing for the result to find out the final result, as the results after obfuscation queries need some processing to make them suitable for the user, and that involves more cost and time.

Accuracy of Results
Where some approaches and techniques affect the accuracy of relevant results returned by SP, which in turn adversely impacts the evaluation of this technique.

Cache Hit Ratio
It can be added in the approaches that use cache technique to answer future queries and reduce the number of connections with SP. Of course, there is a trade-off between each factor and privacy level factor.

Superiority of BTP and Blind Peers
As discussed in the previous section BTP/B L P provides the highest level of privacy because, users send their queries to other users instead of sending them to the SP. More and more queries submitted by a user would enhance the privacy pushing the K-Anonymity metric to zero, and Entropy (E) to 1, because all queries with the SP from the same user would have the same probability while in the same time the user ID would be hidden. At the same time U would be maximum (U = 2), and EE of SP would be 100%. The involvement of other attackers wouldn't arise because of the encryption in both directions.
For Trust Factor, BTP and B L P are also not required to be trusted, which results in achieving the best level of privacy compared to all previous techniques, namely: Dummy, Obfuscation, PIR, and Cache [in the literature review], although some of them have their own methods to avoid trust in other parties.
Because of encryption, the B L A would have lower performance than the one described in [48] or P2PCache [47] approaches, but the computations can be significantly reduced by asymmetric encryption in case when user sends a query with the limited data size whereas in another direction the SP would use symmetric encryption for results such as DES3/AES (Key = 128/192b) with higher level of protection (user changing key each time) and less computation. However, the number of queries sent to the SP in each connection would only be 1, so there would be no need for post-processing of the returned results.
Finally, according to saving the accuracy of result, it can be achieved here. Moreover, the proposed approach can also use the cache techniques. In this case the cache hit-ratio will be maximum because it depended on only real queries of users

Implementation and Results
Here we provide simulations resulting from comparing figures related to factors, and metrics between B L A and other approaches (Enhances-Cache [42], P2PCache [47], and DCA [48]). We have used MATLAB 2015 in addition to Microsoft Excel 2016 for achieving these simulations. The simulation is based on the following hypotheses, which are same as used in some of the group of seven approaches.

•
The study area is divided to 100 × 100 Cells • There is cache in each access point in each cell.

•
The Cache size is 100 K (where POIs of Google-API for the New-York are around 250,000, which required about 250 MB for storage) • Each query is less than 1 KB in size • There are 10,000 users distributed randomly in these cells • There is a list of 100 POIs.

•
The network is 3G/4G WI-FI connection in the smart city environment or in specific areas and infrastructure For the purpose of comparing our approach with the previous ones, we define the following terms which are used as axes.  Figure 15 shows that BTP was very close to DCA, and it took the least time, because BTP used encryption which slightly affected the time of each query; however, it still was better than the corresponding time taken by DCA. The same thing happened with B L P and P2PCache but P2PCache was better from B L P because they both used same scenario and B L P used encryption as an additional measure to achieve more privacy. Enhanced-caDSA was the worst performer because it used an algorithm for generating dummies with higher prediction to be requested in the future, and in the same time it sent many queries to the SP in each connection. Finally, use of encryption in BTP and B L P affected only a little timewise, but enhanced the level of privacy significantly. Moreover, as we explained before, the Blind Approach can be used with the cache technique to enhance the performance if needed. Figure 16 shows that BTP achieved best response time for user's query, but it came very close to the performance of DCA because the dealing with one party as server in BTP with encryption is surely better than dealing with two caches, and frequently checking whether a query is answered or not. While B L P was the worst performer because it used encryption and in the same time required to search and connect with an available peer impacting on the response time, nevertheless providing the best privacy protection.  Figure 17 shows that Blind Approach (BTP/B L P) achieved the lowest cost of number queries alongside DCA and P2PCache. This was because all the techniques sent only one query to SP without any additional dummies or without requesting additional results. Encryption didn't have any impact on the number of queries but it did (slightly) on the size; which can be interpreted as an improvement towards achieving more security and full privacy.

Privacy Level (Entropy)
As is evident from Figure 18, B L A achieved maximum Entropy (E = 1) because the user didn't send any personal information. Moreover, B L A supersedes the group of seven approaches because of the Trust Factor.  Figure 19 shows that B L A attains optimum privacy with maximum Ubiquity (U = 2). The reason for this is that each user in B L A sends many queries belonging to different users available everywhere in the cell.

Cache Hit Ratio
As we discussed earlier, B L A can be used with the cache technique for enhancing performance vs. privacy. Figure 20 shows the result in case when B L A with DCA and P2PCache used (same) Algorithm 2 with Bloom Filter and stored only real queries of users in the cache without any dummies. These measures enhanced the hit-ratio and increased the probability of participation of these queries in future. Figure 21 shows that Algorithm 2 of the Blind Approach required less time for managing the cache as compared to Enhanced-CaDSA which uses the freshness method.

Blind Approach in IoT Applications
Applications of IoT can be found in almost every facet of our life. Let us look at some applications of in some important domains.

Blind Approach and Healthcare Data
Our health is the most important aspect of day to day enjoyment of our lives and is directly linked to the socio-economic and political fabric of the society. With the advent of new technological paradigms, many researchers are suggesting better ways to keep ourselves healthy. In particular, IoT (smart watch, wearable sensors, WSNs, etc.) is providing safe, ubiquitous, and other smarter and ways to better manage health services.
Often health related data is personal and sensitive and requires proper protection. While using the emerging and innovative technologies to manage health, the biggest threat is to preserve privacy and security of the health data. In many applications, health data from devices such as WSNs and RFID tags is continuously streamed to health centers and/or SPs for analysis. Historically, health centers have enjoyed the confidence of protecting health data, but the same cannot be said about SPs.
BTP is the first approach which has proposed efficient and effective solutions in cases such as protection of health data from malicious attacks by SPs themselves. With BTP, users need not disclose the identity to SP or even the health center, and at the same time are not required to reveal to the TP the way it deals with the SP instead of the user. Figure 22 shows working of BTP in this case.

Blind Approach and Smart Transport Systems
An important application in the smart city is that of the transportation. Typically, in a smart city, a smart vehicle would send coordinates of its location to SP to seek traffic related information about and coordinates of the points of interest. While obliging with the request, a scrupulous SP with the help of vehicle location at a given time, may detect user's personal and sensitive information such as home location, habits, movements, workplace, marital status, health condition, religion, etc. Personal information of some individuals may be politically sensitive and linked to their security. Using Bluetooth specifications [49] by generating random IDs (with same location) for a user wouldn't be effective in this case. It is because the knowledge of location of these IDs with the time of request, would enable SP to detect these IDs, leading to finding the real ID of the user. In case the smart vehicle makes use of Obfuscations or Dummies technique; the accuracy and effectiveness would depend on the number of vehicles in each area. As we know that B L Ps approach depends on swapping peers before sending to SP. Thus, the number of vehicles wouldn't affect in any area, as shown in Figure 23. Moreover, SP would collect and store misleading information (places and destinations) about user. It should also be noted that due to swapping, each peer would be blind and unable to do anything beyond reading the query.

Blind Approach and Smart Home
Another application of our approach can be traced in Smart Homes, which use many smart tools such as smart TV, smart light, smart AC, and smart fridge, to name a few. Smart home tools periodically connect to manufactures to enhance the quality of services, which exposes the privacy of users by empowering manufactures to collect a lot of data about users and analyzing it. This data may include home, office, and leisure times of users and their properties. This situation can be avoided by using B L Ps between different zones of Smart Homes depending upon Fogs [50]; and using BTP, depending on the core fog, as shown in Figure 24.

Blind Approach and Underwater Wireless Sensor Network
Another application of Blind Approach [51] used by military or a monitoring organization searching for oil or fossil fuels is shown in Figure 25. Here B L A can be used to prevent eavesdropping, and malicious node accessing data from other sources. B L A can also be used to thwart attacks in this environment such as the attack that monitors the most active node to get information about the location of the main station. In other words, the swapping method in many situations can prevent the attacker to achieve its goal.    Here it should be noted that we are not claiming that B L A can solve all the problems of privacy in all IoT applications; nevertheless, as demonstrated in this article, it provides protection for some applications where the group of seven approaches fail to do so. Indeed, so far, there isn't an approach which would work in all situations. To streamline what could be done in different situations, we are working on a framework which would contain many techniques, including B L A, which would automatically select suitable technique(s) for each application or service. As a drawback of our approach, it should be noted that there is a possibility of breaking into BTP in scenarios such as that in the case of health. This will happen if TP and SP collate to disclose the identity and other information of the user. Although such would be a rare occasion, nevertheless it would be probable. We intend to return to this aspect and propose a resolution in the future.

Conclusions
In this paper, we have introduced B L A as a superior approach for preserving privacy in IoT applications, which has three scenarios to provide effective and efficient solutions in different cases. Our approach amicably resolves the trust issue as users can protect their privacy without having to trust the third party, which has been a challenging open problem in all methods relying on third party trust. The B L A also introduces a new method for sending dummies to the SP without an overhead on the user, server, or network. In addition, B L A provides accuracy in returning the result back to the user.
The simulation and results of comparison demonstrate superiority of B L A over the group of seven approaches, with a slight increase in the average (response) time. This can be viewed as a trade-off between the average (response) time and privacy and security of data. As observed in the simulation results, BTP/B L P achieved the highest level of privacy without relying on the third-party trust or compromising on performance, cost, or accuracy. We have seen that our approach supports many applications of IoT, and also provides flexibility to use the idea of BTP/B L P with the group of seven approaches.
In the future, we intend to provide enhanced protocols for addressing the privacy issue with the M-Health application, and also to focus on critical applications of underwater wireless sensors. We also endeavor to show how we can detect any attempt from BTP to get information about any user. We shall also try to provide a way to reduce the effect of encryption on performance by confining its applications to the most sensitive part (instead of the whole query), and deal with the drawbacks of B L A.

Conflicts of Interest:
The authors declare no conflict of interest.


Simulation and Analysis of 37 GHz Millimeter Wave Propagation Characteristics in Indoor Office Environment

In this paper, the method of shooting and bounding ray tracing/image (SBR/IM) is used in the simulation of indoor office environment, with which the study on millimeter-wave propagation characteristics can be processed well. Through simulation parameters such as path loss, received power and root mean square (RMS) delay spread is obtained and analyzed. All the results of this paper could be used as a reference when setting the position of transmitter and selecting parameters in indoor office environments.

With the rapid development of economy in recent years, the proportion of global urbanization is increasing, and the main places where people live and work are more and more concentrated in indoor environments [3]. Statistics show that for the time people spend, 80% to 90% is indoors, 70% is on mobile phone use, for data transmission, 80% of them is occurred indoors. Nowadays, indoor short-range wireless interconnection (Wireless Local Area Network, WLAN for short) technology is developing rapidly [4], and people are in urgent need of high-quality indoor communication [5].
The most outstanding feature of millimeter wave is that the bandwidth it can provide reach up to hundreds of megabytes. However, due to the serious attenuation cause by obstacles, the apply of millimeter wave may be limited in indoor environment. Therefore, it is necessary to analyze millimeter-wave channel propagation characteristics in indoor radio environment [6][7] In this paper, millimeter-wave propagation of 37 GHz are simulated in indoor office environment.

SIMULATION ENVIRONMENT
The material for study of indoor office environment is from literature [8]. The office space modeled for measurement and the simulation is showed in Figure 1. The measurement environment is a office which width is much larger than depth (49.8m 16m) and its height is 2.7m. The height of partition and desk is 1.0m and 0.7m respectively. There are two transmitter antennas Tx1 and Tx2, for position of the two transmitters in the room, Tx1 is placed in the center and Tx2 is on the left side, which is marked in Figure 1. Both two transmitters are set at the height of 2.6m. The receiver antenna are set at the height of 1.5m, which is just above the desk and the partition. .

Path Loss
Path loss, or propagation loss, refers to the loss caused by the propagation of radio waves in space. It is caused by the radiation diffusion of transmitting power and the propagation characteristics of channels. It reflects the change of the average received signal power in macro scope. In theory, the path loss is the same for the same transceiver distance. But in practice, it is often found that the receiving power at different receiving points of the same receiving distance varies greatly, and even the receiving power at the same receiving point fluctuates greatly at different time points. The path loss function for indoor scenario is definited by the difference value between transmitter and receiver, which function can be shown as following.
PL (d 0 ) is the path loss determined by measurement. The value of d 0 is 1m in indoor environment commonly. d is the distance between transmitter and receiver, n is the path loss exponent and S is the shadow fading. The following two figure is result obtained from simulation. It shows that the path loss cecome bigger while the distanse become longer, and in the scenario of indoor office, path loss is smaller than that of free space under the same condition. For office the path loss exponent is 1.95 and for free space it is 2.

Received Power
The values of received power of 37.1 GHz through simulation are illustrated as shown in Figure 3. Obviously, the power become smaller while Tx-Rx distance become bigger. The received power from two transmitters are both range from -65 dBm to -45 dBm. With Tx2 as the transmitting antenna, the receiving antenna moves on the nine route. Taking three receiving point(P1 to P3 in Figure 1), each point has twenty five paths of received power. For the point P1 the total received power is -6 10 × 9.32 mW, for the total twenty five receiving paths there is a path with no reflection, four paths with once time reflection, nine paths with twice reflection and eleven paths with three times of reflection. The received power of the path with no reflection is  The distribution of received power for point P2 and P3 are shown in Table II. It can be seen in the indoor office environment that the received power of the path with no reflection is the largest, followed by the path with once reflection, the path with twice reflection and the path with three times of reflection contribute little to the total power.

RMS Delay Spread
RMS delay spread is caused by the different arrival time of different path of signal, which is defined as and the definition of the average arrival time is Where i p and i t is parameter of path i, means received power and arrival time.

CONCLUTION
In this paper, three different parameters are discussed: path loss, receiving power and RMS delay spread. In terms of path loss, it is observed that path loss component is smaller than that of free space with reflections from walls and desks and chairs. Although there is less interference in free space, there is no reflection wave, so the path loss index is higher than that of office environment. This point is also reflected in the received power. It can be seen that although the contribution of reflected wave becomes less and less as the number of reflections increases, it is the negligible reflected wave that makes the path loss of indoor environment lower. For the part of RMS delay spread, it is observed that the closer the receiving point is to the transmitting point, the smaller the delay spread is. This paper provides a material for channel research of indoor office. The shortcoming is that there is a lack of NLOS environment, in addition, the research objectives are somewhat single. More detailed research is needed on the study of 37 Ghz millimeter wave propagation.