Relative Depth Order Estimation Using Multi-scale Densely Connected Convolutional Networks

We study the problem of estimating the relative depth order of point pairs in a monocular image. Recent advances mainly focus on using deep convolutional neural networks (DCNNs) to learn and infer the ordinal information from multiple contextual information of the points pair such as global scene context, local contextual information, and the locations. However, it remains unclear how much each context contributes to the task. To address this, we first examine the contribution of each context cue [1], [2] to the performance in the context of depth order estimation. We find out the local context surrounding the points pair contributes the most and the global scene context helps little. Based on the findings, we propose a simple method, using a multi-scale densely-connected network to tackle the task. Instead of learning the global structure, we dedicate to explore the local structure by learning to regress from regions of multiple sizes around the point pairs. Moreover, we use the recent densely connected network [3] to encourage substantial feature reuse as well as deepen our network to boost the performance. We show in experiments that the results of our approach is on par with or better than the state-of-the-art methods with the benefit of using only a small number of training data.

Abstract-We study the problem of estimating the relative depth order of point pairs in a monocular image. Recent advances [1], [2] mainly focus on using deep convolutional neural networks (DCNNs) to learn and infer the ordinal information from multiple contextual information of the points pair such as global scene context, local contextual information, and the locations. However, it remains unclear how much each context contributes to the task. To address this, we first examine the contribution of each context cue [1], [2] to the performance in the context of depth order estimation. We find out the local context surrounding the points pair contributes the most and the global scene context helps little. Based on the findings, we propose a simple method, using a multi-scale densely-connected network to tackle the task. Instead of learning the global structure, we dedicate to explore the local structure by learning to regress from regions of multiple sizes around the point pairs. Moreover, we use the recent densely connected network [3] to encourage substantial feature reuse as well as deepen our network to boost the performance. We show in experiments that the results of our approach is on par with or better than the state-of-the-art methods with the benefit of using only a small number of training data.
Index Terms-Relative depth order estimation, densely connected network, deep learning

I. INTRODUCTION
T HE depth ordinal information of two objects (points) in an image is an important visual cue for many computer vision tasks such as objects classification [4], [5] and semantic segmentation [6], [7], [8]. The objective is to know which one is closer or further (or at the same depth) to the camera, given a pair of pixels. To estimate relative depth order, traditional methods mainly depends on objects' boundary and junction characteristics such as T-junction, convexity/concavity and inclusion [9], [10], [11]. The accuracy of these methods is limited. Recently, Convolutional Neural Networks (CNNs) have achieved remarkable success on many vision tasks such as object recognition [12], [13], [14] and semantic segmentation [15], [16]. Motivated by the powerful visual representation and generalization capability, recent works [1], [2] of depth estimation have also used CNNs to estimate the ordinal information between the point pairs, and demonstrated superior performance. In [1], [2], both methods attempt to explore multiple features, which include the appearance of the points, the local contextual information, the global scene context and so on. The idea is to use the visual cues as much as possible to improve the models' performance. Moreover, they both apply the multi-stream network structure. Zhou et al. [2] R. Deng and S. Liu are with Central South University, China. T. Zhao is with Tsinghua University, China. C. Shen is with the University of Adelaide, Australia. This work was done when R. Deng was visiting the University of Adelaide. Fig. 1: The overall pipeline of estimating the depth order of a pair of points. Given a pair of points, we extract its local contextual information and feed it to the proposed model to perform the prediction. The output of the model is the probability of a three-way classification, which are three ordinal relationships "at the same depth", "further" and "closer".
concatenates all the convolutional features. In contrast, Zoran et al.'s [1] network applies hierarchical concatenation of the convolutional features-the global feature first concatenates with the RoI mask and is fed into a fully connected layer, then concatenates with the other convolutional features and the masks.
Thus, the studies of the recent works have mainly focused on combining various contextual information to train a network, yet without demonstrating if each feature is useful. In this work, we attempt to achieve two objectives: 1) empirically examine the contribution of each context cue; 2) and present a practical model to estimate the ordinal depth information. As we show in the next sections, such an exploration has resulted in several interesting findings.
The global feature vs. the multi-scale local features. Following the insights presented in [1], [2], it makes sense to take advantage of more types of the contextual information for improving the accuracy of the model. However, neither of the them offers an analysis of the contributions of each cue. It is crucial to find if each cue plays an important role in the model. For an ineffective cue, we can remove it to make the model simpler. Motivated by this, we conduct an experiment to examine the effectiveness of each cue. Our result shows that the global scene context makes the least contribution in terms of the performance of the model. We provide the detailed arXiv:1707.08063v2 [cs.CV] 27 Jul 2017 explanation in Section III-A.
Consider two points (red and green) located in the purple bounding box in Fig. 2. If we remove the bookcase or the carpet from the scene, it would not affect the depth order of the points. We argue that the global structure of the scene is not necessarily useful for the task because the global information is redundant. Instead, the local context surrounding the points is critical. The local context contains abundant monocular cues such as occlusion, shadows, texture gradient and so on [17], [18], [19], which determine the relative depth of the objects. In this paper, we make CNN learn these relative depth cues by feeding the local background context surrounding the points. In particular, the context is in the form of multiple scales. Multi-scale features have an extended history application in computer vision and recently it has also been found very useful in the tasks like semantic segmentation [15], [16], stereo vision [20] and high-quality natural images producing [21], integrating with DCNN. Compared with the complex global structure, the local surrounding context is much simpler and easy to learn. Our experimental results show this simple changethat is, from learning the global context to learning the multiscale local context-leads to a significant improvement in the performance.
In this paper, we employ a novel deepening technique, namely densely connected network (DenseNet) [3], to obtain powerful visual representation and improve the performance. The method has been reported to achieve state-of-the-art performance on image classification tasks. Its principal characteristic is the dense connectivity. That is, each layer of the structure is directly connected to every other layer in a feed-forward way. It takes advantage of the feature reuse and strengthens feature propagation. By using the densely connected network, we have 1.7% percent accuracy improvement compared with our baseline model proposed in Section III-B on the NYU Depth Dataset. We compare the DenseNet with the popular deepening technique, deep residual learning (ResNet) [14] to demonstrate its advantages.
Last, we integrate the learned prior (the outputs of the proposed model which are the probabilities of three ordinal relationships) into the energy minimization proposed by Zoran et al. [1], such that we obtain the relative depth of the entire image from the ordinal estimates. The key difference is that we solve the minimization in the log space and introduce a useful smoothness term, which improves the details of the recovered depth map.
In summary, our contributions are as follows. 1) We present a detailed experimental study on the usefulness the contextual information used in prior works [1], [2] by examining their effects for the task of depth order estimation.
2) We present a simple yet effective model, using the multi-scale framework and densely connected network, which  makes the learning much easier and achieves state-of-theart performance. However, our method only uses hundreds of training images, while recent state-of-the-art methods [28] usually used many more images (220K images) for training.
3) Last, we solve a constrained quadratic optimization problem similar to [1] to reconstruct the depth map from the ordinal estimations. We introduce a smoothness term to improve the result.
The rest of this paper is organized as follows. In Section II we summarize recent advances on ordinal relationship estimation and the CNN deepening techniques related to our work. In Section III, we present our method in detail, including the examination of the role of each context, describing the proposed model and how we reconstruct the depth map from the ordinal relationship of thousands of point pairs. In section IV, we provide quantities of experiments and analysis to validate the effectiveness of the proposed model.

II. RELATED WORKS
In this section, we briefly review the works of monocular relative depth estimation, in particular, those using CNN. Furthermore, we briefly review the characteristics of the current popular CNN deepening techniques.
Depth ordinal relationship estimation in monocular images Computer vision approaches handling monocular relative depth estimation were profoundly influenced by psycho-visual theory [17], [18], which suggests T-junctions as one of the fundamental of monocular depth perception. Many works relied on developing the computational model to interpret and extract the T-junctions in an image [9], [11], [8], [9], [10].
Recently, due to the wide adoption of affordable depth sensors, datasets such as the NYU Depth dataset [29] and KITTI dataset [30], [31] become available, thus leading to the trend of solving the problem as a supervised learning task using CNN. Pioneering work of Zoran et al. [1] proposed The image is segmented into many superpixels. According to the superpixels, we find their centroids as the selected points and visualized in (c). We pair these points with their second-order neighbors in (d).
an end-to-end system to estimate the depth order of points pairs. Compared with another ordinal estimation work which also utilized CNN [2], Zoran et al. add one more bounding box which contains the important visual cues for the task and a different location expression in their network structure. We consider such the design very practical, which makes the visual cues be automatically learned and inferred by CNN. Since the NYU and KITTI datasets have no direct annotations for the task, Chen et al. [28] proposed a dataset named Depth in the Wild (DIW) which annotated the ordinal depth information between point pairs. Their deep model is an deep ranking model, and the output is a relative depth map. Our network is an end-to-end system. The output is the same as in [1], [2], which are three relationships, namely, "further", "closer" and "the same". However, we employ different input context cues and a much deeper architecture. We describe the details in the next section.
CNN deepening techniques The number of the layers of CNN has dramatically increased in recent years, from AlexNet [12], VGG [13], Inception [32] to ResNet [14]. The deepening of CNN is not merely repeating the 'Convolution-Relu-Pooling' process. With increasing the depth of a plain network, the performance of a deep model was often observed worse than a shallower model, due to the gradient vanishing and the optimization becoming underfit [14]. To address the issues, a well-designed structure is needed. In VGG [13], 3×3 filters are used throughout, and the entire network is divided into several blocks. Convolutional layers are stacked in the block. Between the blocks, the max pooling operation is used to reduce the feature map size. This design highly influences network structures proposed in recent years, as nowadays most of the structures use 3 × 3 convolutional filters and the block by block structure.
The characteristic of Inception network [32] is that their structure is not only deep but also very wide. In their block, several streams with different filter sizes (1 × 1, 3 × 3) are applied. ResNet [14] is the most successful network structure since it well alleviates the gradient vanishing problem. It is worth mentioning the highway network [22] because its underlying principle is similar to ResNet. ResNet applies skip connections and sum up the feature map every two or three layers to enhance the information flow, which is termed residual learning. Relying on residual learning, ResNet can extend its depth to more than a thousand layers and still achieves impressive performance.
The densely connected network is a new deep structure proposed by Huang et al. [3]. It takes advantage of the above useful designs such as small filters (3×3, 1×1), block by block structure (yet apply average pooling) and skip connection as well. In particular, its primary characteristic is the dense connectivity, which leads to the heavy feature reuse, information flow propagation, and good regularization. Considering these advantages, we apply the DenseNet in our proposed model. To verify our choice, we compare its effectiveness against ResNet and the baseline model, respectively. We observe the improvement over the proposed baseline model. The details are shown in Section IV.

POINTS
Estimating the depth order of point pairs is to explore the points' 3D relationship. It is very challenging, as the only information that we use is the 2D appearances. We propose to tackle this complex problem in multiple stages.
Firstly, since the quality of input features plays an important role in the training process, we analyze the effectiveness of the input feature (context) used in the state-of-the-art works [1], [2] by conducting a series of ablation experiments. We observe several interesting findings in the experiments. Secondly, based on the findings, we present the multi-scale model to make better use of the local contextual information surrounding the points. The method is simple, which extracts three gradually increased bounding boxes around the points (shown in Fig. 2) and feeds the contents of the bounding boxes into a deep CNN, instead of the global scene context used in [1], [2]. Thirdly, motivated by the recent success of very deep CNN, we manage to obtain the better performance by using the DenseNet [3] to deepen the proposed model. To provide a reference, we compare the performance of the DenseNet with the ResNet [14] when they are both used to deepen the proposed model. Last, we reconstruct the depth map from the outputs of the proposed model to deliver an intuitive impression of the quality of the estimates.
Most of depth datasets such as the NYU dataset V2 [29] and KITTI [30] do not provide the annotations for ordinal estimation. Thus before estimating the ordinal relationship, we need to determine which pairs of points should be sampled in an image, then extract the required contextual information according to the selected points. We employ the strategy proposed by Zoran et al. [1] to achieve the goal. We over segment an image into many superpixels [33] and pick the centroid of the superpixels as the selected points. We then paired the points with their second-order neighbors to compare the depth order. The method is simple yet effective since a superpixel varies very smoothly and the centroid of the superpixel can represent the other points within it. Fig. 3 depicts an example of the selected points in an image and how the points are paired. After extracting the context, we start to examine the effectiveness of the context.

A. Examining the contextual information used for depth order estimation
The contextual information used in recent works [1], [2] has three types: the local contextual information, the global scene context, and the location information. As illustrated in Fig. 2, the local contexts are two patches on the comparison points and a bounding box surrounding the patches. The global context is a downscaled version of the input image. The location information that we use here is the mask. Note that in Zoran's model [1], the bounding box is considered as the global scene context. We would like to label it local context since it contains the local structure in an image.
We carry out a series of ablation experiments to examine the effectiveness of these contexts for the task. Before starting the experiments, we need to complete several preparations. Firstly, we simplify Zoran's model to make it more intuitive. The simplified model that we utilize is shown in Fig. 3(d). It removes three fully connected layers from Zoran's model (the first one is next to the concatenation of ROI and Downscaled Image, the second one is next to BB, the last one is next to the fully connected layer which has 300 outputs). However, the number of convolutional layers and the parameter setting of the corresponding convolutional layer in the simplified model, such as the filter size and stride, are the same with Zoran's. We split the simplified model into several basic components according to their types. There basic components are the patches, the bounding box, the masks and the downscale version of the image.
We start the experiments from only one component and test its performance, then gradually add other components one by one into the model and repeat the test. In this procedure, if the performance is improved, we can conclude the feature is likely to be effective for the task. If not, then the feature is not useful. Our test is conducted on the NYU depth dataset. Since the task is to learn the relationship between the points' appearances and the depth order, the first basic component that we keep in the model is the streams of two points' appearances (the patches). In the next round of test, we add the remaining components following the order of adding the bounding boxadding the masks-adding the downscaled image. In every round we test 10 times and select the most accurate result as the model's best performance. Everytime we test a new component, we fine tune the model from the previous model of the best performance. We present the results in the Table I. We test the accuracy of the predictions to measure the performance of the models. We take the result of the basic component model as the baseline. We also test the accuracy of Zoran's model to show the difference in the performance between the original model and the simplified model. Our baseline ( Fig. 4(a)) achieves 47.5%. Adding the bounding box component ( Fig. 4(b)) leads to a significant improvement-the accuracy increases to 52.0%. Applying the masks (Fig. 4(c)) gains a significant improvement as well: the accuracy further increases to 58.7%. However, the accuracy decreases to 58.1% after adding the downscaled image ( Fig. 4(d)). Zoran's model achieves 59.6% accuracy which is almost 1% more than our simplified model. We see three findings from the experiments: 1) The local background context and the location information are highly useful for the task; From the experiments, they contributes the most for the increase of the performance; 2) The global structure may not be required for the task, since adding the global scene context (the downscaled image), the performance shows a slight degradation; 3) The complex network structure, such as the hierarchy concatenation of the different streams [1] in the original Zoran's model, is also helpful for improving the performance.
Why would adding the global structure result in performance degradation? In theory, the global context has the global structure information which certainly contains the structure of the local context. Thus it should be helpful to boost the performance as well. We argue that the global structure may not be well learned, as its structure, which only has two layers, may not be able to efficiently learn the semantic content of the scene.
A recent work [34] by Gonzalez et al. investigates the responses of convolutional filters with semantic parts to analyze the internal representation of CNN. They argue that when CNNs handle the tasks like scene classification, it needs to be very deep, because the task is less related to object parts. According to their finding, we can deepen the network to make more convolutional filters response to the semantic parts of the scene. To demonstrate this point of view, we deepen the stream of the downscaled image from two layers to four layers. We set the filter size of the four layers as 3 × 3 . The first two layers' stride is 2 and the last two layers' stride is 1. As we expected, the accuracy increases to 59.1%, which is slightly better than non-downscaled-image model in Fig. 4(c). Gonzalez et al. suggest very deep architectures such as VGG [13] for various tasks since it has abundant semantic parts for finetuning. Coincidentally, Chakrabarti et al. [35] also utilize VGG to extract the coarse global structure of an image for absolute depth estimation. As a comparison, training from scratch with such a shallow depth is not ideal.

B. The proposed baseline model
As discussed, extracting the global structure from an image is very challenging, and most of the structure is not helpful to estimate the depth order of the points. Thus we choose a different route. We first look at the promising results from the deep visual correspondence embedding model [20] for inspiration to make better use of the local context. The model from [20] achieves improved accuracy of stereo matching by proposing an ensemble model of two patch scales. The large patches with richer information are less ambiguous, and the small patches have merits in details. The model combines the best of two worlds. We hypothesized that this may work better to learn the local structure from observing the local backgrounds at multiple scales with different sizes of contexts.
Therefore, we propose a multi-scale model to incorporate multi-scale local background contexts. We add two larger bounding box scales with the fields of view of 2.25×, 4× of the original bounding box (shown in Figure 6) all surrounding the points pair. In each stream of the model, we utilize strided convolution of four layers for feature map downsampling. The output features from the patches and multi-scale local surroundings are concatenated with the location information and fed into a fully-connected layer with 400 outputs. We train the network end-to-end with the log-softmax loss. The proposed model encourages to aggregate the multi-scale contextual information. We consider that three scales may be ideal for learning the crucial visual cues for the task. We set the model as a baseline in Section IV. In the next subsection, we show how to push the limit of the performance by deepening the proposed model.

C. Deepening with densely connected networks
We employ the densely connected networks proposed by Huang et al. [3] to deepen the proposed model. As we have introduced, it is a very deep CNN structure. The particular design of the network is the dense connectivity. In a densely connected network, the dense connectivity denotes that for each layer, it connects with all the other layers in the network. See Fig. 7(b). The characteristic differs from the pattern of traditional CNN, of which each layer is only connected with its adjacent layers. In a densely connected network, the feature maps of the early layers are concatenated as the input of the later layer. For the k th layer, it receives the feature maps of all preceding layers, i.e., Here H k (.) denotes a non-linear transformation includes Convolution, Batch Normalization [36] and rectified linear unit [37]. As the number of the connections grows quadratically with the depth, Huang et al. provide two solutions to control the growth. The first one is that they present a hyper-parameter k as the growth rate of the network. This growth rate k is representing the number of the outputs in a layer. For instance, k = 12 denotes that all the layers in the densely connected network have 12 output feature maps. The other one is dividing the entire network into several blocks. Thus the dense connectivity is only applied within the blocks. Between the blocks, average pooling is used to downsample the feature map. Now we show how to integrate the densely connected network in the proposed baseline model. As shown in Fig. 5, the idea is simple: the model contains four densely connected blocks with equal numbers of layers and average pooling layers in the streams of the multi-scale local context inputs. In these deepened streams, all the convolutional layers use filters with kernel size 3 × 3, stride 1 with zero-padding to keep the feature map size fixed. The length of the layers in each densely connected block is 5 and the growth rate k is 12. Note that, we have not deepened the streams of two points' appearance.
Since the selected points for comparison often locate in the smooth area, the appearances of the points rarely have complex textures and edges. In contrast, the multi-scale local contexts have abundant visual cues and textures, which should apply the deep structure. We demonstrate its benefit of enhanced feature reuse and alleviating the vanishing gradient issue through the experiments in Section IV. The results show that with deepening, our model achieves a significant improvement. Moreover, we take one step further to provide a reference of its deepening effect by comparing it with the state-of-the-art deepening technique ResNet proposed by He et al. [14]. We take one ResNet block for illustration, which is shown in Fig. 7. The characteristic of ResNet is utilizing skip connections to add up the feature maps every two or three layers to encourage the information flow. A residual block has several different structures. The one that we use here is "bottleneck," which is one 3 × 3 convolution surrounded by dimensionality reducing and expanding 1 × 1 convolution layers [24]. For the test configuration, we simply change the DenseNet blocks in the multi-scale streams to the ResNet "bottleneck" blocks and use max pooling operation instead of average pooling. We provide a comprehensive analysis of the deepening effect performance of both methods in the Section IV.

D. Recovering the depth map
Reconstructing the depth map from the estimates is to infer the global relationship of the selected points (we use each selected point to represent its superpixel) from the local relationship of point pairs. It is challenging since a considerable part of the estimated relationship between the points can be contradictory or ambiguous. We adopt the method proposed by Zoran et al. [1] to find the global solution, which poses this as a constrained quadratic optimization problem. In contrast to their approach, we solve the problem in log-space and introduce a smoothness term which is a reasonable prior for the task. Our objective is as follows: where E(x, ) is the energy function as follows, where o ∈ {=, >, <}, x are the depth values of the selected pixels. ω o,i,j is the depth ordinal estimation (the outputs of the proposed model which are the probabilities for three cases) of the ij-th pair. ∼ N (µ, σ 2 ) is a scalar slack variable for the ij-th pair. θ o (x i , x j , ) is L 2 distance which penalizes the depth estimate when it disagrees with the estimates of the proposed model, which consists of (3) L s (x) is the proposed smoothness term for the adjacent superpixels: It is weighted by the sum of local image gradient and the estimate of 'equal' case of the adjacent superpixels, which is where ρ controls the sensitivity of the image gradient-based weight, k 1 and k 2 control the proportion of the two terms in the above equation.
In the experiment, we set k 1 = k 2 = 0.5. Note that, we only consider adjacent superpixels. For non adjacent neighors, ω i,j = 0. The last term R( ) is a regularization term to bound the , which is where the mean values µ = , µ > , µ < and the variances σ 2 = , σ 2 > , σ 2 < are computed from the statistics of the training set, corresponding to 'equal', 'further' and 'closer' cases, respectively.
For the objective Equ. (1), L, U are the lower and upper bonds for the depth values. Once the objective is solved, we generate the depth map by floodfilling each superpixel with the corresponding values.

IV. EXPERIMENT
We demonstrate the effectiveness of the proposed model on three datasets including the NYU depth V2 dataset [29], KITTI dataset [30] and DIW dataset [28]. The experiments are organized in two categories: 1) We compare the proposed model with several baseline methods to show the benefits of integrating the multi-scale features and DenseNet structure; 2) We compare the proposed model with the state-of-theart methods on each dataset to show that our method performs better.
Training settings The implementation is in Caffe [39]. The contexts are extracted and resized to the specific resolutions as the inputs to the network. We show the size of each input in Table III. The kernel size, the stride, the outputs number of each layer are reported in Table III. Note that for the streams of the scales 1, 2, 3 in Fig. 5, the 'layer 1/2/3/4' means each convolutional layer before the corresponding DenseNet blocks. For each DenseNet block, 3 × 3 kernel size with zero padding are used in it to keep the feature size fixed and the growth k of the DenseNet is 5.
The network is learned end-to-end using a log-softmax loss. We train the model from scratch and use stochastic gradient descent (SGD) for optimization. For NYU V2 and KITTI depth datasets, we trained 300k iterations with minibatches of 256 pairs, for DIW, 400k iterations with minibatches of 128 pairs. The weight decay is 0.0005. We use an NVIDIA GeForce Titan X GPU, on which training process takes roughly 5 hours. For the ablation experiments, we reduce the training iterations to 200k. The other settings keep the same.
Metric error measure We utilize Weighted Kinect Disagreement Rate (WKDR) [1] metric, which is an average disagreement rate, to evaluate all the methods' performance.
where x is the estimated depth map. We set the tolerance level δ = 0.02 which is the same with [1]. l ij is the annotation of the depth order of ij-th pair. x i , x j are the depth values of two points in a pair.  Baseline settings To demonstrate the effectiveness of the multi-scale features and the deepening technique, we present three baseline methods: • Baseline A: The first baseline is a single scale model without deepening. It is a simplified version of the proposed basic model in the Section III-B, which removes the scale 2, scale 3 streams. The method is expected to show the fundamental performance without multi-scale architecture and any deepening method. • Baseline B: The second baseline is the single scale model with deepening (DenseNet block added in the scale 1 stream). By comparing with Baseline A, we will see the difference of the single scale model before and after deepened. • Baseline C: The last baseline is the proposed basic model in Section III-B, i.e. multi-scale without deepening. By comparing with Baseline B, we can explore as to whether deepening or using a multi-scale architecture provides the bigger improvement in performance. We carry out the baseline comparisons compared against the proposed model on all the datasets.

A. NYU Depth Dataset
The NYU depth dataset [29] is a large depth benchmark for indoor scenes, which is collected by a Microsoft Kinect sensor. It consists of 464 indoor scenes and more than 400k images with the resolution of 480 × 640. We use its densely labeled dataset, which has 1449 pairs of aligned RGB and depth images. We use its official train/test split, that is 795 images for training, 654 images for testing. We sample 1600 pairs per image for training and 800 pairs per image for testing. That is totally 127.2 million pairs of points for training and 52.3 million pairs for testing. We consider the number of the training data enough thus we do not apply any data augmentation.
Baseline comparisons The results are reported in Table IV. As we can see, applying the multi-scale features without deepening provides larger performance improvement than deepening on a single stream. Meanwhile deepening the streams of the model is very useful to boost the performance. Among the methods, our model performs the best. Comparisons of deepening methods The second experiment is the comparison test of applying two deepening technique according to Section III-C. The basic model is the proposed multi-scale model without deepening, which we used as Baseline C. As shown in Table V, the methods both help improve the performance. Yet DenseNet outperforms than ResNet. In the training process of ResNet, we observe the overfitting phenomenon. We hypothesize the issue is caused by the duplicated features and the excessive filter numbers of the ResNet block. The three scale features are duplicated due to the overlapping area of the contexts (they overlap at the scale 1 area). In the case, the model needs the regularization to avoid the risk of overfitting the duplicated feature space. However, the 'bottleneck' structure of the ResNet has 64, 64, 256 filters in a block, whose total filter number is six times more than a DenseNet block that we use, which means that the ResNet has a greater chance to overfit the data than the DenseNet. Thus we use the DenseNet model, and the experimental result demonstrates that the method works well for the proposed model. State-of-the-art comparisons The state-of-the-art methods for comparison on the dataset are Eigen et al. [38], Zoran et al. [1] and Chen et al. [28]. Note that, all the methods train and test their models on the densely labeled data, yet Chen and Eigen additionally use the raw dataset which contains more than 290k sparsely labeled depth images to achieve the better performance.
Since the numbers of training images in the densely labeled data and the raw NYU dataset are vastly different, the performances of the models trained on each dataset are reported separately in Table VI. We first focus on the upper part of the table which shows the results of the models trained on the densely labeled data. An interesting observation from the table is that our one scale model (baseline B) already significantly outperforms than Zoran et al.'s result and is slightly better than Chen et al. [28]. Note that Chen et al. apply more than 12k points pairs per image for training, while we only need 1.6k pairs. Our multi-scale model achieves the state-of-the-art performance on the densely labeled dataset.
The bottom part of the table shows the results of the models trained on the raw NYU dataset. Although the number of training data is very different, the performance of the proposed model is still comparable to Eigen et al. [38]. The improvement of Chen et al.'s model from two datasets demonstrates the large training data can greatly boost the performance of the DCNN.
However, for the proposed model, training on the raw NYU dataset can be very time-consuming and also needs massive hard disk space. Because the time of generating the contexts of the points pairs and converting these small image patches to the data format that CAFFE can fast read increases with the training data. We believe the results of training on the densely labeled data have already demonstrated the superiority of the proposed model.

B. KITTI dataset
The KITTI dataset [30] is a large and comprehensive dataset for benchmarking the autonomous driving techniques. It contains a number of outdoor scenes for depth estimation, which has five broad categories: 'City,' 'Residential,' 'Road,' 'Campus' and 'Person'.
We choose 'City' raw dataset for evaluation which includes 27 scenes. We use the train-test split index of the scenes provided by Eigen [38] in the 'City' category, i.e., 18 scenes for training and 9 scenes for testing.
The dataset has no direct annotations for the task. Therefore we use the same method as we did in the NYU depth dataset to generate the annotations. Since the data of each scene is a video sequence, the content of each frame in the same scene has a lot of redundancy. Thus we sparsely sample 200 points pairs per image, which is much less than the number in the NYU dataset to avoid the duplicate pairs. We gather in total 815k pairs for training and 450k pairs for testing. Note that, the ground-truth depths of the KITTI dataset are scattered at irregularly spaced points, which only consists of ∼ 5% pixels of each image, we extract the ground-truth depth closest to each superpixel centroid as the superpixel depth and floodfill the superpixels with the relative depth values.
Baseline comparisons The baseline comparisons on the KITTI dataset are reported in Table VII. We observe two interesting phenomena.
Firstly, the performances of predicting the equal case and the unequal case are very different. We argue that it is caused by learning the data of unbalanced distribution. The structure of the road scene is that a road is always in the center and the buildings on the two sides are along the road, which makes the KITTI dataset has a distinct pattern that the ground truth depth increases with the road's direction. Owing to the pattern, the unequal case accounts for the most of the points pairs, roughly 80% ∼ 90% of the total number of each image. Thus the unequal case is learned very well, yet the equal case is not.
Secondly, with the multi-scale feature and the DenseNet block added, the performance of predicting the equal case show a much more significant improvement than the unequal case.  State-of-the-art comparisons We compare the performance with the state-of-the-art method Eigen et al. [38]. The proposed method achieves the state-of-the-art performance and the recovered examples in Fig. 9 show that the overall structures and the crisp edges at depth discontinuities are captured.

C. Depth In the Wild dataset
The Depth in the Wild (DIW) dataset [28] is a recently released dataset for relative depth estimation. The points pairs are human-annotated. Thus we do not need to manually pick the points and generate the labels. The resolutions of the images are not fixed, roughly at 500 × 400. The dataset uses more than 421k images for training and 74k images for testing. For each image. It only annotates one pair of the points. Thus we have 421k pairs of the points for training and 74k pairs for testing.
Different from the previous datasets, the DIW dataset has two distinct characteristics: 1) The selected points in each image has a relative longer distance between each other, compared with the points pairs generated in the NYU and KITTI datasets, and a lot of points pairs locates at the same row or column; 2) More importantly, the dataset only considers two cases, 'closer' and 'further', where the 'equal' case is removed. Thus for the dataset, WKDR metric is equal to WKDR = . For the first characteristic, we change the three-scale bounding boxes generating strategy: for the points pairs locating in the same row or column, we extract the three scale bounding boxes, whose height or width are 20 pixels, 40 pixels, and 60 pixels, centered on the line between the points; for the other points pairs, we use the same strategy reported in Fig. 6.
Baseline comparisons The baseline comparisons on the DIW dataset are reported in Table IX. The proposed model outperforms the other baseline methods.
State-of-the-art comparisons We compare the proposed model with Chen et al. [28]. We follow their method to show the performance in two ways: 1) training the model from scratch; 2) pretrain on the NYU depth dataset then finetune on the DIW dataset.
The results are reported in Table X. The results show that each model has its own merits. Our model significantly outperforms Chen et al., when both are trained from scratch. While their model has made a considerable improvement when pretrain and finetune from the NYU depth raw dataset and achieved a much better result than ours. Their pretrain model  is based on the NYU raw dataset which has a much lower error rate than our pretrained model (see Table VI) and gains powerful visual representation. The benefits help improve their result.

V. CONCLUSION
Predicting the depth order of the points pairs is a challenging task. By effectively exploring the contexts surrounding the points and deepening the network, it can be performed very well. The proposed framework accomplishes this through the use of the multi-scale local contexts and the DenseNet technique. We achieve state-of-the-art on the task for several datasets with the advantage of using much fewer training data. Future work will extend to solve other mid-level vision issues such as intrinsic image decomposition and improve the framework to exploit more data.


extfieldsofstudy:
Computer Science
Biology

text:
Building Brain Invaders: EEG data of an experimental validation

We describe the experimental procedures for a dataset that we have made publicly available at https://doi.org/10.5281/zenodo.2649006 in mat and csv formats. This dataset contains electroencephalographic (EEG) recordings of 25 subjects testing the Brain Invaders (Congedo, 2011), a visual P300 Brain-Computer Interface inspired by the famous vintage video game Space Invaders (Taito, Tokyo, Japan). The visual P300 is an event-related potential elicited by a visual stimulation, peaking 240-600 ms after stimulus onset. EEG data were recorded by 16 electrodes in an experiment that took place in the GIPSA-lab, Grenoble, France, in 2012 (Van Veen, 2013 and Congedo, 2013). Python code for manipulating the data is available at https://github.com/plcrodrigues/py.BI.EEG.2012-GIPSA. The ID of this dataset is BI.EEG.2012-GIPSA.

Introduction
The visual P300 is an event-related potential (ERP) elicited by a visual stimulation, peaking 240-600 ms after stimulus onset. The experiment was designed to validate the game design of Brain Invaders (1), a visual P300 Brain-Computer Interface inspired by the famous vintage video game Space Invaders (Taito, Tokyo, Japan). The Brain Invaders is based on a P300-based brain-computer interface (BCI) working on a PC. In (1,2) these data were classified using the xDAWN spatial filter (4). This experiment features a training-test mode of operation and both a longitudinal and transversal design. An example of applications of this dataset can be seen in https://github.com/plcrodrigues/py.BI.EEG.2013-GIPSA. Other datasets of Brain Invaders experiment are presented in (5,6). For example, reference (5) features both a training-test (classical) mode of operation and a calibration-less mode of operation, as described in (7)(8)(9). This is the first experiment ever carried out using the Brain Invaders. The complete list of experiments is available at https://sites.google.com/site/marcocongedo/science/eeg-data.
Participants 26 subjects participated in the experiment (7 females), with mean (sd) age 24.4 (2.76). The youngest subject was 21 and the oldest 31. One subject was excluded from the study due to material issues during the experiment. Half of them played games occasionally, that is, around 4.5 hours a week. All subjects were volunteers recruited by means of flyers and of the mailing list of the University of Grenoble-Alpes. All participants provided written informed consent confirming the notification of the experimental process, the data management procedures and the right to withdraw from the experiment at any moment. use an electrode as reference, rather, a, hardware common average reference is used. The amplifier was linked by USB connection to the PC where the data were acquired by means of the software OpenVibe (10,11). Data were acquired at a sampling frequency of 128 samples per second. For ensuing analysis, the application tagged the EEG using software tagging. The tags were sent by the application to the OpenVibe plateform thanks to the Boost inter-process messaging (12). Note that the tagging process introduces a jitter and a latency which artificially modify the ERPs onset. These belong to the hardware and software components of the experiment. In particular, a disadvantage of software tagging is a strong drift over time, resulting in higher jitter (2,13). As a consequence, it is only possible to compare the ERP acquired within the same experimental conditions when the latency is not corrected (14).

Procedures
The interface of Brain Invaders is composed of 36 aliens (symbols) that flashes on the computer screen in 12 groups of six aliens. In the Brain Invaders P300 paradigm, a repetition is composed of 12 flashes (i.e., one for each group), of which two include the Target symbol (Target flashes) and 10 do not (non-Target flash) - Figure 2. At the end of each repetition each alien has flashed exactly twice. The number of Target and non-Target flashed are therefore in a ratio one-to-five.
The total number of Target and non-Target flashes is variable across sessions since the number of repetitions needed to destroy the Target in the Brain Invaders BCI video game depends on the user's performance (7,8). Please see (1,13) for a full description of this paradigm. In any case, since the classes are unbalanced, an appropriate score must be used for quantifying the performance of classification methods, such as the Balanced Accuracy (BA): where A and B (resp. C and D) stands for the number of correctly and non-correctly classified flashes of non-Target (resp. Target) group. For all subjects, the experiment took place in a small room of four-m² surface, containing the PC screen and all the required hardware materials for acquiring the EEG data. The subject was sitting at a distance of 75 to 115 cm from the screen. The EEG headset was placed on all subjects, and the integrity of the whole recording pipeline was checked by performing preliminary tests. The experimenter controlled the session from an adjacent room equipped with a one-way glass window.
Before experiment onset, the subjects were instructed to limit eye blinks, head movements and face muscular contractions, which disrupt the EEG signal. In order to help the subjects maintaining concentration, the subjects were asked to silently count the number of Target flashes.
Each subject participated in a Training and Online session. In the training session the Target alien was chosen randomly at each repetition through the use of a predefined randomised list. The player played all levels for a minimum of three and a half minutes. To be more precise, rather than loading the next level when the Target had been hit, the level was repeated until the minimum time had passed; after three-and-an-half-minutes had elapsed, the level would no longer be reload once the Target was hit. We choose this timeframe because it results in a minimum of four rounds in the worst-case scenario. In order to counter-balance learning and difficulty effects, half of the players played the levels in reverse order (so they played the level estimated most difficult first and the one deemed easier at the end of the experiment, instead of vice versa). We supply an online and open-source example working with Python (15) and using the analysis framework MNE (16,17) and MOABB (18,19), a comprehensive benchmark framework for testing popular BCI classification algorithms. This example shows how to download the data and classify 1s non-Target and Target epochs of signals. This database has been used in the development of Brain Invaders 2 (20), a complete state-of-the-art P300-based BCI chain for single and multi-users.


extfieldsofstudy:
Physics
Computer Science

text:
Framework of Software Testing Based on Cloud Computing

For the problem that efficiency is low and cost is high existing in the traditional software testing method, this paper tested software using cloud testing technology. It introduced related technologies including cloud testing, and described the design of overall architecture for the system in details, designed and implemented the scheduling module using a high priority first scheduling based on dynamic priority. The MATLAB simulation results are that the total turnaround time of the scheduling algorithm in this paper is short when the same tasks are scheduled and increase slowly when the number of tasks is increased than traditional FCFS. This shows that this scheduling algorithm could reduce test cost and realize the automation of software testing under the condition of significantly improving test efficiency and resource utilization.

Introduction
Software testing is the process of executing programs for finding bugs. As an important means to ensure the quality of software, software testing has been paid more and more attention in the software engineering. However, in the traditional single point and localized testing process, the high purchase of test tools, the construction of complex testing environment and other issues limited the development of testing technology seriously. Cloud computing is a new type of distributed computing model, which utilizes virtualization technology to manage the infrastructure resources in the cloud through the network, and form the user-oriented business service model. James A. Whittaker looked forward to "the future of software" and insisted the software testing service based on cloud computing (Whittaker, 2010). To date, the research on cloud testing focuses on the following three aspects. The first is the cloud testing, involving the internal structure of cloud computing, resource allocation, function expansion and many other aspects of testing. The second is testing migration, which means traditional testing methods, managements, processes and frameworks are migrated to the cloud (Parveen, 2010). The last is using cloud to test other software systems. This paper mainly introduced the third method, which is to provide services to users based on the cloud platform and could solve problems existing in the traditional testing methods.
The research of software testing system based on cloud computing is rapidly developing and becoming a hot spot (Tilley, 2012). It has been applied in the fields of banking application software, railway system, mobile service system, which lead to good achievements. Through the research of scheduling strategy, the software testing could have better parallelism and automation, so as to improve the service quality. This paper presented a cloud test solution for software testing, designed an overall structure of testing system and analyzed and implemented the subsystem.

The step of cloud testing
In the process of cloud testing, the user first puts forward the testing request to systems, and then the system accepts the testing tasks. Furthermore, the middleware service and virtual resources are deployed so as to realize the testing execution and supervision. Finally, the testing results are collected. The step of cloud testing is shown in Figure 1.

The design of system architecture
The idea of cloud computing could be tracked back towards the time-sharing mechanism of computers. During that period, individuals carried out calculation and programming through the terminal and PC, and could control their computing environment and select a computing software and customized system in accordance with their requirements. After entering the cloud era, data and programs are disappearing from the desktop, and the whole user interface might depend on a window based on the web browser. There are 3 types of cloud computing, as shown in Figure 2.
(1) Software as a service (SaaS): Users hire specific applications and databases to meet their own needs. At this point, Users can not control the operating system, hardware and frame facilities in the cloud. (2) Platform as a service (PaaS): Expect for servers, users also hire the system software in the cloud. At this point, Users cannot control the operating systems and frameworks in the cloud.
(3) Infrastructure as a service (IaaS): Users can hire as many servers as possible in accordance with the actual requirements through the use of servers. In this model, users can change, control and adjust the cloud operating system. Figure 2 The system architecture

Design and Implementation of Scheduling Sub Module
In the cloud testing, the deployment strategy of parallel testing tasks in testing management module is quite important, due to it involves the deployment and scheduling of virtual devices and the dynamic deployment of testing tasks. This paper designed and implemented the scheduling sub module of the cloud testing system.

Structure of scheduling sub module
After the user submits the testing tasks, the number of virtual machines should be predicted indispensably in accordance with the number of tasks and the analysis of test items, which could reduce the requests due to excessive waste of virtual machines. In addition, it can avoid the wast of testing time due to the insufficiency of testing machines.
. Figure 3 The structure of scheduling sub module

Procedure of algorithm
The testing tasks suitable for cloud testing are independent of each other. In the testing task scheduling process of cloud testing platform, there is no need to consider the dependencies among tasks, which simplifies the algorithm used in the scheduling process.
The scheduling system uses a high priority scheduling algorithm based on dynamic priority, that is, the priority of the testing tasks would alternate with the execution of the tasks or the waiting time of the tasks, and then the priority tasks will be executed first.
(1) In the scheduling process, the number of tasks and virtual machines are assumed to be limited. Assuming the number of independent tasks is n, it could be represented by .
It is now agreed that any task can be executed on any virtual machine, but interrupt processing is not allowed before the completion of relevant tasks. Furthermore, no task can be split into smaller sub tasks. In each scheduling process after the first scheduling, the number of tasks waiting to be executed is assigned to n, and the number of idle virtual machines is assigned to m. Two queues are defined as follow: WaitList stands for testing tasks waiting queue, and FreeVMList represents the idle virtual machine queue.
(2) The user submits the testing tasks through the system interface, and gives the weighted value to each task in accordance with the entrance of WaitList.
When the user submits the testing tasks, the weight of each task in determined. According to the waiting time, operation time and weight value, the priority calculation formula is as follows. (1) Here, j T stands for task j； j M stands for virtual machine j; j q stands for the waiting time; j d stands for the operation time; j w stands for the weight value; j p stands for the priority;  stands for control parameter.
(3) According to the analysis of the number of testing tasks submitted by the users, the number and configuration of virtual machines are predicted. In addition, the operating systems testing systems are deployed automatically according to the different testing environments, which could reduce requests for new resources during the testing execution. However, this kind of prediction based on experience and templates has certain error.
(4) The number of tasks in the WaitList queue is assigned to n, and the number of virtual machines in the FreeVMList queue is assigned to m.
(5) When n>m, the redundant processor is released and the virtual machine j M is assigned to the task j T , and then the corresponding processor is released immediately after the testing task has been executed. (7) A series of new tasks and an error in the execution process are added into the WaitList queue,and then the idle virtual machines are added to the WaitVMList queue as well. Finally step (4) is not repeated until all of the tasks are executed correctly.

Verification
This paper realized the algorithm in above description and the classic FCFS algorithm using MATLAB. Through the simulation and verification, the efficiency of both of two algorithm are analyzed from a mathematical perspective. The assumption of virtual number is 250, the number of testing tasks n respectively are 50,150,250,350 and 450, whose testing times are different. From the verification results as shown in Figure 4, we could make the following conclusions. At first, when the number of virtual machines is larger than the number of tasks, the execution time of the algorithm described in this paper is a constant, and the time of FSFC algorithm in increased due to the increase of the number of tasks. Secondly, when the number of virtual machines is smaller than the number of tasks, the execution time of algorithm in this paper is increased slowly, while the FCFS algorithm is increased fast. At last, the execution efficiency of the algorithm in this paper is nearly 20% higher than that of FCFS algorithm. Figure 4 The efficiency of two algorithm

Conclusion
Combined with cloud computing technology, this paper introduced the structure and framework of cloud testing. Besides, a high priority scheduling algorithm is utilized based on dynamic priority is conducted to realize the scheduling subsystem of the whole system. The proposed scheduling algorithm could significantly improve the testing efficiency and reduce the testing purchase compared with the classical FCFS scheduling algorithm. At last, the implementation of other modules and improvement of system security would be the focus of the next research.


Cappuccino: Efficient Inference Software Synthesis for Mobile System-on-Chips

Convolutional Neural Networks (CNNs) exhibit remarkable performance in various machine learning tasks. As sensor-equipped Internet of Things (IoT) devices permeate into every aspect of modern life, the ability to execute CNN inference, a computationally intensive application, on resource constrained devices has become increasingly important. In this context, we present Cappuccino, a framework for synthesis of efficient inference software targeting mobile System-on-Chips (SoCs). We propose techniques for efficient parallelization of CNN inference targeting mobile SoCs, and explore the underlying tradeoffs. Experiments with different CNNs on three mobile devices demonstrate the effectiveness of our approach.

I. INTRODUCTION
C ONVOLUTIONAL Neural Networks (CNNs) have proven to be one of the most effective approaches to feature extraction [1], [2]. While frameworks such as Caffe [3] or Torch [4] are commonly used for training CNN models, inference using trained CNNs on resource-constrained platforms remains a challenge. We hypothesize that platforms based on mobile system-on-chips (SoCs) will be a major player in the emerging IoT landscape due to their rich feature set and market forces, and thus, we contend that efficient CNN inference on such platforms is increasingly essential.
Forward evaluation of a trained CNN, also known as inference, is computationally intensive. The research community has put forth a number of solutions for accelerating CNN inference on different platforms, including the design of a customized ASIC chip ( [5], [6]), FPGA-based accelerator design ( [7], [8], [9]), and parallelization on server-grade graphics processing units (GPUs). Latifi et al. offered a library for parallel execution of CNNs on mobile devices [10].
We present Cappuccino, a tool for automatic synthesis of efficient CNN inference software targeting mobile SoCs. In addition to the software synthesis capability, Cappuccino features a novel approach to zero-overhead utilization of vector instructions. Furthermore, it considers the effect of inexact computing on classification accuracy, and leverages imprecise arithmetic to further optimize the computation.

II. CONVOLUTIONAL NEURAL NETWORKS
Modern Convolutional Neural Networks (CNNs) have millions of parameters, whose values are obtained during training. Each CNN has multiple convolutional layers, which use 3D filter banks for feature extraction. The convolution result of kernels of a filter bank with Input Feature Maps (IFMs) is accumulated to create Output Feature Maps (OFMs). The number of IFMs, the number of OFMs, and the output size are N , M , and W out × Hout, respectively. The convolution operation is visualized in Figure 1. Thus, a CNN layer has M × N kernels (M filter banks with N kernels each). Kernels have dimension of K × K. Each pixel in an OFM is the sum  of convolutions between kernels and the corresponding pixels in IFMs. To generate adjacent pixels in an OFM, the kernel bank is slid across IFMs by a stride of S. A simplified pseudocode for a convolution operation is shown in Figure 2. The vast majority of CNN inference execution time is spent in convolutional layers [11], and thus, we restrict our discussion to them.

III. CAPPUCCINO
In order to use a CNN for inference on mobile devices, one has to evaluate its forward path with known parameter values. Cappuccino serves this very purpose in that, it synthesizes an optimized SoC-based inference software for a given CNN description. Our current embodiment of Cappuccino synthesizes the CNN in form of an optimized RenderScript program, which exploits the available processing resources on a mobile SoC to execute the computation. Depending on the target SoC, the generated program will be typically launched on multiple CPU cores, the mobile GPU, and the mobile DSP.
As Figure 3 illustrates, Cappuccino requires three inputs. The first is a network description file that contains the CNN architectural information such as number, size, and type of its layers. The second input is a model file, which contains the weight and bias parameter values. Cappuccino reorders CNN parameters to improve the performance of vectorized operations. Parameter reordering does not change the model size, and occurs during compile-time. The third input is the validation dataset that was originally used during training of the CNN. Using this dataset, Cappuccino analyzes the impact of optimizations, such as inexact computing on the given CNN, to determine the suitability of utilizing imprecise arithmetic arXiv:1707.02647v1 [cs.DC] 9 Jul 2017  for the given CNN. The result of this analysis guides the corresponding decision during software generation.

A. Thread Workload Allocation
Convolutional layers contain three main sources of parallelism: Kernel-Level Parallelism (KLP), Filter bank-Level Parallelism (FLP), and Output-Level Parallelism (OLP). In accelerating a CNN, one or more of these types of parallelism should be used for workload allocation to threads.

1) Kernel-Level Parallelism (KLP):
In KLP, parallelization is obtained by executing the computations for convolving a kernel with corresponding IFM pixels in parallel. Hence, each thread computes one multiplication, and the final result is generated via accumulation by an eventual reduction operation.
2) Filter bank-Level Parallelism (FLP): In FLP, filter banks exploit parallelism by allocating kernel computation to separate threads. In this case, each thread computes the convolution of an entire kernel. Subsequently, a reduction addition yields the final result.
3) Output-Level Parallelism (OLP): In OLP, computation of output pixels are carried out in parallel. That is, each software thread computes the 3D convolution of an entire filter bank of N kernels and corresponding pixels in IFMs.
An advantage of OLP is that a kernel loaded by one thread can be reused by other threads that are responsible for generating another pixel in the same OFM. In SoCs with efficient cache systems, it is possible to load each kernel once and use it W out×Hout times. In contrast, data loaded in KLP and FLP cannot be reused as efficiently by the same or other threads. Moreover, in KLP and FLP the required reduction incurs additional overhead for thread synchronization and inter-thread data transfer. As such, Cappuccino uses OLP as its primary workload allocation policy at the thread level. Furthermore, it utilizes vector processing to exploit KLP and FLP within each thread.
The output of each convolutional layer is a 3D data structure which includes α = M ×W out×Hout elements (also referred to as pixels). These elements can be uniquely identified using three variables: the OFM (m), the column (w), and the row (h) number. Each element is the result of a convolution between Fig. 5. Efficient vector processing enabled by data reordering from either row or column major to map major. Groups of u Elements, in either gray or blue, in the same spatial location from consecutive feature maps form a vector (assuming u = 4).
the corresponding window of an IFM and a specific filter bank ( Figure 4). Each thread is identified using a unique index x, where x ∈ [0, α). The identifier index is used to compute values of w, h, and m.

B. Data Reordering for Vector Processing
Cappuccino uses vector processing to further optimize intrathread workload execution. Before executing a vector instruction, it is necessary to load all of the operands. In most SoCs with vector processing support, the memory bus is wide enough to load multiple words of a contiguous block in one memory access. To utilize this feature, known as memory access locality, model parameter values have to be shuffled around. Conventionally, IFMs and kernel parameters are stored in either row-or column-major order. Therefore, data elements stored in the adjacent memory addresses are either the next element from the same row/column or the first element of the next row/column. If we represent an element's address using (Layer, Row, Column) format, the data stored in a row major format reads: (0, 0, 0), (0, 0, 1), · · · , (0, 1, 0), (0, 1, 1), · · · , (0, 2, 0), (0, 2, 1), · · · (1) In a u-way vector processor, one wants to load at least u operands with a single memory access. Cappucino reorders the model data to achieve this goal. In particular, we propose to store the model data in a map major order, as opposed to row or column major, so that a thread can apply vector instructions to corresponding elements of different maps. Absent of this optimization, vector processing would incur significant overhead at the boundaries of a kernel. For example, assuming u = 4, we reorder the model data in the following order (2): A 3D representation of this transform is shown in Figure  5. When model data is reordered, Cappuccino reads IFMs as super-words (vectors), performs vectorized convolution, and accumulates the result. The optimized computation is shown in the algorithm of Figure 6.  1) Zero-Overhead Dynamic Reordering of OFMs: Note that model data can be reordered and written to a new model file without any overhead as it happens statically at compiletime. However, reordering the input to an intermediate CNN layer is not as straight forward. In CNNs, the output of a layer becomes the input to the next layer. It follows that the output of a layer has to be reordered to allow the use of vectorized operations in computing the next CNN layer. This process has to happen dynamically, and thus, is expected to incur time and energy overhead.
Cappuccino avoids the dynamic data reordering overhead by directly storing elements of the OFMs in map major order as they are computed. Parameters w, h, and m are used to determine the location of the output element that thread x generates. To store OFMs in map major format, one has to swap the priorities associated with these parameters. For example, the result of computations by the second thread (x = 1) is by default stored in the second location of the output memory. After reordering, however, the second element of the output memory must contain (m = 1, h = 0, w = 0). Such an output can be directly used as the input to the next layer without any overhead. Figure 7 illustrates the idea.
To create the output in the reordered map major format, we generate indexes for stacks of u layers, instead of a single layer (Figure 7). That is, we start indexing the second row only after all first rows of all u layers are indexed. Equations (3) and (4) map a thread id to w and h, respectively. For computing the value of m (map index), it is required to see which stack and layer a particular output belongs to ( Figures 5 and Figure 7). Equation (5) computes the value of m.

C. Inexact Computing
Modern mobile SoCs tend to support a number of predefined imprecise computing modes that are likely to results in faster or more energy efficient execution [12]. On such platforms, the target processing mode has to strike a balance between the implementation metrics, e.g., runtime or energy dissipation, and the inference classification accuracy.
For example, RenderScript offers two imprecise computing modes for applications that do not need a strict implementation of the IEEE 754 standard, called relaxed and imprecise computing modes. In both modes, the implementation of floating point arithmetic is not fully compliant with the IEEE standard 754 for handling denormalized numbers. The imprecise computing mode is more efficient, but has a lower arithmetic accuracy. In this mode, operations resulting in -0.0 can return +0.0, and operations on INF and NAN are unsupported. Perhaps more importantly, vector processing is only available under imprecise computing modes, in current version of RenderScript. Vector processing under the RenderScript precise computing mode would result in sequential processing of vector elements.
Cappuccino analyzes the given CNN layer by layer to determine the best matching computing mode for every layer of the CNN. In every layer, it utilizes the validation dataset to measure the classification accuracy under different processing modes. Subsequently, Cappuccino determines which layers of a CNN can be processed using inexact arithmetic and which ones demand a precise implementation. The goal is to execute as many CNN layers as possible in inexact modes, under user specified constraints in terms of acceptable degradation in classification accuracy.
In our discussions, the term accuracy arises in reference to either arithmetic accuracy or classification accuracy. The former measures the numerical difference between values computed in exact vs. inexact arithmetic. The latter indicates the inference classification performance of CNNs, for example by measuring the percentage of true positive predictions.

A. Setup
We used Cappuccino to implement three modern CNNs: AlexNet [1], GoogLeNet [2], and SqueezeNet [13]. Subsequently, the parallelized implementations are evaluated on three different smartphones with different generations of Qualcomm Snapdragon SoCs. In order to increase the precision in measurements, all experiments have been repeated 100 times, the minimum and maximum observations are omitted, and the average of the remaining 98 observations are reported. In all of the experiments, the smartphones were put in airplane mode, their screen brightness were fully dimmed, and their background processes were stopped to the extent possible. B. Runtime and Energy Efficiency 1) Speedup: We executed the synthesized programs on the platforms and measured the execution time. Table I summarizes the results. Programs synthesized by Cappuccino offer a speedup of at least 31.95X (GoogLeNet on Galaxy S7) and at most 272.03X (SqueezeNet on Nexus 5) compared to the baseline implementation of single-threaded Java. Moreover, the execution time in all but one case is below a second.
2) Effect of Inexact Computing: To determine the best inexact computing mode, we use Cappuccino to measure the classification accuracy of the aforementioned CNNs in computing modes supported by target platforms. This analysis  is performed on 5000 random images of ILSVRC 2012 validation dataset [14]. The classification accuracy in imprecise mode turns out to be identical to the exact mode. Hence, Cappuccino recommends utilization of imprecise computing in all layers. Table I demonstrates the effect of imprecise computing on execution time. In our experiments, use of imprecise computing mode offers up to 8X speedup compared to the same implementation under exact arithmetic. Note that RenderScript incarnation of the imprecise computing mode enables vector processing in addition to other optimizations, such as using a rapid exception handling for denormalized numbers.
3) Comparison with Related Work: Table III compares the performance of software synthesized by Cappuccino with the state-of-the-art work [10]. The proposed solution under exact arithmetic improves the execution time by 1.38X. In addition, when the synthesized software is both parallel and imprecise, it shows up to 11.47X speedup compared to CNNDroid [10]. 4) Energy Consumption: Cappuccino invokes many threads, which increases the instantaneous power consumption compared to a sequential program. However, software synthesized by Cappuccino runs drastically faster than a sequential equivalent. This results in reduction of energy consumption. Table II compares the energy consumption for running SqueezeNet on Nexus 5. Reported numbers are computed by running each program 1000 times, and calculating the average. Measurements are performed twice to showcase repeatability (2000 runs total).

VI. CONCLUSION
In this paper we presented Cappuccino, a platform for efficient synthesis of CNN inference software targeting mobile SoCs. Cappuccino leverages RenderScripts via which, it utilizes CPUs, the GPU and the DSP that commonly exist on a mobile SoC to execute a CNN efficiently. Cappuccino performs an assessment on the impact of inexact computing on execution time and classification accuracy. Subsequently, it selects an inexact computing mode that best fits a layer of a CNN. Compared to sequential implementations, programs synthesized by Cappuccino achieve a speedup of at least 31.95X and at most 272.03X, and improve the energy consumption by 7.81X.


extfieldsofstudy:
Computer Science
Medicine

text:
An Algorithm of Image Encryption Using Logistic and Two-Dimensional Chaotic Economic Maps

In the literature, there are many image encryption algorithms that have been constructed based on different chaotic maps. However, those algorithms do well in the cryptographic process, but still, some developments need to be made in order to enhance the security level supported by them. This paper introduces a new cryptographic algorithm that depends on a logistic and two-dimensional chaotic economic map. The robustness of the introduced algorithm is shown by implementing it on several types of images. The implementation of the algorithm and its security are partially analyzed using some statistical analyses such as sensitivity to the key space, pixels correlation, the entropy process, and contrast analysis. The results given in this paper and the comparisons performed have led us to decide that the introduced algorithm is characterized by a large space of key security, sensitivity to the secret key, few coefficients of correlation, a high contrast, and accepted information of entropy. In addition, the results obtained in experiments show that our proposed algorithm resists statistical, differential, brute-force, and noise attacks.

Introduction
Converting secret information such as images from a decipherable form to an undecipherable one has been recently investigated to fulfill the demand of a secure route of image transmission through different communications channels such as Internet and wireless networks. Encryption schemes have been introduced based on chaotic models to handle that issue and to provide such secure routes [1,2]. The Data Encryption Standard (DES) algorithm is a traditional scheme and has been used for encrypting images [3]. It has some drawbacks, among which is that the algorithm's efficiency is low when dealing with large images [4,5]. The theory of chaos is an important theory that possesses some complex features such as sensitivity to the initial conditions, unpredictability, and bifurcation. It has been recently adopted by researchers in the process of encryption [1,2]. Chaos has presented some popular models, which include some complex properties that help provide a robust encryption scheme. Such models include the logistic map, Lorenz attractor, the Hénon map, and some chaotic economic systems [6]. Such complex properties of chaotic maps may be reflected in an analogy to certain properties of the cryptographic process of ideal ciphers like diffusion, confusion, balance, and avalanche.
The cipher schemes proposed based on chaotic characteristics have shown a robust and efficient way to tackle image broadcast very quickly with highly secured routes via different types of telecommunications. Since Matthews [7] published his first algorithm of encryption in 1989, which depended on some chaotic characteristics, a number of chaos-based image encryption schemes with have been developed in literature. For instance, in [8], the authors created a cipher image by dividing it into some blocks and used some kind of permutations applied to each block using the logistic map. A two-dimensional discrete chaotic system has been introduced in [9]. It used the Chebyshev chaotic sequences to scramble each pixel in the novel image. Increasing the key space is another aspect that makes the algorithm used for encryption very difficult to break, and this has been performed in [10]. A multi-chaotic system has been adopted in the process of encryption in [10]. In [11], the Rossler system has been included in an encryption algorithm and has been used to change the image's pixel values and position in order to make the ciphered images have high uncertainty. A logistic chaotic function has been used in [12] with the one-time pads in order to increase the encryption space, and hence, robust cipher images can be obtained. The shuffling approach has been introduced in [13], presenting a good scheme of encryption based on the cat map. It is a one-dimensional chaotic map with some advantages such as simplicity and high efficiency [14]. However, it has some weaknesses that may make it possible for the encryption scheme to be easily broken. This weakness includes a small key space and a weak security [15,16].
It is important here to highlight some of the recent and state-of-the-art algorithms in cryptography that will be used in comparison with our proposed algorithm. For example, in [17], Sivakumar et al. introduced a new image encryption algorithm based on pixel position permutation and a random key stream. In [18], the authors combined DNA sequences with a chaotic map to introduce an image algorithm of encryption. Based on discrete wavelet transform and multi-chaos, an image algorithm was proposed in [19]. The Arnold transformation has been used to encrypt images with high security in [20]. The idea in [20] depended on splitting the image randomly into several parts, and then, each part was coded by the Arnold transformation to increase the security level. Adopting the classical confusion-diffusion structure, an image encryption algorithm has been introduced by Hua et al. based on a 2D logistic-sine-coupling map [21]. In [22], a simple piecewise linear chaotic map has been utilized to construct a bit-level image encryption algorithm. Recently, a robust image encryption/decryption algorithm via 1D chaotic economic map was proposed in [1]. The advantage of that algorithm is that the periodic recovery of the plain image is removable.
Some of the disadvantages existing in those algorithms have led us to propose a new 2D chaotic algorithm for image encryption. The algorithm published in [1] is constructed based on a one-dimensional chaotic map, but this one depends on a two-dimensional chaotic system. The number of parameters in the current algorithm is more than that in [1], and therefore, the security level is higher than the one in [1]. The current paper is organized as follows. In Section 2, a brief introduction about the 2D chaotic map used in the algorithm is presented. In Section 3, the algorithm's steps are introduced in detail. In Section 4, we discuss our obtained results due to applying the algorithm on some popular images and make some comparisons with the results obtained by existing algorithms in the literature. In Section 5, we give some conclusions.

The Two-Dimensional Map
Recently, the adoption of chaotic dynamical systems to encrypt images has drawn researchers' attention. Such systems possess many parameters that have influences on the systems' behaviors and are suitable in the encryption scheme. Those parameters have some important usage, as they are used as security keys in the process of encryption. Small changes in the value of the system's parameters may make the system enter a region where chaos arises. The chaotic behavior of such systems is an important aspect when cryptographic algorithms are constructed. In this work, we adopt the chaotic economic map introduced in [23] that takes the following form: where: Q n = x 1,n + x 2,n , n = 0, 1, 2, ...
Equation (1) shows the chaotic behavior of a map with six distinct parameters. These parameters are important in the economy. First, a stands for the market size and should always be positive, while the parameter b > 0 represents the change of price within the market. Economically, the condition a > b and a > c should be always satisfied. The parameter c ≥ 0 is called the cost of producing one unit of a good, and k > 0 is known as the speed adjustment. The chaotic behavior of the economic map (1) is detected at the parameter values: x 1,0 = 0.11, x 2,0 = 0.12, a = 2, b = 0.5, and c = 0.5. Figure 1a shows the bifurcation diagram of Equation (1) with respect to the parameter k for these parameters. Numerical simulation has shown that when the system (1) approaches the value k = 0.6490, a two-cycle period appears, and so on, for other different cycle periods until the system becomes chaotic. This is clear in the Lyapunov exponents plotted in Figure 1c at the same values of the parameters. Lyapunov exponents tell us the rate of divergence of nearby trajectories, a key component of chaotic dynamics. Figure 1b shows the time series for both variables x 1,n and x 2,n . Finally, Figure 1d shows the complex chaotic behavior of the system (1).

The Algorithm
It is known that any original image has strong correlations among its pixels. In the encryption process, it is important to scatter the pixels to break their correlations. The well-known approach to do that is the shuffling process, which is applied to each array of the original image. Suppose that the original image has the dimension M × N and P = (p i,j ) 1≤i≤M,1≤j≤N is the array of its pixel positions. To produce the shuffling array, Sh rc P , we continue as follows: 1.
(Shuffling of rows) In this step, we recall the logistic map x n+1 = rx n (1 − x n ). The logistic map is used to generate a set of random values in the interval [ The rows of the P array are changed according to those random values, and hence, a new array Sh r P is obtained: (Shuffling of columns) Using a similar technique as in (1), we generate a new set of random values in the interval [1, N], say j 1 , j 2 , ..., j N , j k = j l ∀ k = l. The columns of Sh r P take the following form: Now, the steps of our encryption algorithm are constructed as follows: Step 1: Read the original image, then convert it to a gray image, say P.
Step 2: Perform row and column shuffling to generate the shuffling array, say Sh rc P .
Step 7: Perform the bit-wise XOR between the values of B 1 and I, e 1 = bitxor(B 1 , I), where I equals the value of the array P after reshaping it to be a vector of size 1 × MN.
Step 8: Perform the bit-wise XOR between the values of B 2 and I, e 2 = bitxor(B 2 , I).
Step 9: Perform the bit-wise XOR between the values of e 1 and e 2 , e = bitxor(e 1 , e 2 ).
Step 10: Convert the values of e from binary to decimal, say E.
Step 12: Reshape the set E to be an array of size M × N, say C as the cipher image.
In addition, the steps of our decryption algorithm are described as follows: Step 1: Read the cipher image, C with size M × N.
Step 3: Convert the values of E from decimal to binary, say e.
Step 4: Repeat Step 4 in the encryption algorithm to generate MN values for the two vectors, Step 5: Do the following preprocessing for the generated values in Step 4: x i = f loor mod x i × 10 14 , 256 Step 6: Convert the preprocessing values in Step 5 from decimal to binary, Step 7: Perform the bit-wise XOR between the values of B 1 and e, F 1 = bitxor(B 1 , e).
Step 8: Perform the bit-wise XOR between the values of B 2 and e, F 2 = bitxor(B 2 , e) Step 9: Perform the bit-wise XOR between the values of F 1 and F 2 , z = bitxor(F 1 , F 2 ) Step 10: Convert the values of z from binary to decimal, say Z.
Step 11: Reshape Z to be an array of size M × N, say Sh.
Step 12: Perform row and column shuffling to get the decryption image, say D.

The Secret Key Generation
Assume that P = (p i,j ), i = 1, 2, ..., M, j = 1, 2, ..., N, is the plain image, where p i,j refers to the value of the pixel at the position (i, j) and (M, N) is the size of the plain image P. The secret key will be calculated via the key mixing proportion factor as follows: Then, we change the initial condition β 0 according to the following formula: where β 0 = x 10 , r 10 , x 20 , r 20 , x 1,0 , x 2,0 , a, b, c, k and K = K u , u = 1, 2, · · · , 10, respectively. After that, take two initial values for the logistic map, x 10 , x 20 , two parameters for the logistic map, r 10 , r 20 , two initial values for the system, x 1,0 , x 2,0 , and four system parameters, a, b, c, k.

Remark 1.
It is not difficult to note that 0 < K u < 1. Therefore, the value of β 0 on the right-hand side of Equation (3) must satisfy the conditions of Map (1).

Security Analysis
A good encryption algorithm possesses the following properties: (i) histograms of cipher images that are completely uniform; (ii) good information entropy; (iii) very low relationship between two adjoining pixels; (iv) resistance to statistical and differential attacks; (v) sensitivity to the secret keys; (vi) huge key space; (vii) high contrast; (viii) resisting noise attacks. Those aspects are tested for the proposed algorithm.

Statistical Analysis
Statistically, plain and cipher images must be different and therefore similarity must be hidden in order to avoid the attempts of attackers to break the algorithm.

Analysis of the Histogram
The distribution of pixel values in the image is shown by a histogram analysis. The histogram of the cipher image should be uniformly distributed. Therefore, obtaining any useful statistical information will be very hard. The third and fourth columns of Figure 2 display cipher images and their corresponding histograms.
The histograms of the obtained encrypted images are approximately uniform, are different from those histograms of the plain images, and show that the proposed algorithm is secure enough for image encryption. The cipher images have histograms that have an approximately uniform distribution. The chi-square test [24] is used to check this uniformity using the following formula: where the gray level number is k = 256 and O i and e i represent the observed and the expected occurrence frequencies of each gray level (0-255). Using a level of significance that α = 0.05, the chi-square values of the cipher images are given in Table 1.

Entropy Process of Information
For a source with symbol S, the process of entropy is denoted by H and is calculated by the following formula [25].
where p(s i ) refers to the probability of symbol s i and the bits are used for entropy. The theoretical result of entropy H(S) is eight if N = 256 = 2 8 gray values for an image with equal probability. The information entropy shows the distribution of gray values. The information entropy for the cipher images is exhibited in Table 2. The obtained values are near the theoretical value of eight. Recently, the authors in [26] proposed a statistical test for the block entropy and introduced qualitative and quantitative results. We have selected 100 non-overlapped blocks of size 16 × 16 randomly from each cipher image. We have calculated the information entropy for each block based on Equation (5), and the average entropy was recorded. From Table 2, all actual block entropy values are passed the theoretical block entropy values at α = 0.01 and α = 0.05. Therefore, we may conclude that the cipher images is random-like after applying our proposed encryption algorithm.

Examination of Correlation
In this subsection, the vertical, horizontal, and diagonal correlations between two adjacent pixels in a cipher image are evaluated. This examination can be carried out by selecting randomly 5000 pairs of two neighboring pixels from the original and cipher images. The coefficient of correlation for each pair is obtained using [4,27]: where: and: where the gray values of any two neighboring pixels of an image are denoted by x and y. Figure 3 demonstrates the horizontal distribution of the relationship between two neighboring pixels in the original image and its corresponding ciphered image. Table 3 shows the relationship of two neighboring pixels in the original and cipher images. Figure 3 presents those correlations for the plain and the cipher one. The correlation coefficients of the cipher image for our proposed algorithm and other algorithms are shown in Table 4. The correlation coefficients of two adjacent pixels of the cipher image are close to zero.

Sensitivity Analysis
One of the important features of a good encryption algorithm is the sensitivity to the security key and the plain image. Therefore, any small change in the security key or the plain image tends to a completely different cipher image [28].

Differential Attack
In order to get some insights about the influence of changing one pixel in the cipher image, some measures such as NPCR (Number of Pixels Change Rate) and UACI (Unified Average Changing Intensity) are used. They have the following formulae: where: The width and height of the two ciphered images I 1 and I 2 are denoted by W and H. In Table 5, each gray level is coded in such a way that only eight bits are used in the coding. The obtained calculations demonstrate that the percentage of the NPCR is acceptable, as it is close to 99.6%, while the percentage of the UACI is close to 33.4% is also acceptable for the assigned images. Therefore, we can conclude that our encryption algorithm can survive against differential attack.

Key Sensitivity Test
Some key sensitivity tests are carried out in this subsection in order to check our proposed algorithm. Figure 4a-c shows the decrypted images of the ciphered images of lena, barbara, and house using the proper values of the secret key. In the secret key, if the value of a is changed to 1.99999999999999 and the other values of the secret key are unchanged, the results give the decrypted images shown in Figure 4d-f, which are completely unreadable and different from the original ones. Figure 4g-i shows the decrypted images of the ciphered images with a similar security key except c = 0.49999999999999. Based on those obtained results, one can conclude that any small change in one of the security keys leads to a bad image being produced, and then, the original plain image cannot be retrieved. Therefore, our algorithm possesses strong security keys.

The Analysis of Our Key Space
A strong encryption algorithm has a large enough key space. For high security, the key space should be greater than 2 100 [2,29]. The proposed algorithm uses two initial values for x 0 and two initial values for r 0 . The logistic map is utilized for row shuffling and column shuffling. In addition, there are six initial values, x 1,0 , x 2,0 , a, b, c, and k, which are used in the two-dimensional chaotic economic map. Therefore, our security keys consist of ten values. Therefore, using a precision of 10 −14 [2], the size of the proposed algorithm key space is 10 140 , and it is more than 2 100 . Table 6 shows that our proposed algorithm has a key space larger than the key spaces in [1,6,21,22]. Therefore, the key space of our algorithm is large enough to resist brute-force attacks. This makes our proposed algorithm sufficiently robust to oppose a wide range of brute-force attacks.

Noise Attacks
The security of our algorithm against noise is tested. Two famous types of noises are incorporated into the encrypted images. The first is Gaussian noise with variances of 0.01 and 0.1. The second is salt and pepper with densities of 0.05 and 0.1. The Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR) are used for measuring.
Assume that X and Y denote the plain image and decrypted image for the ciphered image after the noise is incorporated. MSE and PSNR are calculated by the following equations: (10) and: where I is the maximum possible pixel value of an image. The high similarity between the plain image and the decrypted image has been measured using the PSNR. Table 7 and Figure 5 show the results. The results show that the images with salt and pepper noise are better than the images with Gaussian noise because the PSNR is greater than 18.6. In addition, the histograms of the decrypted images and the plain images are approximately similar.

Contrast Analysis
The difference investigation of the image permits the observer to entirely understand the texture of images. The ciphered image has high contrast levels as a result of the high randomness given by the map (1) in the process of encryption. We have evaluated the contrast of the ciphered image and the effectiveness of Map (1) in image encryption applications. This can be carried out by the following formula: where p(i, j) is used to refer to the number of gray levels in the co-occurrence matrices. The contrast analyses of both the plain and cipher images for lena (256 × 256), barbara (256 × 256), and house (256 × 256) are shown in Table 8. Table 9 shows that the contrast analysis of the introduced algorithm overcomes the algorithm in [6]. It has the best result. Table 8. Contrast analysis of plain and cipher images at a = 2, b = 0.5, c = 0.5, x 1,0 = 0.11, x 2,0 = 0.12, and k = 0.87.  Table 9. Contrast analysis of the plain (lena (256 × 256)) and cipher image using the proposed algorithm at a = 2, b = 0.5, c = 0.5, x 1,0 = 0.11, x 2,0 = 0.12, and k = 0.87 and other algorithms.

Gray Value Degree Analysis
Measuring the gray difference of a pixel requires one to know its four neighbors. This can be carried out using the following formula: where I(x, y) refers to the value of a pixel at location (x, y) and I(x −ý) denotes its four neighboring pixels. Now, the average difference of the whole image is measured by the following: where M and N refer to the row and column numbers in an image. Using (11) and (12), the Gray Value Degree (GVD) is defined as, where the average neighborhood gray difference of an original image is denoted byÍ, whileĪ(x, y) refers to those of the encrypted image. Table 10 presents the degree values of the computed gray value degree for distinct images using the proposed algorithm. One can see from Table 10 that our obtained degree values are close to the ideal value, which is one. Table 11 shows some validations of the proposed algorithm versus existing algorithms in the literature.

Peak Signal to Noise Ratio Analysis
PSNR is used to carry out the evaluation of the encryption algorithms. It is applied to both images, the original image and the encrypted one. It considers the original image as a signal, while the encrypted image as a noise. PSNR is calculated by using Equation (11). Table 12 shows the PSNR values of some different images. It is shown that those values are low, which means it is difficult to retrieve the original image from the encrypted one in the case of hacker attacks.

Computational Complexity
We make a comparison of the computational complexity between the traditional DES, AES, and other algorithms in the literature including our proposed algorithm. The results appear in Table 13. Based on Table 13, we can concluded that the time of our algorithm is less than the traditional DES and AES algorithms and the other algorithms in the literature and is completely acceptable for encryption of an image. Table 13. The speed of encryption for each algorithm. DES, Data Encryption Standard.

Conclusions
The current paper introduced a new cryptographic algorithm based on a two-dimensional chaotic economic map and logical operator bit-wise XOR . The size of the key space for our proposed algorithm is 10 140 , which allows our algorithm to survive against attacks. The numerical experiments have shown that the algorithm is very sensitive to the security keys, and hence, any bit change or random guessing about the proper value of the security keys fails to retrieve the original image. The obtained results by the proposed algorithm have shown that the ciphered image is random-like since the information entropy of the cipher image of size 256 × 256 is very close to the theoretical value of eight, the NPCR is close to 99.6%, and UACI is close to 33.4%. Therefore, our algorithm is extremely sensitive to any small change in the original image. According to the obtained results, we may conclude that the proposed algorithm owns a high level of security that allows it to prevent any attempts of attack by hackers.