Improving Performance During Camera Surveillance by Integration of Edge Detection in IoT System

This paper is proposing an IoT-based camera surveillance system. The objective of research is to detect suspicious activities by camera automatically and take decision by comparing current frame to previous frame. Major motivation behind research work is to enhance the performance of IoT-based system by integration of edge detection mechanism. Research is making use of numerous cameras, canny edge detection-based compression module, picture database, picture comparator. Canny edge detection has been used to minimize size of graphical content to enhancing the performance system. Simulation of output of this work is made in MATLAB simulation tool. Moreover, MATLAB has been used to give comparative analysis among IoT-based camera surveillance system and traditional system. Such system requires less space, and it takes less time to inform regarding any suspicious activities. KEywoRDS Camera Surveillance, Canny Edge Detection, Cloud Server, IoT, MATLAB, NET Platform

INTRoDUCTIoN
At present, IOT (Gubbi et al, 2013) is a hot cake for tech-savvy people and its applications are continuously in demand. It became possible because the dependency on Internet is increasing dayby-day.
IoT (Iqbal et al, 2017) saves a lot of time. It happens because with the help of this technology user manages their household appliances without any delay. In addition to the home automation system, the internet of things also works on an industrial automation system. In the initial system, the household devices are managed by the consumer at some distance.
The layout of the internet of things is arranged in Figure 1. In this layout working of the internet of things is explained. IoT system faces a lot of challenges (Tiwari & Maurya, 2017) during implementation. In the second system equipment which is installed in a manufacturing unit is managed from a distance. When the internet of things (Miraz et al, 2015) is used, chances of failure are very less.
It happens because different types of protection devices such as fire alarm and theft alert system are employed. Working of IoT based security system (Alem et al, 2016) is shown below: The compression of data could be performed using an edge detection mechanism . However, there have been several research work that focused on edge detection using Fuzzy logic (Suryakant, 2012).
When the technology of the internet of things is used for Home Automation it provides several benefits which are explained below:

Excellent Functioning of Equipment
The user interference is not a requirement in houses where the internet of things equipment has been used. This equipment provides a very high level of performance. The equipment which provides programming and controlling facilities to its user is called IOT (Gubbi et al, 2013) gadgets.

Consume Less Time
When internet of things devices (Lam et al, 2016) are applied the consumer does not carry out the same task on regular basis. Due to this less time is consumed. Consumer obtained high-level performance because the transmission is done through the device to the device. Due to this, consumers obtained accurate output.

Economization of Money
With the use of the internet of things (Dharampal et al, 2015) devices, energy, and resources are used in a very perfect way. This technology can be adopted in a very efficient way if the devices are kept under observation.

Improve Lifestyle
Our lifestyle has improved due to the use of AI-based applications. The devices which are made based on artificial intelligence increase the level of our satisfaction. It will make our life more convenient. As a result, it is sure that it will improve our lifestyle. Many researchers have focused on the security and privacy feature (Zhou et al, 2018) of IoT. At many locations, NFV enabled IoT Architecture (Miladinovic & Schefer-Wenzl, 2018) has been used. IoT is suffering from challenges and issues from an Indian perspective (Yadav et al, 2018).

Neat Atmosphere
At present, based on sun position and the direction of blowing wind it is possible to manage the opening & closing of windows. At this point, a program is available for this management work. It also reduces air pollution because we can easily observe the discharges of carbon from various sources. With the help of this technology, it is possible to identify forest fires. Based on load requirement a smart device (Yadav et al, 2018) also reduces energy consumption.

Health Monitoring
It is quite easy to recognize health-related problems when the equipment which is made based on internet of things technology has been utilized. This equipment provides great support in the favor of doctors when used in hospitals. In addition to this, this equipment also helps old people and handicapped citizens.
Several IoT applications (Bali et al, 2018) have been studied in present researches. It is observed that the IoT real-time specification (Haikun et al, 2018) is found challenging. This research is making use of IoT with surveillance cameras (Alexandrie et al, 2017) to minimize crime.

EDGE DETECTIoN
In the present world, where the way of living life is very modern, edge detection of the image is highly required. It is mostly required in forensic and the army. Therefore, absolute knowledge of this detection algorithm is compulsory. Edge detection played a great role in image processing. When the intensity value or pixel value of the image changes sharply then this change in pixel value is easily recognized by edge detection. In the present time, its various types are ready for use. Within the algorithms of image processing, a great role is played by edge detection. It has many applications like deformation of the image, identification of the sample, partition of the image, and its removal. Most of the information towards the image is provided by the edge. Therefore a great role is played by edge detection in the method of image processing. It provides the dimension of a picture is supplied by it. It will identify the image in the form of an object in the company of its identified edges. Several unusual edges are provided when this technique is used for an image. When these unusual edges are integrated, it will design the summary of the object. With the help of this technology accurate edges are identified. It is a very important property of this technique. In addition to this, it will also recognize the right direction of the object in the picture. The total amount of storage space required for the accumulation of image edges is less in comparison to the whole image. These image edges include information regarding object dimension and its direction. The definition of an edge is provided from a different point of view. Out of these different points of view, one is "It changes either the brightness or the color of an image ". In normal circumstances, in an image edges are represented in different forms. These forms are: 1.
Step edge 2. Ramp edge 3. Roof edge 4. Line edge Different researchers invented different edge detection algorithm. But based on the used order of derivative they are divided into two types. Out of these two types, one is Gradient-Based and the other is Laplacian Based.
Canny edge detector-In the middle of edge detection algorithms that are available at present, the Canny edge detection algorithm is used on regular basis for the last few years. The algorithm of image processing is considered for the last thirty years and it is used for the accumulation of image information. In the image of a scene, various types of information are present like the size, color, and direction of objects. In the primary step, the object is separated from its surrounding. After that edges are identified to design the summary of the object. Due to this reason, from a computer and image processing point of view, his technique is very valuable. "Canny Edge Detection Algorithm" used two-level for each image. To achieve improved edge mapping, the technology of a Canny edge detector is used in all parts of an image. But, in this process, some false edges are determined in a smooth area and some appropriate edges remain unidentified. This problem is resolved by a blocklevel canny edge detector. It supplied improve efficiency at each block of the image

CANNy EDGE DETECTIoN ALGoRITHM
It works in the form of the operator which is used for (Alem et al, 2016) edge detection. It uses an algorithmic program that has several steps. In images, it will detect a large number of edges. The Discovery of the best possible edge detection algorithm is the only plan of this method. An edge detector is considered in the form of the best detector when it will perform good detection. The genuine edges which are located inside an image are made clear by the algorithm. In situations where marked edges are almost similar to the edge of a real image, it is considered that its location is good. It gives a response in a single attempt because the noise of the image does not create artificial edges. Due to this, a given edge in the image is only visible once, and whenever possible.
To provide these requirements in written form mathematics of modifications is used by this detector. This method detects the assignment by which a certain method becomes optimized. This algorithm is considered best in comparison to other edge detection algorithms. The compulsory standards about canny edge detection are described below: 1. Highest Precision: It is very essential that no edge will remain undetected and there should be no response for non-edge. 2. High-quality tracking: The space detected by the detector in the middle of edge pixels and the actual edge is the least. 3. Exclusive Response: It means that a given edge in the image is only visible once, and whenever possible. The algorithm mainly has five steps in it.
The algorithm works on a different level: 1. Smoothing: It is the primary level in which an image is made blur to eliminate noise. For this purpose, median filtering is applied.

Finding gradients:
The edges should be made clear at points where the progression of the image has vast dimensions. 3. Minimum repression: It means that limited maxima are considered in the form of edges. 4. Double thresholding: Applied for determination of appropriate edges 5. Edge detection by output lag: Ultimate edges are identified when the edges which are not related to a certain edge has been removed completely.

Social Applications of Edge Detection
By considering how this detection technique has been used in Macao society the popularity of this detection technique in public demand is displayed.
• License Plate Detection: In the present world, cars are used everywhere. Therefore it is certain that intelligent traffic control will become a trend in the future. Therefore, we will discuss here how this detection technique is used for the identification of the license plate. The technology of license plate detection is commonly applied at the gates of the toll plaza, in car park areas. It has been found, this technology will provide some useful significance if some modification is done. 1.Sample image gray scaling and QDPA operator edge detection. 2Detection of hidden information in medical images. A comparison is made between phase-based detection methods and usual edge detection methods. From this comparison, it has been found that the basic difference in the middle of these methods is that in addition to edges of the object, some furtive information of test object is also detected which cannot be identified by conventional methods. Conventional methods are not able to identify this information because the color of these details and their immediate area are almost similar.

Advantages of Canny Edge Detector
1. Canny edge detection is very flexible. 2. It can modify itself to recognize the edges with different characteristics. 3. Gives reliable detection. 4. It is best for edge detection because it follows all the standards of edge detection and it is used in a very effective manner.

Disadvantages of Canny Edge Detector
1. If the quality of leveling is essential in the three-dimensional field it will work out slowly. 2. It is unfair in the direction of vertical and horizontal edges and in the case of revolving equilibrium it does not provide a satisfactory estimation.

oBJECTIVE
Objectives considered in this paper have been discussed in this section. The major objective of this work is shown below: 1. Research would review traditional Camera Surveillance Systems and study their limitations. 2. Implementing an IoT based camera surveillance system. The system would provide a security (Iqbal et al) mechanism to detect suspicious activities. 3. Using Canny Edge detection  mechanism to minimize the size of graphical content to improve the performance of the security system. 4. Using Cloud (Biswas et al, 2014) Space to reduce the cost of storage. 5. Proposing a comparative analysis of the proposed system with traditional work.

METHoDoLoGy/TECHNoLoGy
There have been many research methodologies that are needed in the case of such research work.
Various types of researches are basic research, Quantitative researches, theoretical research, Qualitative Researches. This is an Experiment Research where IoT based Camera Surveillance system has been technically proposed. In this proposed system, Numerous Cameras, Canny Edge Detection (Dharampal et al, 2015)) based compression module, Picture Database, Picture comparator is used. The simulation of the result of the proposed module is made in MATLAB. In addition to this, MATLAB has been used to provide Comparative Analysis among IoT based Camera Surveillance system (Alexandrie et al, 2017) and Existing system. To achieve objective implementation of Home automation IoT System is required to using WI-FI (ElShafee et al, 2012). There are several mechanisms to proposed a smart surveillance system (Rai et al, 2019) that should be energy efficient (Kodali et al, 2018) and should be helpful even for physically disabled people (Faroom et al, 2018). The use of interaction based sensing  could play a significant role in the proposed work.

PRoPoSED MoDEL
For protecting the surveillance of a camera an edge detection technique is used here. This technique is formed based on canny. For the storage of consecutive images, one by one the image records are utilized.
In this process, initially, a camera is used to click an image. After that, a Canny edge detection model is applied to this graphical substance It is done for reducing the data size. After that these frames are accumulated on the system. In the same way, whenever a frame is captured it is reconfigured and processed by the edge detection technique. In this method, a comparison is made in the middle of the latest and the previous frame. If any variance or doubtful action is detected, that means if the current frame and previous frame are different then the status is updated to records. These records are placed in a remote cloud. In addition to this, a warning device is also activated in this condition.

RESULTS oF IMPLEMENTATIoN
At this point, a system that is used for the surveillance of the camera is put into operation. In this system, the technology of the internet of things is used. For putting this system into service MATLAB & ASP.NET are used. In this research, the images are taken through the camera with the help of the MATLAB tool. After capturing, it compares a fresh image in the company of the previous image. In this way, the modification in images is noticed. In this work, a component that is used for the capturing of an image has been formed in MATLAB.

Image Capturing Components
The components which are used for capturing pictures and for saving them in jpg form is as follows: rrr1 represents the extent of complementary in the middle of two pictures. A signal goes in the direction connected with the web link in situations where rrr1 does not match with I.

Comparison Between Traditional and Purposed System
The simulation of different image frames would be made here. A comparison of form n is made with the form n plus one. Comparison time and space consumed has been considered in this simulation. The time consumption during frame comparison is considered in this simulation. The image comparison time has been taken using the tic toc function. The tic function is placed before the module and toc function has been placed after the operation. As the Matlab script runs the time taken during operation is recorded in another variable. The variable is taken and stored to perform the comparison between traditional patterns where the edge detection mechanism is not applied and the proposed pattern where the edge detection mechanism has been applied. The edge-based image comparison would take lesser time as compared to the image without edge. Here below the MATLAB code is given which is used for the comparison of the proposed system with the traditional system on the base of Time Consumption.
Comparison of Time consumption in the middle of the present and future analogy system has been plotted in the Tabular and graphical form here.
In this graph, the time in second in the case of traditional work is compared to the time in second in the case of the proposed model. The following figure is showing that the proposed work (green line) is plotted below the traditional work (red line).

MATLAB Code to Simulate Queuing Delay Between Traditional and Proposed System
Here below the MATLAB code is given which is used for the comparison of the proposed system with the traditional system on the base of Queuing Delay. The algorithm for the process flow of plotting is shown below:

MATLAB Code to Simulate Comparison of Frame Size Between Traditional and Proposed
Here below the MATLAB code is given which is used for the comparison of the proposed system with the traditional system on the base of Frame Size. The algorithm for plotting the size of the frame is discussed below: 1. set tf to 9 2. set pf to 5 3. set ii to 10 and repeat to step 7 until ii is not equal to 500 4. set x (ii) to ii 5. set y1 (ii) to tf * i i 6. set y2 (ii) to pf * i i 7. set i to i + 10 8. put a hold on command on 9. sketch y1 in terms of x in red shade. 10. sketch y2 in terms of x in green color. 11. Set title to " Comparative analysis of traditional image size " 12. Set horizontal label as " Number of frames " 13. Set vertical label " Size in Kb " 14. Set legend to " Traditional work ", " Proposed work" for indexing Benchmark studies of file size in the middle of present and future analogy system has been plotted here in graphical form.
In this graph, the file size in kb in the case of traditional work is compared to the file size in kb in the case of the proposed model. The following figure is showing that the proposed work (green line) is plotted below the traditional work (red line).

CoNCLUSIoN
With the help of work which has been done here, the problems related to security systems are settled down. These problems are physical surveillance, insufficient storage space, and delayed notification. In this research, the observation of the camera has been made automatic and fast. In the research work which has been done here the requirement of human interference is completely removed. In addition to this, whenever a doubtful action is detected signal is transmitted at remote locations. Due to the use of the edge detection method in this work, the time required for doing frame comparison is less. In addition to this, the cost required for accumulation is also reduced. It happens because


Locality Constrained Analysis Dictionary Learning via K-SVD Algorithm

Recent years, analysis dictionary learning (ADL) and its applications for classification have been well developed, due to its flexible projective ability and low classification complexity. With the learned analysis dictionary, test samples can be transformed into a sparse subspace for classification efficiently. However, the underling locality of sample data has rarely been explored in analysis dictionary to enhance the discriminative capability of the classifier. In this paper, we propose a novel locality constrained analysis dictionary learning model with a synthesis K-SVD algorithm (SK-LADL). It considers the intrinsic geometric properties by imposing graph regularization to uncover the geometric structure for the image data. Through the learned analysis dictionary, we transform the image to a new and compact space where the manifold assumption can be further guaranteed. thus, the local geometrical structure of images can be preserved in sparse representation coefficients. Moreover, the SK-LADL model is iteratively solved by the synthesis K-SVD and gradient technique. Experimental results on image classification validate the performance superiority of our SK-LADL model.

Introduction
Sparse representation which encodes the signals or images using only a few active coefficients, has been successfully applied to many areas across signal processing and pattern recognition. Signals can be represented as a linear combination of a relatively small number of atoms in an over-complete dictionary. Sparse representation models can be classified into two categories by the learning method, synthesis dictionary learning (SDL) and analysis dictionary learning (ADL) [1,8]. A typical SDL model expects to learn the over-completed dictionary by minimizing the reconstruction errors such that it can linearly represent the original signals. Instead of learning an over-complete representation dictionary in SDL, the ADL model mainly focuses on learning a transformation matrix, and constructing sparse analyzed vectors. The ADL model has aroused much attention as it has a more intuitive illustration for the role of analysis atoms and high efficiency.
The traditional SDL model views the signal reconstruction problem as a pure approximation task, which may overlook signal intrinsic attributes, such as signal structure and density. To overcome this issue, several sparse coding models incorporating the geometrical structures of the image space have been proposed. They are based on the locally invariant idea, which assumes that two close points in the original space are likely to have similar encodings. Thus, additional regularizers are incorporated in order to satisfy specific requirements in various application scenarios. For example, graph regularized sparse coding based on nonlinear manifold learning is proposed for image classing and clustering applications [3,4,11,14]. Motivated by recent progress in sparse representation, we combine the ADL model with manifold learning, called locality constrained ADL with a synthesis K-SVD solver (SK-LADL). The flowchart of our proposed method for classification is illustrated in Fig. 1. The novel strategy incorporates the graph regularizer to emphasize the correlations between neighbors of training data, which could enhance the discriminative ability of the analysis dictionary. The rest of this paper is organized as follows. The basic dictionary learning framework is presented in Section 2. The locality constrained analysis dictionary learning model is presented in Section 3. The K-SVD algorithm is introduced in Section 4. The classification methodology adopted in our model is reviewed in Section 5. Experimental results on image classification are presented in Section 6. Relevant conclusions are finally given in Section 7.

Synthesis Dictionary Learning
Given signals Y = [y 1 , y 2 , · · · , y n ] ∈ R m×n , let D = [d 1 , d 2 , · · · , d k ] ∈ R m×k be a synthesis dictionary with a serials of atom d i , and X = [x 1 , x 2 , · · · , x n ] ∈ R k×n be the sparse coefficient matrix. The main idea of synthesis dictionary learning (SDL) is to approximately reconstruct the original signals Y by the combination of dictionary atoms d i (1 ≤ i ≤ k) with respective weight factors or coefficients X. The sparse optimization problem of SDL can be formulated as follows.
where Y − DX 2 F is the reconstruction error term and λ is a positive weight coefficient [1].
Sparse representation of the above SDL model can be classified in two basic tasks, i.e., sparse approximation and dictionary learning. The task of sparse approximation is to find a sparse solution to an underdetermined linear system. Some algorithms, such as matching pursuit (MP), basis pursuit (BP) and shrinkage method, have been well developed. On the other hand, dictionary learning is dedicated to search an optimal signal space to support the attribution of sparse vector under a certain measure. There exist a variety of numerical algorithms presented to achieve this objective, e.g., the method of optimal directions (MOD), K-means singular value decomposition (K-SVD) and recursive least squares (RLS).
The sparse analysis dictionary learning (ADL) model aims to learn a projective matrix (i.e., analysis dictionary) Ω ∈ R k×m with k > m to implement the approximately sparse representation of the signal y ∈ R m in transformed domain [7,8]. Specifically, it assumes that the product of Ω and y is sparse, i.e., x = Ωy with x i 0 = k − l, where 0 ≤ l ≤ k is the number of zeros in x ∈ R k . The sparse optimization problem can be formulated as follows. min where the representation error term X − ΩY 2 F shows the disparity between image representations in the transformed space and the coefficients with target sparsity level. The optimization problem in equation (2) above can be rewritten in the vector form as If the signal y i is known, its analysis representation with respect to a given dictionary can be obtained via multiplying by Ω. However, when the observed signal is contaminated by noise, the clean signal has to be estimated first in order to get its analysis representation, which leads to the analysis pursuit problem. Some algorithms like backward-greedy (BG), optimized-backward-greedy (OBG), and greedy analysis pursuit (GAP) have been proposed to address this problem [8].
The computational complexity of the basic ADL model is very high, and some recent work has relax the l 0 -norm in equation (3) to l 1 -norm or thresholding function [10].

Locality Constrained Analysis Model
The local geometric structure such as texture and contour information can be added to the sparse representation problem to upgrade the accuracy of signals approximation. Locality is more essential than sparsity, since locality leads to sparsity but not necessary vice versa [15]. The manifold learning theory can be used to quantify intrinsic geometric locality structure information. Many previous works have demonstrated that local structure embedded in high-dimensional vector space is very important to characterize both the intrinsic structure and discriminative structure of images [2]. Moreover, since the data is more likely to reside on a low-dimensional sub-manifold embedded in the high-dimensional ambient space, the geometrical information of the data is important for discrimination [4,6,14]. Therefore, by preserving the locality characteristics of the training samples, the discriminative ability of the learned dictionary can be improved. Motivated by previous works, a natural assumption here is that if two samples y i and y j are close to each other in the intrinsic geometric structure of the data distribution, the sparse representation vectors x i and x j over dictionary Ω , should be also close to each other. By use of techniques from Laplacian eigenmap method, we employ the graph Laplacian to build the local manifold structure. Specifically, we construct a supervised nearest neighbor graph G = {Y, W } with N vertices, in which each vertex denotes a data point, and the similarity matrix W over training samples is defined as where δ is kernel width and N(y i ) is the k-nearest neighbors of sample y i . To map the supervised nearest neighbor graph G to the sparse representation matrix X we minimize the following graph Laplacian criterion where L = B − W is called as graph Laplacian matrix, B is a diagonal weight matrix with ith diagonal entry is B ii = W ji . Thus, with the regularizer T r(XLX T ), the corresponding representations of any two points y i and y j are expected to maintain the same local structure of them. Then the sparse representation problem can be formulated as where α is weight coefficient, L is a weight matrix depended on the chosen manifold. As the Laplacian is symmetric and positive semidefinite, the objective (6) is a convex optimization problem. We rewrite the Laplacian regularizer T r(XLX T ) in equation (6) to the vector form as The optimization problem in equation (6) above can be rewritten in the vector form as When the dictionary Ω and all the other vectors {x k } k =i are fixed, we obtain the following optimization problem After some manipulations, problem (9) can be further cast as Then, merging two l 2 -norm terms in (10) yields the final SK-LADL model as where I is an identity matrix and R is initialized as an identity matrix, each element of p i is defined by

Solving the Problem
In this section, we adopt an alternative strategy to solve the SK-LADL model. The synthesis K-SVD and gradient technique are incorporated into the optimization procedure. The iterative optimization algorithm contains the following two steps: Update {X} Fixing the dictionary Ω, the solutions for X in the present iteration can be obtained by utilizing the synthesis K-SVD. Denote the two combined matrices in the parentheses of problem (11) as In the original K-SVD algorithm, the matrix Q new is column-wise l 2 normalized. The optimization problem (11) is equivalent to the following problem: The optimization problem in (15) can be solved by the K-SVD method, with entrance parameters being y new , Q new and T 0 . Let x k R be the corresponding coefficients of the kth column of Q new , and denoted as q k . Let E k = Y new − j =k q j x j R , then discard the zero entries in x k R and E k , with corresponding results marked asx k R andẼ k respectively. We optimize the following problem to obtain q k and x k R .
q k ,x k R = arg min DecomposingẼ k by K-SVD method, we haveẼ k = U ΣV T . Letx k R = Σ (1, 1) V (:, 1). After that, the nonzero values of x k R are replaced byx k R . Then the dictionary R t in Q t new can be used for the next iterator, i.e., R t+1 = R t . And the Q t+1 new is updated accordingly with R t+1 and the Laplacian matrix L. Update {Ω} Fixing X, the solution for Ω is computed through matrix derivation, followed by normalization of the rows of Ω. We add the following regularization term into the object function. The formulation in this situation is given as After omitting the independent terms, an equivalent problem is obtained as follows. min where β is a parameter which indicates the weight of the penalty term. The role of this term lies in avoiding singularity and over-fitting issues as well as ensuring the stable solution of dictionary. Let the first derivative w.r.t. Ω be zero and we can obtain the closed-form analytical solution for the dictionary as Then we normalize rows of the dictionary, which results in better performance empirically. The above training procedure is outlined in Algorithm 1.

Algorithm 1 Algorithm for solving our proposed model
Input: Training data Y , model parameter α,β. Output: The analysis dictionary Ω.

Classification Methodology
For classification of test samples on SK-LADL model, preliminary representation coefficient x i are obtained via the operation of multiplying Ω by the testing sample y i . Keeping in view that the proposed SK-LADL method imposes sparsity on coefficients, a hard thresholding operator is used to maintain the sparse characteristic of x i . The operator reserves elements with T 0 biggest absolute values and sets the others to be zero. Then, the multivariant ridge regression model is used to obtain a linear classifier from the sparse representations X for training samples Y : min where δ is a parameter for regularizing the solution, H = [h 1 , h 2 , · · · , h n ] is the label matrix of sparse analysis representation X, and each column h i = [0, · · · , 0, 1, 0, · · · , 0] T describes the label vector of the ith sample. The optimal solution of above problem is With the optimized W * , a testing sample y i can be predicted by picking the index of the maximum element of W * x i . Then let the predictive vector h ipre = W x i . The predictive vector h ipre has a very approximate shape to the corresponding real label vector h i , with only one element obviously bigger than others. Therefore, the location of the largest element in h ipre is utilized to determine the category.

Experiment Setups
In this section, we evaluate our SK-LADL model on four public image datasets, Extended YaleB (EYaleB), AR, Scene15 and UCF50. The above databases are widely used in evaluating the performance of sparse representation-based classification methods. The features are provided by [5] and [9]. On EYaleB and AR datasets, random features are generated by the projection with a randomly generated matrix. On Scene15 datasets, features are achieved by extracting SIFT descriptors, max pooling in spatial pyramid and reducing dimensions by PCA. UCF50 is a large-scale and challenging action recognition database. It has 50 action categories and 6680 realistic human action videos collected from YouTube. AR face database contains illumination, expression, and occlusions variations. We choose a subset consisting of 2600 face images from 50 males and 50 females. We compare our model with some state-of-the-art approaches: SRC [13], K-SVD [1], D-KSVD [16], LC-KSVD [5], ADL-SVM [10] and SK-DADL [12] with the above benckmark image datasets features. For fair comparison, the experiment settings we follow are in accordance with [5] and [12]. The sparsity is set as 45 in all the methods, and the dictionary atom is set between 500 and 600 which is the integral multiple of the number of classes in different datasets. There are four parameters in SK-LADL model, i.e., α, β, δ and k, where δ = 0.01 and k = 3 are preset and the parameter α and β are tuned by 5-fold cross validation and optimized by using grid search strategy. We firstly search in the larger range of [10 −3 , 10 −2 , · · · , 10 2 , 10 3 ] for each parameter and then search a smaller grid with proper interval size determined by preliminary classification results.The best parameters we set in each database are listed in Table 1.

Results and Analyses
We repeat the experiments 5 times on different selected training and testing image features, and the mean accuracies are reported. Table 2 shows the mean classification accuracy results on different datasets. As can be seen, our method achieves notably higher accuracy than SRC, KSVD and LC-KSVD on all four databases. This is mainly due to the locality preservation achieved in our method for ADL model, which ensures that similar training samples tend to have similar coding coefficients. The SRC model that directly uses all training samples as the dictionary will introduce noise for the sparse representation. The locality constraint on representation coefficients can narrow the selection of representative analysis atoms on geometric manifold, enhance the representational ability of homogeneous samples to some extent, and this may help to overcome the above noise disadvantage. Our method also achieves favorable results compared with the two ADLbased methods. This is mainly because the ADL-SVM and SK-DADL model only utilize joint-or post-learned classifiers with analysis representation without inherit the underlining structure of training samples. Therefore, with only a simple post-learned classifier, the integration of the locality information yields attractive discrimination for SK-LADL model compared to other conventional ADL models. As for the testing efficiency, Table 3 shows the time for classifying one testing image on databases EYaleB (dictionary size = 570) and AR (dictionary size = 600). As can be seen, our method performs better than SK-DADL, due to the locality constraint on sparse coding, which could generate similar analysis representation of heterogeneous samples. The accuracies and time costs in tables can demonstrate that our SK-LADL model has huge potential in pattern classification tasks.  Fig. 2 shows the confusion matrix for our proposed SK-LADL method on Scene15 dataset. It presents proportion of images in each category classified to all categories. We can observe that most images can be classified into the right category, with some class even getting all right classification. From the figures, we can conclude that the desired effect of our proposed SK-LADL method is reached.

Conclusion
In this paper, we proposed a novel discriminative analysis dictionary learning model with locality constrained regularizer(SK-LADL), which takes into account the intrinsic geometric structure of the training samples by introducing the locality constraint term into the framework. To effectively solve the proposed objective function, an iterative algorithm based on the conventional synthesis K-SVD and gradient technique are applied to our SK-LADL model. Experimental results on four benchmark image datasets classification demonstrate the superiority of the SK-LADL method over other state-of-the-art ADL approaches. In the future, we will incorporate locality constraint on dictionary atom and coefficients, addressing the problem when both training and test images are corrupted.


Dominant Patterns: Critical Features Hidden in Deep Neural Networks

In this paper, we find the existence of critical features hidden in Deep NeuralNetworks (DNNs), which are imperceptible but can actually dominate the outputof DNNs. We call these features dominant patterns. As the name suggests, for a natural image, if we add the dominant pattern of a DNN to it, the output of this DNN is determined by the dominant pattern instead of the original image, i.e., DNN's prediction is the same with the dominant pattern's. We design an algorithm to find such patterns by pursuing the insensitivity in the feature space. A direct application of the dominant patterns is the Universal Adversarial Perturbations(UAPs). Numerical experiments show that the found dominant patterns defeat state-of-the-art UAP methods, especially in label-free settings. In addition, dominant patterns are proved to have the potential to attack downstream tasks in which DNNs share the same backbone. We claim that DNN-specific dominant patterns reveal some essential properties of a DNN and are of great importance for its feature analysis and robustness enhancement.

Introduction
Deep Neural Networks (DNNs) are proved as susceptible to adversarial perturbations [29,7,15,17,2,27]. The basic procedure of attacking a well-trained DNN f is to generate an imperceptible perturbation δ from an image x, such that the perturbation could change the prediction of f . Mathematically, the perturbation is dependent on x and f , which could be denoted as δ(x, f ), and satisfies where the latter constraint ensures that the perturbation is imperceptible by restricting it in an p norm bound ξ. By contrast, the goal of defenders (also network trainers) is to train a network insensitive to any imperceptible perturbations, that where X is the dataset that f works on.
Typically, δ is regarded as a special noise, which has seldomly been treated as a meaningful signal. However, if we drop the prejudice that "the images are signal, while the perturbations are noise", we will discover that the output of the DNN is actually determined by x and δ jointly. In other words, although the amplitude of δ is too small to be perceived by human beings, it could still be meaningful and contain much information from the perspective of f , and sometimes even has a more significant effect compared to x. With this idea, corresponding to equation (2), we come up with a question:  Figure 1: An example of dominant pattern. The dominant pattern is generated from a given DNN and has been mapped from [-10,10] to [0,255] for better observation.
Given an f , is there a perturbation δ that can provoke a high response from f and make it insensitive to natural images x? That is, for a well-trained network f , can we find a δ(f ), satisfying f (δ(f ) + x) = f (δ(f )) and δ(f ) p ≤ ξ, ∀x ∈ X.
To some extent, equation (3) can be regarded as a "dual" version of equation (2). For a DNN f , it is actually pursuing a δ(f ) that can dominate DNN's output and thus we name it as dominant pattern.
The dominant pattern is small and could be ignored in magnitude but is a critical feature hidden in DNN. Theorically, dominant patterns are data-independent and DNN-specific which could reflect some essential properties of DNN. Therefore it is a very valuable things and is helpful for network analysis, feature analysis, and perhaps robustness enhancement.
In this paper, we design an algorithm to find the dominant pattern for a given DNN. The crucial idea is to keep the output insensitive to x ∈ X, so that make δ more relevant to the nature of DNN .
Here, we achieve such insensitivity by optimizing a designed Cosine-Similarity Loss (CoSLoss) to encourage the similarity between f (δ) and f (δ + x).
As an example, we demonstrate δ(f ) and its influence on the DNN in classification task in Figure  1, where δ is the dominant pattern found in a well-trained VGG19 [26] model on ImageNet. The pattern is predicted as "iguana", and added with it, the original images are amazingly all mispredicted as "iguana" by VGG19.
A direct application of dominant patterns is the universal attacks. Such attacks generate a set of adversarial examples by crafting only one perturbation, named the Universal Adversarial Perturbation (UAP), which could be realized by a dominant pattern. Comprehensive experiments show that dominant patterns outperform state-of-the-art (SOTA) UAP methods. Actually, a UAP pursues a perturbation to affect the output, i.e., to let the output be a function of both x and δ, while the dominant pattern pursues to determine the output, i.e., the output only depends on δ. This essential difference between them accounts for the gap in their performance.
Notably, the dominant pattern is a unique nature of DNNs instead of that for specific data. On one hand, the dominant pattern can be calculated from any unlabeled image and perform significantly better than previous label-free UAP methods. On the other hand, any downstream task that uses the DNN's backbone, i.e., the main part of this DNN, will also be controlled by the dominant pattern. The special properties of dominant patterns make them intrinsic and critical features hidden in DNNs, and provide a new viewpoint for understanding DNN's vulnerability and enhancing their robustness in the future.
The major contributions of our work can be summarized as: • We raise an idea that a well-trained DNN can be controlled by a special perturbation named dominant pattern. We verify the existence of dominant patterns by designing an algorithm based on cosine similarity to pursue insensitivity in the feature space.
• Since dominant patterns are independent of data, any unlabeled natural image set can be used in our algorithm. We substantiate that when dominant patterns are used as UAPs, they can outperform several SOTA methods, especially in label-free settings. 2 • We verify the dominance of these patterns when transferred to downstream tasks, indicating that dominant patterns are actually intrinsic features hidden in DNNs as a potential helper for model analysis and robustness enhancement.

Related Work
Recent studies have shown that DNNs are susceptible to maliciously constructed small noise called adversarial perturbations. In this section, we roughly classify them as image-specific attacks and image-agnostic ones as described below.

Image-specific attacks
Szegedy et al. [29] first demonstrates the existence of small perturbations that can fool DNNs. Since then, many attacks have been proposed, including Fast Gradient Sign Method (FGSM) [7], Projected Gradient Descent (PGD) [15], C&W [2], and DeepFool [17]. The general attack procedure is to optimize the input sample to adversarially maximize the loss, while restricting the magnitude of perturbations. It is also demonstrated that the adversarial examples generated by attacking a DNN may transfer to other DNNs as well [6,32,3]. These attacks are image-specific because they craft different perturbations for different samples.

Image-agnostic attacks
Image-agnostic attacks, in contrast, change a DNN's predictions on different inputs with a single imperceptible universal adversarial perturbation (UAP). The first UAP is proposed by Moosavi-Dezfooli et al. [16], where an iterative procedure based on Deepfool [17] is designed. Mopuri et al. [20] propose a Network for Adversary Generation (NAG) to model the distribution of adversarial perturbations. Omid Poursaeed et al. [25] present a framework named GAP which can craft universal perturbations for both classification and semantic segmentation tasks. Different to previous methods, GAP can achieve targeted attacks. The methods mentioned above require the knowledge of original training data, which may not be feasible in practice. Thus, some researchers try to train universal perturbations without using any training data [19,18,21,14]. In [19] and [18], the perturbations are generated via corrupting the extracted features at multiple layers. By contrast, [21] trains a generative model to craft UAPs by utilizing the class impressions, which is a generic representation of the samples belonging to certain categories, while [14] considers the model uncertainty to craft an insensitive universal perturbation. Recently, more types of UAPs have come out to fit different demands. Philipp Benz et al. [33] propose a framework tailored for the class discriminative universal attack, where the perturbation fools a target network to misclassify only a chosen group of classes. Targeted UAPs can also be found by exploiting a proxy dataset instead of the original training data [34].

Discovering the dominant pattern
In this section, we formally define dominant patterns and then propose an algorithm to find them.
As the name suggests, the dominant pattern is a small perturbation δ that can determine the output , where X can be any unlabeled natural image set with or without intersection on the target DNN's training set. We keep the output feature of DNN unchanged by solving the optimization problem as maximize where ξ is a small value ensuring the quasi-imperceptibility of δ, and L is a loss function used to measure the similarity between vectors as discussed in detail later.
Since the constraint function δ p ≤ ξ can be easily realized by clamping the variable's value during training, the key to the algorithm falls into how to quantify the similarity between f (δ) and f (δ + x). It is worth noting that feature f (·) is a high-dimensional vector. Typically, there are several metrics to measure the similarity between vectors, such as cosine similarity, Euclidean distance, and Kullback-Leibler divergence, which are defined as follows.

Cosine Similarity Loss (CoSLoss).
Cosine similarity is the cosine of the angle between vectors. It treats the input vectors a, b as normalized, which is a judgment of orientation and not magnitude, and its outcome is bounded in [0, 1]. The higher the cosine similarity is, the more similar the two vectors are. Therefore, the CoSLoss can be defined as where n denotes the dimension of a and b, a i and b i denote the i-th dimension of the vector a and b, respectively.

Euclidean Distance Loss (EDLoss)
. Euclidean distance is defined as the length of the line segment between the two points. The loss according to this similarity metric can be directly designed as 3. Kullback-Leibler Divergence Loss (KLDLoss). If we treat the input vectors a and b as two probability distributions, then Kullback-Leibler divergence is defined to measure the similarity between two probability distributions. KLD is an asymmetric measurement function used to measure how well a probability distribution a fits the target distribution b.
In our work, we regard f (δ) as the target distribution, since δ is the main part we optimized.
Theoretically, CoSLoss is a better choice, because EDLoss is easily affected by magnitude, while KLDLoss is an asymmetric metric thus is not easy to optimize. To verify our choice, we choose a classification task to compare the impact of different losses on feature similarity. We define a metric called dominance ratio, i.e. the ratio of perturbed images (δ + x) that are predicted as the dominant pattern's class, to measure the performance of these losses. We attack VGG19 [26] on ImageNet with three mentioned loss, which is calculated at the logit layer of DNNs since it is highly related to the dominance ratio. In Table 1, we list the dominance ratio of patterns found by minimizing different losses (the detailed setting could be found in Section 4). Because cosine similarity outperforms the others, we choose it as the objective in the algorithm.
The optimization is iteratively proceeded over training data to minimize the CoSLoss. Adam [10] optimizer is applied and the perturbation is clamped by ∞ norm in each iteration to satisfy the magnitude requirement. The details are summarized in Algorithm 1.

Algorithm 1 Finding dominant patterns for a given DNN
Input: DNN f , loss function L CoS , training set X, perturbation magnitude ξ, batch size b, maximum number of epochs m, learning rate for Adam optimizer lr.

Experiments
In this section, we first verify the existence of dominant patterns, showing that the proposed algorithm can indeed find patterns dominating DNNs. Further, the layer-wise output of a DNN is visualized to better understand the dominance process. After that, we demonstrate that dominant patterns can be applied to UAP tasks and defeat state-of-the-art methods. Moreover, we discuss its superiority over UAP methods in terms of their dominance and task transferability.
In our experiment, the public datasets we used include validation set of ImageNet ILSVRC2012 [4] (50, 000 images), MSCOCO 2017 [13] (118, 287 images) and CIFAR10 [11] (50, 000 images). Among them, the ImageNet validation set is not divided into training and test sets, therefore we randomly split it into two parts, where 40, 000 pieces are used for training, and the rest 10, 000 are used for testing. We also create an artificial dataset with 4, 000 images in four classes (1, 000 per class) by using BigGAN [1] of which the official model can only generate four classes.
In UAP tasks, comprehensive baselines are chosen for comparsion, including UAP [16], GAP [25], NAG [20], FT-UAP [5], TUAP [34], FFF [19], AAA [21], GD-UAP [18], and PD-UA [14]. For fair comparison, we inherit all hyperparameter settings from the respective papers, and the generated dominant patterns and UAPs are restricted with δ ∞ ≤ 10 for images in range [0, 255]. To evaluate fooling performance, the most common metric Fooling Rate is used, which denotes the proportion of test images for which the prediction of the target DNN is changed after adding the perturbation.

Dominant patterns
We first implement Algorithm 1 on four representative DNNs with ImageNet validation set and COCO 2017 training set. The calculated dominant patterns are shown in Figure 2. Since the max magnitudes of the results are restricted to ξ = 10 which is nearly invisible, we map the magnitudes from [-10,10] to [0,255] for better observation.
Visually, the found patterns are regular textures, which contain certain information. According to Table 2, although such a pattern is meaningless compared with natural images, the corresponding DNN identifies it as a certain class (named dominant class) with very high confidence. We repeat the experiments several times and find a stable dominant class for each setting. Quantitatively, their dominance ratio (defined in Section 3) are also reported. After adding a dominant pattern, more than three-quarters of images are misled to the dominant class, which originally belong to 1,000 classes. The high dominance ratio demonstrates that the generated dominant patterns successfully attract DNN's attention and take control of its output.
By taking a closer look at the respective performance of dominant patterns trained on ImageNet and COCO, we find that for VGG19 and RN152, their dominant classes have not changed when given different data prior information. However, that is not true for the other two DNNs. Actually, the training procedure of dominant patterns depends on training data. Since the distribution of training set varies from each other, dominant patterns trained with them may belong to different classes.
To investigate how these seemingly meaningless patterns dominate the DNNs, we visualize the layer-wise output of VGG19 corresponding to a randomly chosen natural image x, the dominant pattern δ, and the perturbed image δ + x in Figure 3. The horizontal axes denotes the ReLU layer index. Obviously, along with the forward propagation from input to output, the dominant pattern embedded in δ + x gradually seize the control role such that the features become less related to x and more similar to δ. As a numerical comparison, the Structural SIMilarity [31] (SSIM) of δ + x v.s. δ and that of δ + x v.s. x are given.  The above is not a cherry-picked case. We repeat this experiment for all the 10,000 images in the test set and report the average SSIM with standard deviations in Figure 4, from which one could observe a similar trend. This phenomenon indicates that the damage of dominant patterns permeates through the whole DNN, and has already been accomplished at the output of the backbone (the 16 th ReLU layer). That is to say, dominant patterns destroy DNNs by disabling their ability to extract meaningful features from natural images. Since feature extraction is a key in the computer vision, this inspires us that dominant patterns are attributes for DNNs and can serve as an all-in-one perturbation for DNNs with the same backbones in different tasks.

Performance in UAP task
In a universal attack task, one seeks a universal perturbation δ that can change the prediction of most images from dataset X, i.e., the attackers find a δ p ≤ ξ that satisfies Compare this equation with the dominant pattern's definition in equation (3), it can be easily found that the dominant patterns can also change DNN's prediction. Therefore, they can be applied to UAP tasks. We discuss its performance from two aspects according to the type of training set used.
Train with original data. The most attack-friendly situation is to train UAP on labeled data that is also used for training the DNNs, which is called label-dependent attack. In this case, UAPs could achieve high fooling performance. The fooling rates calculated by applying different attack methods to different DNNs are shown in Table 3, where the best fooling rates are highlighted in bold, and the second-best ones are underlined. For comparative experiments whose results are not provided in the corresponding papers, we mark them with '-'. From the table, it can be concluded that our dominant patterns defeat the SOTA method. Figure 5 shows some adversarial examples generated from dominant patterns for different DNNs.  use an unlabeled proxy dataset that is irrelevant to target models. Such data can provide data prior information, leading to label-free attacks. Some attackers also attempt to conduct data-free attacks, where no data is utilized during training.
According to Algorithm 1, finding dominant patterns is a self-supervised process thus does not requires any data labels, which satisfies a label-free setting. We use MSCOCO [13] and artificial images generated by BigGAN [1] as our proxy dataset. Details of these datasets are stated in Section 4.1. The fooling rates are reported in Table 4, where the performance of data-free UAPs (row 1-4) and label-free UAPs (row 5-7) are given for comparison. Combined with Table 3, it can be concluded that dominant patterns obtained from proxy data not only outperform all label-free and data-free attacks, but also are comparable to SOTA label-dependent attacks. The outstanding performance further enhances the practicality of dominant patterns. Notice that, the performance gap between COCO and BigGAN is mainly due to the lack of diversity in BigGAN pictures (with only four classes).   Table 5. Here, the top-k ratio means the overall proportion of top k most frequently occurred classes.
Although it has been discovered in [16] and [30] that UAPs tend to aggregate images to some dominant labels, dominant patterns gather perturbed images to several certain classes to a large extent that most UAPs cannot achieve, as shown in Table 5. This verifies our claim that "UAPs are not dominant patterns". The direction aggregation implies that DNNs' vulnerability is class-dependent and there are certain flaws that are extremely fragile. TUAP is a newly proposed method that can take into account the multiple dimensions of the logit vector to fully utilize information. It satisfies some of the conditions for obtaining dominant patterns, so that it can perform well on dominance ratio.
Dominant patterns can be transferred to downstream tasks. According to Figure 3 and dominant patterns' definition, even the features extracted by the backbone of DNNs can also be controlled by dominant patterns. Theoretically, this phenomenon will be kept if the backbone of a pre-trained DNN remains unchanged in the downstream tasks. However, when using the pre-trained network, it is common for trainers to freeze the parameters of the backbone and then train a head for downstream tasks. This is a potential risk and can be exploited by our dominant patterns.
To verify this assumption, we transfer the VGG19, which pre-trained on ImageNet, to CIFAR10 classification task by freezing the parameters in convolution layers (backbone) and fine-tune the ones in fully connected layers (classifier head). And then we use the dominant pattern obtained from the pre-trained VGG19 to attack the new DNN fine-tuned in CIFAR10. Consequently, the accuracy in CIFAR10 task is promoted to 87.02%, and the results are shown in Table 6. The dominant pattern still serves as a strong perturbation which shows extremely high confidence and dominance ratio. Note that since CIFAR10 is a 10-category classification task, the theoretical highest fooling rate is 90% because the clean samples which share the same prediction with the dominant pattern will not change their predictions after attacked.
More importantly, the reported CoSLoss remains at a very low level on the fine-tuned network. This indicates that the DNN is still in control of the dominant pattern, even though the task has already changed. This phenomenon further verifies that the proposed dominant pattern can attack any DNNs that use the same pre-trained backbone without any changes. It is also the reason why we call the dominant pattern "the unique nature of the DNN".

Conclusion
In this work, we provide a new perspective on perturbations, and propose that there are dominant patterns hidden in DNNs, which is a critical feature of DNN and can help understand and improve the DNN's robustness. Although small and invisible in the image space, dominant patterns have some meaningful patterns that have the ability to mislead the majority of natural images to their own classes. Through numerical experiments, we verify the existence of such pattern and visualize how its dominance takes place. Further, we demonstrate the outstanding performance of dominant patterns in UAP tasks, and analyze their particularity beyond the traditional attacks. In the future, it is possible for researchers to go deeper into the properties of dominant patterns, and apply it to realize better model defense in further study. Our proposal reveals the underlying threat hidden in DNNs, but since we focus only on the digital space and white-box attacks, it does not induce imminent threats to the applications of DNNs in the physical world.
A Error bars for experiment on Universal Adversarial Perturbation task    The confidence and dominance ratio are not so satisfying as the ones trained with ImageNet and COCO dataset, especially for VGG19 and DN121. We think this is a foreseeable result, considering that the BigGAN based dataset is not exactly a natural image set, but close to the natural image distribution. More importantly, due to the restriction of open source, the generated images can only cover 4 over 1,000 classes in ImageNet, which makes it a rather challenging task.

13
C Adversarial Examples generated by utilizing proxy image set.