extfieldsofstudy: Physics Business

text:
Research on design scheme of urban public transportation for energy conservation

The transportation system promotes the development of the metropolitan area, while the number of vehicles in the transportation system is increasing, and the speed of energy consumption is also increasing. The consequences are not only the destruction of human living environment, but also many urban problems such as traffic congestion and urban environmental pollution for people living in the metropolitan area. In order to solve these problems, the article proposes a design scheme for urban public transport from the perspective of energy conservation. Through the modular design method, combined with the existing electric vehicle drive technology, the public transport is divided into two modules: the ride module and the drive module; at the same time, the existing traffic information service system is integrated to construct service system of the information guide to support the public transport scheme. The public transport design provides a possible solution to environmental pollution and traffic congestion.

Introduction
A metropolitan area refers to a cluster of cities with a population of more than 25 million and a population density of more than 250 people per square kilometer [1]. In terms of spatial form, it is an area consisting of the urban core and its surrounding towns, e.g. Tokyo Metropolitan Area and Paris Metropolitan Area. The urban transport network of a cluster of cities is the infrastructure that supports and ensures its development [2]. While the transportation network system promotes urban development, its growth rate of energy consumption is also accelerating [3]. Transportation accounts for more than 21% of the world's total energy consumption, while developed regions such as the European Union account for about 34% of the world's energy consumption, and the annual growth rate of 30% continues to rise [4]. The energy consumption of China's transportation network system accounts for about 8% of the total energy consumption, which is increasing with the improvement of China's urban motorization level [5,6]. Traffic consumes a lot of energy, which has brought great damage to the environment. In addition, there are traffic congestion, environmental pollution, parking supply and demand contradictions and other issues. Yuan Jiadong, et al. [7] pointed out that light rail transit systems with a medium transport capacity play an important role in meeting the public transport demands in metropolitan areas and that developing urban public transport with light rail transit systems with a medium transport capacity serving as the backbone is an important direction for the construction of metropolitan areas in China. Zhao Rujiang, et al. [8] pointed out that the personal rapid transit (PRT) system (a kind of driverless public transport system for individual transport), compared with the conventional public transport system, has obvious advantages. It was successfully put into operation in Masdar City and London Heathrow Airport, setting up a good example for Guangzhou to mitigate the traffic congestion in its urban area. Zhao Ling, et al. [9] analyzed the need to develop IoT and set up a guidance system for urban traffic congestion, providing a new solution to urban traffic congestion.
The light rail transit system with a medium transport capacity mentioned in the previous research has a large capacity, is suitable for operation in multiple periods, but it cannot provide passengers with comfortable, flexible, and convenient experience as private cars. The PRT system can provide passengers with comfortable, flexible, and convenient experience but it is difficult for it to finish the point-to-point distance. The public information management system proposed in the previous research is applicable to both private and public vehicles and can improve the efficiency of traffic management, but it is difficult to fundamentally solve urban traffic problems due to the national preferential policy for public transport and the limited transport and road resources. Based on the current situations, a modular concept of public transport means oriented to personal travel is proposed in the present paper to improve personal riding experience and to solve the problem of traffic congestion arising in the urban transport.
2. The influence of travel modes within the metropolitan area on urban transportation energy 2.1. Characteristics of public transport In terms of spatial structure, Tokyo Metropolitan Area is composed of a core city, i.e. Tokyo City and surrounding towns such as Yokohama, Tama, Saitama, Tsukuba, Chiba in a circular layout. Such a layout is based on rail transit systems. In Tokyo Metropolitan Area, there are rail transit systems like the metro and railway that are rapid and have a large capacity [10]. Similarly, Paris became the core of Western European Metropolitan Area due to its developed transport systems. The urban rail transport system includes metro lines with a total length of 250 km and nearly 300 stations, covering the whole urban area. In addition, there are convenient transport systems by water, land, and air between Paris and other European cities [1]. They constitute a multi-level public transport system. According to the analysis on transport development of major metropolitan areas around the world, a multi-level public transport system usually consists of various transport modes such as rail transit, mass transit, and ground transit [11]. From the perspective of travel, public transport systems allow passengers to transfer between them. The corresponding mode of public transport organization in a metropolitan area is "trunk + shuttle" [11], and the model diagram is shown in Figure 1. usually including the travel for entertainment, shopping, visiting relatives, etc. [12] Public transport modes in metropolitan areas -"trunk + shuttle" determine that most people transfer between different modes of transport, i.e. travel of multiple modes. In the real life in metropolitan areas, the commuting travel is related to non-commuting travel, sometimes with the non-commuting travel integrated in the commuting travel. For example, many commuters integrate non-work travel in the long-distance commuting or rest. The commuting travel is relatively complex. It is necessary to analyze the process of travel by public transport in personal commuting travel. A personal commuting travel by public transport usually consists of several basic processes, i.e. from home to a public transport station, travel by public transport, and from another public transport station to work place. Depending on specific conditions, the commuter may transfer for several times and select various transport modes during the travel (see Figure 2). With the addition of the multi-mode transport from the work place to home, such travel processes constitute a commuting travel chain. At the same time, a [home]-[work place]-[home] travel chain model can be built (see Figure 3). A travel chain is a round trip composed of travel purposes arranged in a chronological order to complete one or more activities. It contains a large amount of information about time, space, mode and activity type.  A complex travel chain hinders the popularization of multi-mode travel, makes it difficult to shift to public transport, and increases the dependence of travel on private vehicles such as cars [13]. The multimode travel based on public transport is popularized because it is a green transport mode with less negative impact on the environment which can improve the overall transport operation efficiency and contribute to the sustainable development of cities. The integration of non-commuting activities to the commuting travel process makes the commuting travel chain more and more complex. In this case, personal travel requires convenient and rapid modes of transport. Mass transit systems do not have such features but private vehicles (cars) are rapid, convenient and comfortable. This leads to the contradiction of selection between private vehicles such as cars and public vehicles for travel (see Table 1). The preference is to choose private vehicles such as cars, and the increase in the use of private vehicles such as cars has made air pollution more serious. The emission of atmospheric pollutants from urban traffic based on cars is an important factor causing smog and photochemical smog pollution, and is one of the most important sources of air pollution in urban environments [14,15]. In order to prevent pollutants from exceeding the capacity of the urban atmospheric environment, the relationship between the influencing factors of urban transportation energy consumption should be analyzed. The intensity of travel demand directly affects the amount of traffic, which in turn affects traffic energy consumption and pollutant emissions [16,17]; when the demand intensity of travel is constant, the intensity of energy directly affects urban traffic energy consumption and pollutants. How much is discharged. Therefore, in ensuring the existing urban travel intensity and effectively reducing the energy intensity, the pollution of urban traffic to the urban environment can be reduced. It can be understood as reducing the travel of private vehicles such as cars and optimizing the existing public transport to reduce energy consumption and reduce environmental pollution. 3. Traffic information service system and traffic energy management system An intelligent transportation system (ITS) is a real-time, accurate, efficient, comprehensive transport and management system functioning in a large scope and in all aspects which is established by effectively integrating and applying such advanced technologies as the information technology, data communication and transmission technology, electronic sensor technology, electronic control technology, and computer processing technology to the whole transport management system based on complete infrastructure including roads, ports, airports and communication [18]. One of its functions is to release real-time dynamic public transport travel information to the public transport travelers so that they can select their transport mode and adjust their route in a real-time manner according to the travel information.
An advanced travel information system (ATIS) is one of the key subsystems for ITS development [19]. The ATIS collects, processes, transmits and releases transport information by applying advanced communication and information technologies, and finally provides real-time dynamic transport information services for travelers (see Figure 4).

Figure 4. Function of ATIS
The integration of Internet, cloud computing, big data and other advanced technologies with ITS not only enhances the functions of ATIS, but also constitutes a network system for transportation energy. As for the construction of energy network, Jeremy Rifkin of the United States in the Third Industrial Revolution proposed that renewable energy should be the main energy unit, and the use of Internet technology to coordinate the control of multiple energy sources and energy storage equipment to achieve sustainable development of energy consumption [20]. For example, based on the concept and architecture of smart grid, a railway energy management system (REMS) is constructed to realize the optimal utilization of energy [21]. The combination of traffic energy network and traffic information service system not only helps the effectiveness of personal travel, but also can effectively use energy, achieve the goal of reducing energy consumption, and make the urban environment sustainable development. Therefore, the design of public transport should be closely combined with these two aspects.

Public transport modular design method
According to point 2 of part 2 above, there is contradiction of selection between private vehicles and public transport means for personal travel. In order to get rid of this contradiction, we proposed a solution to transform the personal travel chain model into the travel chain model of the public transport means on the basis of multi-mode transfer for personal travel (see Figure 5). In terms of the concept of set, a transport mode can be described as: transport mode = {rail transport, road transport, waterway transport, air transport,…}. In order to realize changes of transport modes for vehicles, the modular method is applied to the design of new public vehicle. Modular design is a method by which a series of modules is divided and built on the basis of analyzing products with the same functions but different properties and specifications within a certain range, so as to quickly form products meeting different requirements through the selection and combination of modules. In the modular design, functional modules are divided according to the following principles [23]: 1) Ensure the functional and structural independence and integrity of modules; 2) Allow easy connection and disconnection of interface elements between modules; 3) Minimize the correlation between modules. Namely, ensure weak coupling between modules and strong coupling between internal parts of modules; 4) Keep moderate granularity of disconnection; 5) Use typical components and ensure versatility, etc. According to the principles described above, modules are divided for a Tesla electric vehicle (see Figure 6). The electric vehicle falls in the road transport mode, which does not allow changes in other transport modes if its original combination mode remains unchanged. It is necessary to restructure the modules. The electric vehicle is divided into two modules under such conditions as the information and communication technology, IoT technology, unmanned driving technology, drone technology, and motor technology, i.e. a riding module and a driving module (see Figure 7). Similarly, a public vehicle is divided into two modules according to the functions, namely, a riding module and a driving module. The riding module is a fixed module that only meets the need of personal travel. There are two modes available: the work and the leisure. The driving module is a module variable corresponding to the mode of transport. The driving module = {rail driving, road driving, waterway driving, air driving, …} (see Figure 8). The driving module can be realized by the information and communication technology, IoT technology, drone technology, unmanned driving technology, and motor technology. For example, in a road driving module, the motor is installed on the wheel to drive the vehicle directly. Based on the information and communication technology and IoT technology, the unmanned driving technology enables vehicles to reach their destination safely. In an air driving module, the information and communication technology, IoT technology, and drone technology enable people in the riding module to go to their destination conveniently and comfortably. In the design of new public vehicle, modern advanced technologies are applied and advantages of private cars and public vehicles are integrated to make the new public vehicle rapid, comfortable and convenient as private vehicles and environmentally friendly and conducive to sustainable urban development as public vehicles. In order to ensure the normal operation of new public vehicles, there must be a new public transport service system adequate to them.

Public transportation information service system
The modular design is also used to build a service system for the new public vehicle. The service system mainly consists of four modules of data management and control center, ITS communication network, onboard terminal, and intelligent terminal (see Figure 9). Its core module is the data management and control center that uses the cloud computing technology for real-time management and analysis of massive information data so as to coordinate and plan the data flow and directly schedule and control the new public vehicle. The information data mainly comes from the module of ITS communication network. The ITS communication network module is connected with the existing intelligent transport system, especially its subsystem ATIS, to obtain traffic information such as traffic flow, road condition, weather, temperature, traffic accidents around the clock. The new public vehicle selects the best route depending on such transport information. The onboard terminal module is a system embedded in the new public vehicle to receive commands from the data management and control center, make the route plan according to the transport information, autonomously generate the combined route of the riding module and the driving module, and enable the new public vehicle to send passengers to their destination. In addition, the new public transport vehicles can also sense each other and implement real-time information exchanges. The intelligent terminal module is an application installed on the existing terminal devices including personal computers, tablet computers, smart phones, smart bands and other handheld intelligent devices. Handheld intelligent devices are connected to the data management and control center over the wireless network and exchange information with the onboard terminal.  The service system runs according to the process flow as follows: the user of the handheld intelligent device logs in to the server of the service system through the application, enter the origin and destination according to the prompt, and send a personal travel request to the data management and control center. According to actual transport conditions, the data management and control center analyzes the data, provides the best route and time options, generates a combined route, and issue a command to the new public vehicle closest to the traveler. During the travel, the new public vehicle dynamically adjusts the combination of the route and the riding module and driving module in a real-time manner according to the transport conditions, and finally reaches the destination safely. The operation process flow of service system is shown in Figure 10.

Conclusion
Urban problems such as the urban traffic congestion and transport-induced environmental pollution attract more and more attention of all the countries. Transport policies mostly give priority to the public transport. Facing the accelerated urbanization and complex personal travel chain, most people prefer convenient, rapid and comfortable private vehicles (cars), which hinders personal travel from transition to public vehicles. Based on the information and communication technology, mobile internet technology, IoT technology, and unmanned driving technology, etc., the research on conceptual design of public vehicles is done with the modular design method. The design divides a public vehicle into two parts: the riding module and the driving module. The former is a fixed module and the latter is a changing module. The vehicle runs in such a mode as to provide the best route in a real-time manner according to the analysis data of the service system and at the same time, the riding module and the driving module are combined according to the actual conditions. The new public vehicle provides convenient personal travel services, and enables people to use private vehicles (such as cars) less frequently, and thus mitigates traffic congestion and environmental pollution.
The limitation of the conceptual design of the new public vehicle lies in that it is an urban design targeted to metropolitan areas and it is difficult for the vehicle to operate in cities outside metropolitan areas. Resources like advanced technologies, abundant capital, outstanding talent in metropolitan areas make the realization of the conceptual design of new public vehicle possible. The mature application of the internet of vehicles and the unmanned driving technology is a key factor for the success in realizing the new public vehicle and a challenge for realizing the scheme. The internet technology (including the mobile internet) will improve the operation efficiency of the new public vehicle in the aspect of operation and management. As for production, the modular design provides a foundation for industrial batch production. Mass production can be done through the assembly of modules. All these conditions are the means to realize the conceptual design of the new vehicle, so as to fulfill the ultimate goal of improving the travel efficiency for public transport travelers in metropolitan areas.


extfieldsofstudy:
Computer Science
Medicine

s2fieldsofstudy:
Computer Science
Medicine

text:
Comparison Analysis of Linear Discriminant Analysis and Cuckoo-Search Algorithm in the Classification of Breast Cancer from Digital Mammograms

Objective: Breast cancer is the most common invasive severity which leads to the second primary cause of death among women. The objective of this paper is to propose a computer-aided approach for the breast cancer classification from the digital mammograms. Methods: Designing an effective classification approach will assist in resolving the difficulties in analyzing digital mammograms. The proposed work utilized the Mammogram Image Analysis Society (MIAS) database for the analysis of breast cancer. Five distinct wavelet families are used for extraction of features from the mammograms of MIAS database. These extracted features are statistical in nature and served as input to the Linear Discriminant Analysis (LDA) and Cuckoo-Search Algorithm (CSA) classifiers. Results: Error rate, Sensitivity, Specificity and Accuracy are the performance measures used and the obtained results clearly state that the CSA used as a classifier affords an accuracy of 97.5% while compared with the LDA classifier. Conclusion: The results of comparative performance analysis show that the CSA classifier outperforms the performance of LDA in terms of breast cancer classification.

Introduction
The increasing risk of breast cancer severity is more for the women in their entire lifetime. The sickness is fairly more among middle-aged than in young-aged women (Henriksen et al., 2019). Based on the information of the American Cancer Society, breast cancer is the second leading cancer next to lung cancer among women and it occurs quite rare for men (DeSantis et al., 2013). The risk of breast cancer initiates in the breast cells of the human body and these affected cells can easily invade its neighbour tissues. The illness effect may be subject to the cancer type, risk level and patients' oldness. In general, the identification of breast cancer is found either by perceiving a lump in the breast or through mammogram screening (Falk et al., 2018). This lump is classified as either benign or malignant tumours; where benign tumours are normal or controllable and malignant tumours are abnormal or vigorous in invading the adjacent tissues. The clinical physicians use microscopic study for the detection and classification of breast cancer conventionally (Documet et al., 2015). But nowadays, machine learning algorithms that utilize the computational techniques pave them an easier way for the analysis and classification of cancer.
The detection at an earlier stage is very significant in

Comparison Analysis of Linear Discriminant Analysis and Cuckoo-Search Algorithm in the Classification of Breast Cancer from Digital Mammograms
Sannasi Chakravarthy S R*, Harikumar Rajaguru the diagnosis of breast cancer. This is attained by the use of mammogram screening; where mammogram gives an x-ray imaging of the patients' breast (Sundaram et al., 2011). The sign or symptom of microcalcification in the breast is straightforwardly identified with the help of mammogram. The microcalcification is typically a tiny deposit of calcium in the breast area which is found as spots (white) on the obtained mammograms images (Peairs et al., 2017). The study herein proposes a computer-aided diagnosis for the breast cancer classification as either benign class or malignant class from digital mammograms. The digital mammogram images used for the analysis are obtained from the Mammogram Image Analysis Society (MIAS) dataset. The work involves the performance analysis by considering 80 mammogram images from the MIAS database; where the first 40 mammogram images are benign and the rest 40 are malignant. All the works in this paper are carried out using MATLAB 2013a.

A. Feature Extraction
The examination of breast cancer in this work starts with the extraction of features from the taken mammogram images in the MIAS database as in Figure 1. The feature extraction is intended to transform the input raw mammograms into a set of features that can be provided as input to any classification algorithm (Abirami et al., 2017). The feature extraction is attained using Discrete Wavelet Transform (DWT). Daubechies (DB4), Haar (HAAR), BiorSplines (BIOR4.4), Symlet8 (SYM8) and DMeyer (DMEY) are the five preferred wavelet families used for feature extraction from the raw-input mammogram images at level 4 decomposition (Abirami et al., 2017). This step will offer twelve statistical features as Mean, Median, Mode, Maximum, Minimum, Range, Standard Deviation, Median Absolute Deviation, Mean Absolute Deviation, L1 norm, L2 norm and Max norm from the input mammograms (Amin et al., 2015).
These extracted features are then served as input to the Linear Discriminant Analysis (LDA) and Cuckoo-Search Algorithm (CSA) classifiers which are discussed in the following subsection.

B. Classification using LDA
The LDA as a classifier is analogous to the Principal Component Analysis (PCA) method. The PCA method intends to determine the component axes which maximize the variance of given data. As analogous to PCA, the LDA classifier similarly determines the component axes which maximize the variance of given data and additionally it determines the axes which maximize the separation among different output classes. This could be useful in classification problems (Mika et al., 1999). The Bayes classifier gives a highly effective classification result that assigns each input features to the most likely output class given its predictor values. If the terms are properly specified, then the Bayes classifier has the minimum possible error rate than all other classifiers (Tharwat et al., 2017). Thus LDA as a classifier that efforts to approximate the Bayes classifier (Kan et al., 2015). The five general steps of LDA classifier is i. Calculate the d-dimensional mean vectors for various classes from the dataset.
ii. Calculate the scatter matrices (both within-class and between-class).
iv. Now sort the Eigen vectors by reducing order of Eigen values and select k Eigen vectors with maximum value of Eigen values to generate a d × k dimensional matrix W.
v. Use this above generated Eigen vector matrix W to renovate the samples onto the different subspace. And it can be represented shortly by matrix multiplication as Y=X×W.

C. Classification using CSA
The same set of twelve statistical features are now served as an input to the CSA classifier. The CSA is a popular nature-inspired and metaheuristic optimization technique that depends on the exciting breeding characteristics such as brood parasitism of certain species of cuckoos (Yang et al., 2010). CSA is principally imitates the breeding behaviour of cuckoo, which involves dumping of eggs inside the nests of other host birds and making these host to nurture their chicks. The traditional cuckoo-search algorithm is modified for the proposed problem. Levy flight denotes the random flight characteristics of birds and is performed to obtain the next position based on the current position as (1) where and α represent the entry-wise multiplication and step size. In general, α>0 should be taken as scaling factor for step size; here α=1 is considered for the classification problem. The random walk is observed by using levy flight where the random step size is calculated from the levy distribution (Gandomi et al., 2013) as, where the value of γ is kept as 0.2 for classification which indicates the infinite variance with infinite mean. Now the cuckoo's successive steps principally provide a random-walk approach and this follows power-law like step length distribution using a heavy tail (Cui et al., 2017). The Mantegna algorithm (Gandomi et al., 2013) is used for a symmetric Levy stable distribution.
The three general rules of cuckoo-search algorithm can be summarized as: i. Each cuckoo bird lays an egg at a time and then dumps those egg in a host nest randomly.
ii. The best host nests with supreme quality eggs will be conceded over to the subsequent generations.
iii. The amount of host nests available is constant and the cuckoo's egg is revealed by the host one with a probability . Now the host one can either dispose of the cuckoo's egg away or just remove the nest in order to construct a newer nest in another new location.
The proposed work considers the target value for benign and malignant images as 0.1 and 0.85. The LDA and CSA classifiers are evaluated using performance metrics is discussed in the succeeding section.

Results
The comparison of performance analysis of the classifiers are done using the standard metrics like Error Rate (ER), Sensitivity (SE), Specificity (SP), Precision (PR), Accuracy (ACC) and Matthews Correlation Coefficient (MCC) (Zhu et al., 2010). These metrics are calculated based on the confusion matrix (Powers, 2011) that is made of True Positives (TP), False Negatives (FN), True Negatives (TN) and False Positives (FP).
The Table 2 gives the comparative analysis of LDA and CSA classifiers used for the classification of mammograms as either benign or malignant and its graphical analysis is shown in Figure 2.
As in Table 1, the LDA classifier along with dmey wavelet misclassifies 10 malignant mammogram images as benign and 16 benign mammogram images as malignant out of 80 total mammogram images; this misclassification leads to the highest error rate of 32.5 and lowest MCC value of 35.40 as shown in Table  2 and Figure 2 when compared with all wavelets used in LDA classifier. The lowest error rate as 15 with the highest value of MCC as 70.35 for LDA classifier is obtained for the sym8 wavelet and so it misclassifies only 8 malignant mammogram images as benign and only 4 benign mammogram images as malignant. Similarly for CSA classifier, the highest misclassification or error rate with lowest MCC value is found for dB4 wavelet as in Table 2. However the haar wavelet with CSA classifier gives the lowest error rate of 2.5 and the highest MCC value of 95.12 as in Figure 2. This is because the CSA classifier using haar wavelet correctly classifies almost   Srivastava et al., (2014) Hybrid type of features with k-nearest neighbour Classifier 87 Saini et al., (2015) Texture features with artificial neural network. 87.5 Pawar et al., Wavelet coefficient features with genetic fuzzy system. 89.47 Gardezi et al., (2016) Curvelet based grey level cooccurrence matrix and geometric invariant shift transform with support vector machine classifier.

93.75
Harefa et al., Texture features with support vector machine classifier. 93.88 Pratiwi et al., (2015) Texture features with radial basis function neural network. 93.98 Gautam et al., (2018)   all 40 taken malignant mammogram images as malignant and falsely classifies only 2 benign mammogram images as malignant images. This impact will leads to the highest accuracy of 97.5% for the CSA classifier with haar wavelet and the LDA classifier with sym8 wavelet will give the highest accuracy of 85% over other wavelet families. The above discussion noticeably evident that the performance of nature-inspired CSA classifier using haar wavelet outperforms the performance of LDA classifier in classifying the mammogram images.
The comparison of classification accuracy of other related existing works (Srivastava et al., 2014;Gautam et al., 2018) with the proposed model is discussed in Table III. As in the table, the comparison is based on the methodology with different feature extraction approaches using various classification algorithms. The natureinspired based proposed classifier with haar wavelet gives higher classification accuracy than the other related systems. This is because of the simplicity and proficiency of CSA in tackling highly non-linear problems.

Discussion
The work intends to propose a system for the classification of mammogram images from MIAS data corpus into either benign class or malignant class. The work utilizes five individual wavelet families for the extraction of statistical features from the input mammograms. And these features are served as input to the LDA and CSA classifiers respectively. The smoothness nature of haar wavelet along with CSA classifier provides an improved accuracy of 12.5% when compared with LDA classifier. The future work of the study is to implement a dynamic class-label classification for the identification of breast cancer with some other datasets.


Analysis of the unambiguous state discrimination with unequal a priori probabilities

In this paper, we study unambiguous state discrimination regarding advanced attack on phase-coded quantum key distribution protocol. We propose the method of optimal unambiguous state discrimination probability derivation as a function of a priori probabilities for signal states. The expression obtained as an example in case of two signal states explicitly demonstrates the additional term dependent on small deviations from equal a priori probabilities that may take place in real quantum key distribution implementations. Precise estimation of optimal unambiguous state discrimination probability is significant for complete evaluation of quantum key distribution security.

Introduction
Throughout the last few decades the field of quantum cryptography has been rapidly developing and advancing. It has emerged with the first papers [1,2] dedicated to the descriptions of protocols which allow secure distribution of a finite bit sequence between legitimate partners, and it is still in the focus of research groups, for instance [3][4][5]. Not only the protocols are of special interest, but also different types of attacks are studied in order to find successful countermeasures.
In this work we would like to concentrate on zero-error unambiguous state discrimination (USD) attack that is a considerable threat for protocols utilizing weak coherent states. USD attack requires eavesdropper (Eve) tapping into quantum channel of legitimate parties (Alice and Bob), errorlessly measuring the states and resending the modified states to Bob in order to preserve detection statistics [6]. We explore the phase-coded protocol which utilizes several pairs weak coherent states sent with unequal a priori probability. Imperfect state preparation that can result in the sending probabilities' inequality is the immanent part of every practical set-up. For example, quantum random number generator may cause unequal probability of state preparation [7][8][9]. Hence we examine the influence of unequal a priori probability on the discrimination probability.
USD measurement is subject of research for almost three decades. General approach to discrimination between linearly independent states was introduced in [10,12,13]. The solution for minimum achievable probability of inconclusive outcome for three states was given by [11]. The method of minimizing the probability for N symmetric states was considered in [12]. The special case of equal a priori probabilities for N states was discussed in [13]. Bounds of unambiguous state discrimination probabilities have been studied for the case of N linearly independent states in [14][15][16]. Several approaches to numerical optimization were proposed as well in [16,17]. Implementations of USD in field of quantum computations also take place, e.g. for purpose of quantum cloning operation [18] or USD between oracle operators [19].

Method description
To perform unambiguous discrimination of the N signal states |f i Eve determines special positive-operator valued measure (POVM). It consists of projection operatorsÂ i which are related to probabilities of successful state discrimination P i (for each state) and operatorÂ 0 that is related to obtaining inconclusive result which is always present due to the nonorthogonality of the states and introduced in order to make the sum of the projection operators satisfy the decomposition of the identity: Extracted from (1) the operatorÂ 0 is expressed as:Â and according to [11]Â 0 is subject to condition: The latter provides maximal allowed values for probabilities P i . We specify operatorsÂ i as follows: where |v i is state that forms biorthogonal basis with the signal states Thus one needs to optimize the average probability of USD: where p i is a priori probability of sending each state. One may use Lagrange multiplier method in order to do so. Hence, the function to be optimized is P from expression (5) and the following expression is bound [11]: Let us introduce orthogonal basis |u i obtained by, for instance, Gram-Schmidt process. For simplicity let us denote matrix of the operatorÂ 0 (inconclusive result) as A in this orthonormal basis, and operators |v k v k | as V (k) respectively.
Thus system of equations is as follows: where λ is Lagrange multiplier. Insofar as where, Tr(X) = i X ii is trace of arbitrary matrix X, adj(A) is adjoint matrix of matrix A, I is identity matrix, hence p n + λ Tr adj(A)V (n) = 0, det A = 0.
Since sum of p i is equal to unit than Lagrange multiplier is as follows: and consequently p i is expressed as A. A. Gaidash, S. S. Medvedeva, G. P. Miroshnichenko

Example
As an example let us consider two signal states. Their overlapping is denoted as B. Thus signal states can be described in terms of orthonormal basis (obtained by Gram-Schmidt process) as follows: Considering condition (6) may be derived in the following form: thereby the probability P 1 can be denoted as Lagrange multiplier is as follows: and probabilities p 1 and p 2 are as By substituting expression (15) in expression (17) we find and consequently: Taking into account symmetry of expressions (15), (17), (18) with respect to P 1 and P 2 following expression for P 1 is derived analogously: By substituting expressions (20) and (21) in (5) we obtain optimized USD probability: This result has well-known form [11] if p = p 1 = p 2 = 1 2 . Defining p 1 = 1 2 + ∆p and p 2 = 1 2 − ∆p, where ∆p is considerably small deviation from equal a priori probabilities, we get Therefore, the value of P has quadratic term dependent on small deviations from equal a priori probabilities.

Discussion and conclusion
In this work we analyze the probability of unambiguous discrimination for arbitrary number of states with unequal a priori probabilities. The proposed method provides system of equations (expressions (11) and (3)) that can be solved in order to find optimized USD probability as function of a priori probabilities. We consider rather simple and wellstudied example for two states; the result is the same as in [20]. However, the authors of that paper obtain result only for two states. Concerning the method described in this paper it is unfortunate that the amount of calculations for higher number of states grows rapidly so it might be rather difficult to obtain analytical expressions similar to expression (22).
The results are important mostly in the field of quantum key distribution. In order to achieve certain level of security, one should consider various attacks, estimate probabilities of their success, and apply corresponding countermeasures to them. Thus estimation of optimal USD probability is crucial for bounding Eve's information during, for instance, advanced USD attack [6]. However consideration of an ideal case is not enough in this instance since there is an additional quadratic term related to slightly unequal (e.g. due to the bias in quantum random number generator) a priori probabilities in expression (23) that might provide additional information to Eve. Precise estimation of optimal USD probability is a significant step towards complete evaluation of quantum key distribution security.


extfieldsofstudy:
Medicine
Computer Science

text:
Non-Technical Loss Detection in Power Grids with Statistical Profile Images Based on Semi-Supervised Learning

In order to keep track of the operational state of power grids, the world’s largest sensor system, smart grid, was built by deploying hundreds of millions of smart meters. Such a system makes it possible to discover and make quick response to any hidden threat to the entire power grid. Non-technical losses (NTLs) have always been a major concern for their consequent security risks as well as immeasurable revenue loss. However, various causes of NTL may have different characteristics reflected in the data. Accurately capturing these anomalies faced with such a large scale of collected data records is rather tricky as a result. In this paper, we proposed a new methodology of detecting abnormal electricity consumptions. We did a transformation of the collected time-series data which turns it into an image representation that could well reflect users’ relatively long term consumption behaviors. Inspired by the excellent neural network architecture used for objective detection in computer vision, we designed our deep learning model that takes the transformed images as input and yields joint features inferred from the multiple aspects the input provides. Considering the limited amount of labeled samples, especially the abnormal ones, we used our model in a semi-supervised fashion that was brought about in recent years. The model is tested on samples which are verified by on-field inspections and our method showed significant improvement for NTL detection compared with the state-of-the-art methods.

Introduction
By deploying a large number of sensor devices, the smart grid becomes the largest sensor network in the world. Particularly, China installed nearly 500 million smart meters (SM) by the end of 2017. The smart meters have greatly improved the information and automation level of the power grid, and made it possible to identify anomalies. Anomalies, which include meter failure and electricity theft, are the primary source of non-technical losses (NTL) in the power grid. The NTL cause significant revenue losses; $58.7 billion dollars are lost each year due to NTL [1]. Further, it also affects the power system operation because of the uncertainty of the real consumption [2]. Due to the huge scale of the collected data and the complicated real-world operational environment, the NTL detecting methods required are excellent in terms of both efficiency and reliability. However, the existing expert system based approach sometimes is unreliable as it fails to comprehensively exploit the underlying correlations among different features. Or the other way round, in order to cover extremely diverse forms of abnormal behaviors, far more field knowledge should be integrated, and sometimes human efforts are necessary. As a result, efficiency suffers.
In this paper, we introduced deep learning to help with the detection of NTL as it has already demonstrated its great potential and capacity in a variety of fields. Neural networks' hierarchical structures are designed to automatically extract high level features containing more abstract semantic information from a large scale of input data. So, in this problem setting where the prominent features are latent and intertwined with a lot of factors, deep neural networks are just fit for the job in terms of both efficiency and accuracy. Yet, there are some problems we have to solve: (a) The quality of collected data is relatively low because of data noise and missing data. As is often the case in real-world scenarios, the data records may suffer from various types of noises. Generally, two types of noises are distinguished: feature (or attribute) noise and class noise. Class noise means the labels assigned to samples are not totally correct, which is likely to happen as the ground truth labels are given based on the result of on-field inspections in this problem.
In this paper we only focus on the feature noises. Generally speaking, this type of noise can be summarized as the noise that interferes the mapping from the input x to the output label y, including slight value disturbance, erroneous data records as well as data out of sync. Missing data is also a common phenomenon in large-scale sensing systems. Situations get worse when unavailable data points emerge in high density or even large chunks are missing. (b) The correlation between labels given by on-field inspection and the original data records is not intuitional. For those verified customers, we only have general labels denoting whether anomalies ever occurred in a customers' history records. There is no further information that indicates when and how faults or fraudulent events occurred. So we have to find distinct patterns of positive/negative samples for the classifier/detector. It is better if we can make the data records more human friendly and distinguishable for ordinary people without the aid of carefully designed criteria. (c) The proportion of labeled data is rather small compared to that of the unlabeled. And the abnormal samples are obviously fewer than normal ones. This fact makes it impossible for us to treat the problem as a simple supervised classification problem.
In this paper, we first proposed a data transformation method. By analyzing the abnormal cases, we found that statistical characteristics of the data can better reflect the consumption behaviors of the electricity users. The method we designed focuses on the feature patterns in a relatively longer term by utilizing the data in a certain time range. This transformation makes it easier to get features that can distinguish between normal and abnormal samples. Moreover, the transformed data form naturally has better noise resistance performance and it can eliminate the difficulties resulting from different value ranges of different users. We also adopt a semi-supervised model [3] brought about in recent years. This model does not depend on any presupposed prior distribution of the training data in class space and proved itself a reliable one by testing it on the real customers' data.
There is an abundance of previous work about NTL detection or anomaly (fraud) detection in electricity industry. The approaches can generally be categorized in classification based and non-classification based solutions. The non-classification based approaches usually use methods like clustering, statistical analysis, and so on. Ford et al. [4] and Cody et al. [5] forecasted the customers' energy consumption. If the difference between actual and forecasted consumption exceeds the limit imposed by the authors, the customer is considered to be committing fraud. Similar ideas are adopted in [6,7], where researchers first construct the distribution of the historical data collected by meters. Then a similarity measurement is introduced to find outliers, which are considered fraud or anomaly. As far as the classification based approaches are concerned, Nagi et al. [8] used extracted features from historical customer consumption data and trained a support vector machine (SVM) model with them to detect NTL. Apart from the sole consumption measurements, P. Glauner et al. [9] also utilized the customers' geographical location to compute the inspection rate and the NTL rate in its neighborhood. And other models besides support vector machine (SVM), such as logistic regression (LR), k-nearest neighbor (kNN), and random forest (RF) are trained to testify the methodology. As for the classifier, there also exists works using neural networks (NN) as in [10,11]. However, reference [11] did a comparison among different algorithms and found out the single gradient boosted machine (GBM) outperformed any ensemble or any other classifiers.
Our main contributions to these problems are as follows: • We designed a novel way of visualizing customers' data records by utilizing more potentially useful real-time electrical information than sole energy consumption measurements. The new data format statistically constructs a visual representation of customers' behavior in a certain time range and provides a new aspect of viewing their operational states. Such transformation is also tolerant of slight data error or missing, which is quite common for the sensor systems in industrial areas. The convolutional based neural network that is universally used in computer vision is then introduced to extract visual patterns for further classification.

•
The inefficient on-field inspection limited the ground truth we can make use of for supervised training. This work combined a semi-supervised learning method based on consistency loss with our proposed network architecture to deal with the common situations in real world scenarios where the training samples are partially labeled. It turns out the consequent overfitting problem can be addressed pretty well.

Data Transformation and Network Design
The major goal of this paper is to detect abnormal energy consumption for a certain customer from SM data, whatever it is caused by metering failure or fraudulent usage. The data records are from an electrical information collection system constructed by China Southern Power Grid Co., Ltd. (CSG).
We first extracted features from the electric parameters. Then we did a data transformation and formed training samples based on those features. An inference network was designed afterwards and was trained in a semi-supervised fashion considering the fact that the Figure 1   Electric parameters are collected at higher frequencies than energy consumption. It not only contains more information about consumption behavior, but also allows us to detect and respond to NTL with lower delay.
The abnormal consumption that really matters and would cause NTL is the kind that last for a period of time. To be specific, anomalies that take on the forms of impulse signals, sudden changes, or transient data missing, for example, would not make any significant differences to the overall energy consumption (EC) of a customer, and neither would they cause great NTLs. It is a better choice to take them as the consequences of communication failure or temporal metering devices' malfunctioning and treat the data records from a statistical point of view. Figure 2 presents the three-phase voltage records from two customers, normal customer A and abnormal customer B. These two customers are verified by on-field inspections. From the time series in Figure 2a we can see that for the normal customer A (upper), apart from several sudden drops in the single-phase voltage, the voltage values for three phases are highly identical for the rest of the time. However, for abnormal customer B, it is a consistent and almost periodic phenomenon that Figure 1. The power triangle in the AC circuit. S is the complex power and the magnitude of S is called apparent power. Active power P is represented as the real axis while reactive power is represented as the imaginary axis of the vector diagram. ϕ is the angle of difference (in degrees) between current and voltage.

Statistical Profile
In an alternating current (AC) circuit, electric parameters like voltage, current, etc., are usually denoted in phasor form. The definition of power is more complex than that in the direct current circuit.
S is the complex power and its magnitude is the apparent power. V denotes voltage in phasor form and I denotes current in phasor form. And the complex conjugate of I is denoted I * . ϕ is the angle of difference (in degrees) between current and voltage. There are also active power and Sensors 2020, 20, 236 4 of 19 reactive power, whose explanations and correlations with complex power are explained in Table 1 and Figure 1 respectively. In a power grid, each customer is generally assigned a specific transformer. As the voltage on the supply side is fixed, the voltage on the customer side should be certain and it is called rated voltage. Besides, there are maximum power limits for customers that should not be exceeded. The limit is contracted power (CP) and varies from customer to customer.
Electric parameters are collected at higher frequencies than energy consumption. It not only contains more information about consumption behavior, but also allows us to detect and respond to NTL with lower delay.
The abnormal consumption that really matters and would cause NTL is the kind that last for a period of time. To be specific, anomalies that take on the forms of impulse signals, sudden changes, or transient data missing, for example, would not make any significant differences to the overall energy consumption (EC) of a customer, and neither would they cause great NTLs. It is a better choice to take them as the consequences of communication failure or temporal metering devices' malfunctioning and treat the data records from a statistical point of view. Figure 2 presents the three-phase voltage records from two customers, normal customer A and abnormal customer B. These two customers are verified by on-field inspections. From the time series in Figure 2a we can see that for the normal customer A (upper), apart from several sudden drops in the single-phase voltage, the voltage values for three phases are highly identical for the rest of the time. However, for abnormal customer B, it is a consistent and almost periodic phenomenon that voltage on phase c differs significantly from the rest of the phases. From the scatter plot of normal customer A in Figure 2b, the several outliers correspond to those impulses in Figure 2a and they behave more like noises in this case. This justifies what was discussed above: focusing on the long term consumption behaviors would help us draw safer conclusions in this NTL detection problem. It is not appropriate to just take each row of data records as a training sample. This is why we proposed our data transformation method. Further experimental comparison between single-collecting-point samples and our statistics-based data representation will be shown in a later section.
Here, we did not aim to get a precise mathematical representation of the feature distributions. Rather, we designed a data transformation inspired by kernel density estimation (KDE), a non-parametric distribution estimation method. In this work, we did a 2-dimensional KDE transformation of the features obtained in previous section.
The process can be briefly thought of as replacing each discrete point in a scatterplot with a kernel distribution. The idea of this estimation method is that the positions where the points appear have the higher occurrence probabilities. So, for the Gaussian distribution we use as basic kernels, the peaks are where the points are located. The advantage of the distribution profile over a scatter plot is that it gives more intuitive impression of customers' behavior patterns during a period of time. With the extension of time used for statistical analysis, the estimated distribution becomes more plausible and closer to the scatter plot.
In this work, we used 2-D KDE to reflect the correlation and variation tendency in the space constructed by two feature indices. As the estimation method is density based, outliers that are rather sparse would have small pixel values thus contributing little to the visual patterns. Figure 3a shows a scatter plot and 2-D KDE profile of the same piece of voltage records data for the afore-mentioned normal customer A. Darker areas in the estimated profile image represents locations where points emerge in high densities and outliers in scatter plot on the left side can hardly be observed in the estimated profile. Technically, the time span needed to generate distribution profile images can vary from sample to sample. If presented with more detailed event annotations as ground truth, we can generate more samples with different time durations as a kind of data augmentation. However, in this situation where specific event logs are not available, it is hard to implement such data augmentations and a fixed time duration of 10 calendar days is adopted throughout this work. energy consumption (EC) of a customer, and neither would they cause great NTLs. It is a better choice to take them as the consequences of communication failure or temporal metering devices' malfunctioning and treat the data records from a statistical point of view. Figure 2 presents the three-phase voltage records from two customers, normal customer A and abnormal customer B. These two customers are verified by on-field inspections. From the time series in Figure 2a we can see that for the normal customer A (upper), apart from several sudden drops in the single-phase voltage, the voltage values for three phases are highly identical for the rest of the time. However, for abnormal customer B, it is a consistent and almost periodic phenomenon that voltage on phase c differs significantly from the rest of the phases. From the scatter plot of normal customer A in Figure 2b, the several outliers correspond to those impulses in Figure 2a and they behave more like noises in this case. This justifies what was discussed above: focusing on the long term consumption behaviors would help us draw safer conclusions in this NTL detection problem. It is not appropriate to just take each row of data records as a training sample. This is why we proposed our data transformation method. Further experimental comparison between singlecollecting-point samples and our statistics-based data representation will be shown in a later section.  Here, we did not aim to get a precise mathematical representation of the feature distributions. Rather, we designed a data transformation inspired by kernel density estimation (KDE), a nonparametric distribution estimation method. In this work, we did a 2-dimensional KDE transformation of the features obtained in previous section.
The process can be briefly thought of as replacing each discrete point in a scatterplot with a kernel distribution. The idea of this estimation method is that the positions where the points appear have the higher occurrence probabilities. So, for the Gaussian distribution we use as basic kernels, the peaks are where the points are located. The advantage of the distribution profile over a scatter plot is that it gives more intuitive impression of customers' behavior patterns during a period of time.
With the extension of time used for statistical analysis, the estimated distribution becomes more plausible and closer to the scatter plot.
In this work, we used 2-D KDE to reflect the correlation and variation tendency in the space constructed by two feature indices. As the estimation method is density based, outliers that are rather sparse would have small pixel values thus contributing little to the visual patterns. Figure 3a shows a scatter plot and 2-D KDE profile of the same piece of voltage records data for the afore-mentioned normal customer A. Darker areas in the estimated profile image represents locations where points emerge in high densities and outliers in scatter plot on the left side can hardly be observed in the estimated profile. Technically, the time span needed to generate distribution profile images can vary from sample to sample. If presented with more detailed event annotations as ground truth, we can generate more samples with different time durations as a kind of data augmentation. However, in this situation where specific event logs are not available, it is hard to implement such data augmentations and a fixed time duration of 10 calendar days is adopted throughout this work.

Feature Engineering
The AC voltages and currents are three-phase data. It is fair to treat each of the three phases equivalently because here we only aim to decide whether anomalies/NTL occurred without having to know specifically which phase was going wrong. With regard to the voltage and current, the following features were extracted: Deviation from rated voltage where V r is the rated voltage and V i u is the i-th phase voltage. As the voltage should remain steady for a certain customer, the maximum voltage deviation among the three phases from the above formulas can indicate the degree of how abnormal a customer is.
Unbalance degree of voltage and current on three phases where S can either be voltage or current. Unlike the voltage, there exists no such standard value for the current. However, the discrepancy of voltage or current values in three different phases should not be too large if the system is running as expected. After all, any significant value decrease in any phases would directly result in the drop of EC. Formula (5) well represents the difference of voltage or current in three phases.
Active Power. Although there are many different types of customers and their real-time active power collected once an hour may vary greatly, we can use the contracted power to normalize it to make the power feature comparable among customers.
Power factor. This feature is customer-invariant and has the value range from 0 to 1. Load rate. It indicates the percentage that the power accounts for of the maximum power volume. The formula is more like an approximation of the apparent powers normalized by the contracted power. In most of the 2-dimensional distribution profile image channels, we used the load rate as the reference axis x.

Generation of Super Images
We used a Gaussian kernel with fixed standard deviation σ and formed images as shown in Figure 3. Notably, with all the features mentioned above, we can use combinations of them to get short time distribution profiles of these feature pairs. Seven grey-scale maps are generated altogether as described in Figure 4. With each of these maps constituting one channel, a super image with seven channels is obtained and this is the input for the detecting neural network. Multiple samples can be generated from a customer by having a sliding window of fixed size move along the time axis of a customer's data records. The comparison between samples from statistical profile and per-row record will be shown in later section in terms of separability and final detection performance.
Before feeding the super image to the network, we have to search the bounding boxes of patterns in each image channel. Position information obtained in this stage serves as auxiliary input for the network to highlight the position of the pattern objectives. We achieved this goal by setting a pixel threshold, and pixel values higher than the threshold are included in the bounding box. The choice of threshold value is listed in Appendix B.  Blue lines indicate the columns of features used as x-axis data while black lines indicate the column used as y-axis data. The rectangular box in red is the sliding window that covers a certain time range of data. Here we have another feature named calculated power factor. This is the power factor we calculated with the power and load rate values.

Framework
The on-field inspection requires great deal of human efforts, so the number of verified customers is quite limited. Semi-supervised learning distinguishes itself by utilizing both the labeled and unlabeled data. There are different fashions of implementing the goal: self-training [12], co-training [13], transductive SVM [14], graph based methods [15], and generative models [16]. With the development of deep neural networks, novel models such as generative adversarial nets (GAN) [17] and tons of its variant versions emerged. The idea of GAN is brilliant and GANs showed their superiority in various tasks. However, GANs usually require prior knowledge about the label distribution of the unlabeled samples. There is also other previous research that made significant progress in this branch of semi-supervised learning, like Virtual Adversarial Training (VAT) [18,19] and ladder network [20]. These methods either explicitly or implicitly depend on prior knowledge about the distribution of labels. As we have no idea about the label distribution in unlabeled data, we have to utilize semi-supervised learning in a way that is independent of such prior knowledge. In this work, we used consistency regularization based techniques [3,[21][22][23] to train our model. Throughout the paper, the ensemble strategy we adopt is from the mean teacher model [3].
In most cases, overfitting results from the scarcity of labeled training samples. What semisupervised learning does is utilize unlabeled samples to prevent overfitting. If there exists an oracle that can give the labels of unlabeled samples, we can introduce a regularization term to the optimization objective to constrain the model and make the model give identical predictions with the oracle. In the mean teacher model, such an oracle is called teacher model in this case. It is obtained by ensembling the student model. The prediction it yields is called a pseudo label. Although the pseudo labels are not perfectly correct, as training goes on, the student model becomes better and better and the pseudo labels are more and more reliable as a result. It is much like a student that would learn well with the right instructions from the teacher. In turn, a good student would make the teacher improve. Gradually, both the student and teacher would become excellent in the end. Formula (7) gives the mathematical forms of how ensembling is implemented. And Figure 5 is the overall framework of the model. Blue lines indicate the columns of features used as x-axis data while black lines indicate the column used as y-axis data. The rectangular box in red is the sliding window that covers a certain time range of data. Here we have another feature named calculated power factor. This is the power factor we calculated with the power and load rate values.

Framework
The on-field inspection requires great deal of human efforts, so the number of verified customers is quite limited. Semi-supervised learning distinguishes itself by utilizing both the labeled and unlabeled data. There are different fashions of implementing the goal: self-training [12], co-training [13], transductive SVM [14], graph based methods [15], and generative models [16]. With the development of deep neural networks, novel models such as generative adversarial nets (GAN) [17] and tons of its variant versions emerged. The idea of GAN is brilliant and GANs showed their superiority in various tasks. However, GANs usually require prior knowledge about the label distribution of the unlabeled samples. There is also other previous research that made significant progress in this branch of semi-supervised learning, like Virtual Adversarial Training (VAT) [18,19] and ladder network [20]. These methods either explicitly or implicitly depend on prior knowledge about the distribution of labels. As we have no idea about the label distribution in unlabeled data, we have to utilize semi-supervised learning in a way that is independent of such prior knowledge. In this work, we used consistency regularization based techniques [3,[21][22][23] to train our model. Throughout the paper, the ensemble strategy we adopt is from the mean teacher model [3].
In most cases, overfitting results from the scarcity of labeled training samples. What semi-supervised learning does is utilize unlabeled samples to prevent overfitting. If there exists an oracle that can give the labels of unlabeled samples, we can introduce a regularization term to the optimization objective to constrain the model and make the model give identical predictions with the oracle. In the mean teacher model, such an oracle is called teacher model in this case. It is obtained by ensembling the student model. The prediction it yields is called a pseudo label. Although the pseudo labels are not perfectly correct, as training goes on, the student model becomes better and better and the pseudo labels are more and more reliable as a result. It is much like a student that would learn well with the right instructions from the teacher. In turn, a good student would make the teacher improve. Gradually, both the student and teacher would become excellent in the end. Formula (7) gives the mathematical forms of how ensembling is implemented. And Figure 5 is the overall framework of the model.

Inference Network
We get the distribution of features to expose their visual patterns for the model to capture. For this job, convolutional based neural network has already proved its superiority and has achieved remarkable results in various image recognition and computer vision tasks [24]. Also, there exists previous work that encoded time series as images to achieve improvement encoding/decoding problems [25]. For visual object detection problems, there are a series of famous works about twostage object detection [26][27][28]. The regions where objects lie in an image are proposed by bounding boxes for further classification. Our task shares the same purpose of highlighting the desired consumption pattern, so these previous works serve as great examples of how to classify objects that occupy only part of the entire image. Figure 6 gives a detailed example of how we can decide a sample is abnormal or not. The structure of our inference network is illustrated in Figure 7.

Inference Network
We get the distribution of features to expose their visual patterns for the model to capture. For this job, convolutional based neural network has already proved its superiority and has achieved remarkable results in various image recognition and computer vision tasks [24]. Also, there exists previous work that encoded time series as images to achieve improvement encoding/decoding problems [25]. For visual object detection problems, there are a series of famous works about two-stage object detection [26][27][28]. The regions where objects lie in an image are proposed by bounding boxes for further classification. Our task shares the same purpose of highlighting the desired consumption pattern, so these previous works serve as great examples of how to classify objects that occupy only part of the entire image. Figure 6 gives a detailed example of how we can decide a sample is abnormal or not. The structure of our inference network is illustrated in Figure 7. Parameter values of student model are updated at the end of each training iteration and those of the teacher model are obtained by ensembling the student model. What triplet loss does is minimize the feature distances between input with the same labels while maximizing the feature distances of input with different labels. For the unlabeled samples, labels needed for computing triplet loss are from the teacher model's predictions.

Inference Network
We get the distribution of features to expose their visual patterns for the model to capture. For this job, convolutional based neural network has already proved its superiority and has achieved remarkable results in various image recognition and computer vision tasks [24]. Also, there exists previous work that encoded time series as images to achieve improvement encoding/decoding problems [25]. For visual object detection problems, there are a series of famous works about twostage object detection [26][27][28]. The regions where objects lie in an image are proposed by bounding boxes for further classification. Our task shares the same purpose of highlighting the desired consumption pattern, so these previous works serve as great examples of how to classify objects that occupy only part of the entire image. Figure 6 gives a detailed example of how we can decide a sample is abnormal or not. The structure of our inference network is illustrated in Figure 7. Normally, the unbalance degree decreases as the load rate grows as shown in the left in (a). As the current on three phases grows, small disturbance would not have much effect on it. So the unbalance degree would turn low and steady. The right figure contradicts this fact. In (b), the power factor is 1 for the most of the time, but the power data is nearly zero. This is a typical sign of customers conducting theft. for the most of the time, but the power data is nearly zero. This is a typical sign of customers conducting theft.   Figure 7 is the overall structure of our inference network. Although each statistical profile is treated as a channel of a super image, the information each of these channels carries, such as pattern shapes and the regions patterns lie in, are totally different and should be processed separately. Every channel in a super image goes through a shared network whose internal structure is shown in Figure  6 (right). The internal structure of the shared network is much like that in [27] except that we search the region of interest here by setting a pixel value threshold to get a rectangular area where the pattern lies. So there is only one bounding box for an input channel. After obtaining every channel feature, we concatenate them together to get a final feature representation of an input super image. The convolutional based network denoted as 'ConvNet' used the same set of parameters to extract graphic information of all the image channels and its detailed architecture is listed in Appendix A Table A1.

Ensembling Method and Consistency Loss
Formally, let denote the weight parameters in the neural network and denote the parameters at training step t. The ensembling strategy is as follow: Then, consistency cost is introduced to minimize the prediction difference between the teacher and student models: where, is the noise applied to input image. Apart from the consistency cost, we introduced another loss item, triplet loss [29]. This is originally proposed for face recognition tasks. However, the goal of face verification is more demanding than general classification problems and what triplet loss does is to constrain points of same classes to have consistent embeddings while pushing points of different classes far away in the embedding space. Let , be two input samples with index i and j. Let y and h denote the label and extracted features respectively. Equation (8) is an explanatory formula of how triplet loss works:  Figure 7 is the overall structure of our inference network. Although each statistical profile is treated as a channel of a super image, the information each of these channels carries, such as pattern shapes and the regions patterns lie in, are totally different and should be processed separately. Every channel in a super image goes through a shared network whose internal structure is shown in Figure 6 (right). The internal structure of the shared network is much like that in [27] except that we search the region of interest here by setting a pixel value threshold to get a rectangular area where the pattern lies. So there is only one bounding box for an input channel. After obtaining every channel feature, we concatenate them together to get a final feature representation of an input super image. The convolutional based network denoted as 'ConvNet' used the same set of parameters to extract graphic information of all the image channels and its detailed architecture is listed in Appendix A Table A1.

Ensembling Method and Consistency Loss
Formally, let θ denote the weight parameters in the neural network and θ t denote the parameters at training step t. The ensembling strategy is as follow: Then, consistency cost is introduced to minimize the prediction difference between the teacher and student models: where, η is the noise applied to input image. Apart from the consistency cost, we introduced another loss item, triplet loss [29]. This is originally proposed for face recognition tasks. However, the goal of face verification is more demanding than general classification problems and what triplet loss does is to constrain points of same classes to have consistent embeddings while pushing points of different classes far away in the embedding space. Let x i , x j be two input samples with index i and j. Let y and h denote the label and extracted features respectively. Equation (8) is an explanatory formula of how triplet loss works: Additionally, the labels in the formula are not all ground truthed in the labeled data set and the triplet loss is applied for all the input in a training batch. For those input without labels, we use the predictions given by the teacher model as pseudo labels.

Training Algorithm
With the framework and the loss items introduced, the pseudo code of our training algorithm is presented in Algorithm 1. Notably, the coordinates of bottom left and top right vertices of bounding boxes are sent to the network as auxiliary input. The Region of Interest (RoI) pooling operation is executed on the feature maps and the coordinates of bounding boxes are linearly projected on the feature map. As for our strategy of drawing mini-batches, we separately draw samples from labeled data set and unlabeled data set by random while maintaining the ratio of 1:3 between labeled and unlabeled samples.

Algorithm 1. Training algorithm.
Require: x i = input images Require: L = set of training input indices with known labels Require: y i = labels for labeled inputs, i ∈ L Require: bb i = bounding box of sample i Require: W_u = unsupervised loss weight Require: f θ (x) = neural network with trainable parameters θ as student model Require: f θ (x) = neural network as teacher model whose parameter θ with initial value θ Require: α = moving average momentum for parameters Require: η = random Gaussian noise added to the input for t in [1, num_iterations] do draw a mini-batch B from labeled and unlabeled samples randomly f i ← f θ (x i∈B , bb i∈B , η) evaluate network outputs f i ← f θ (x i∈B , bb i∈B , η ) evaluate network outputs Find triplets T of components <i, j, k> in B where y i = y j and y i y j ; y i is the pseudo label given by update θ using ADAM optimizer update θ by θ t = αθ t−1 + (1 − α)θ t end for return θ

Experiment Settings and Metrics
The electric parameters used in our experiment are from real-world SM records. We use the data from a power company in China to train and validate our model. The detailed information of our dataset is shown in Table 2. There are 193 verified and 2929 unverified customers in total, among which 54 of them are labeled as abnormal customers by on-field inspection, whatever the causes are, and 139 of them are labeled as normal ones. As is introduced in Section 2, data records of a customer can be transformed into numbers of samples as the sliding window moves along the time axis. Here we adopt the sliding window of a 10-day time range regardless of the features' sampling frequency, and the overlap between windows is five days, that is to say, each image channel of a sample consists of 240 points if the corresponding feature is collected hourly. As a result, it would make 8797 labeled training samples and more than 130,000 unlabeled samples in total. However, in order to significantly cut down time for training, we randomly selected 50,000 samples from part of the unlabeled users.
In order to justify the generalization ability of our method, we split the verified customers randomly while maintaining the ratio between the number of customers in training and validation sets, which is approximately 1:3. This process is repeated for three times. More experiments settings are listed in Appendix B. For the detection problem where the negative samples are far more than positive ones, overall precision is not a suitable criterion to assess the performance of our algorithm. However, the precision and recall of the positive samples alone would give a direct impression of the detection results.
As is in most binary classification problems where the output is a two-dimensional vector that gives the corresponding probability of each class the input belongs to, we take the hypothesis with higher probability as the prediction. In other words, the decision threshold is 0.5. When the threshold varies, the accuracy and the sensitivity, true positive rate (TPR) and false positive rate (FPR) of the classifier also change. Generally speaking, classifiers with high TPR and low FPR are excellent for classification problems.

Separability
To justify our method of transforming the data, we visualized linear separability of the training data using a visualization technique called t-distributed stochastic neighbor embedding (t-SNE) [30]. It non-linearly projects features from high-dimensional space to lower-dimensional space while trying to maintain their original relative positions. As shown in Figure 8a is the 2-D projection of the 6-D feature vector (load rate, voltage deviation, voltage unbalance degree, current unbalance degree, power factor, and power) and each point represents the features at one sampling time stamp in the validation set. In Figure 8b, we use the same features but take a longer time range of 10 days to form each point in the figure. Points in Figure 8c are our proposed seven-channel 'super images' generated from a 10-day time range of features.
Obviously, our transformation method makes the classification problem more feasible as the result in Figure 8a shows that it is nearly impossible to separate samples of different labels. This is an indication that viewing the data in the longer term is a better solution to address the detection problem.
From Figure 9 we can see that as samples go through the inference network, points of the sample labels gradually gather together, especially for the abnormal samples which are plotted in orange. This is further evidence that our data transformation, together with such network structure, works well for our classification goal.  indication that viewing the data in the longer term is a better solution to address the detection problem.
From Figure 9 we can see that as samples go through the inference network, points of the sample labels gradually gather together, especially for the abnormal samples which are plotted in orange. This is further evidence that our data transformation, together with such network structure, works well for our classification goal.

Detection Results
Verified customers were randomly split into a training set and validation set for three separate times and the training and validation process is carried out based on the data. Detection accuracy was not suitable to assess our algorithm performance because in this situation where negative samples are in absolute majority, the overall accuracy would still seem rather high even if our algorithm fails to find out any positive samples. As a result, receiver operation characteristic (ROC) and area under curve (AUC) [31] score as well as precision-recall (PR) curves are used as assessment criteria.
Similar to any other anomaly detection scenarios, our data set is quite imbalanced where most of the data is negative/normal in both training and validation set. So apart from overall performance, we are particularly interested in our algorithm's performance on positive/abnormal samples. Figure 10 and Table 3 give the results from a single run of each train/validation splits. The mean and standard deviations report of each validation set is available in Appendix C.
(a) (b) Figure 9. Linear separability of samples in validation set changes as samples go through the inference network.

Detection Results
Verified customers were randomly split into a training set and validation set for three separate times and the training and validation process is carried out based on the data. Detection accuracy was not suitable to assess our algorithm performance because in this situation where negative samples are in absolute majority, the overall accuracy would still seem rather high even if our algorithm fails to find out any positive samples. As a result, receiver operation characteristic (ROC) and area under curve (AUC) [31] score as well as precision-recall (PR) curves are used as assessment criteria.
Similar to any other anomaly detection scenarios, our data set is quite imbalanced where most of the data is negative/normal in both training and validation set. So apart from overall performance, we are particularly interested in our algorithm's performance on positive/abnormal samples. Figure 10 and Table 3 give the results from a single run of each train/validation splits. The mean and standard deviations report of each validation set is available in Appendix C. problem.
From Figure 9 we can see that as samples go through the inference network, points of the sample labels gradually gather together, especially for the abnormal samples which are plotted in orange. This is further evidence that our data transformation, together with such network structure, works well for our classification goal.

Detection Results
Verified customers were randomly split into a training set and validation set for three separate times and the training and validation process is carried out based on the data. Detection accuracy was not suitable to assess our algorithm performance because in this situation where negative samples are in absolute majority, the overall accuracy would still seem rather high even if our algorithm fails to find out any positive samples. As a result, receiver operation characteristic (ROC) and area under curve (AUC) [31] score as well as precision-recall (PR) curves are used as assessment criteria.
Similar to any other anomaly detection scenarios, our data set is quite imbalanced where most of the data is negative/normal in both training and validation set. So apart from overall performance, we are particularly interested in our algorithm's performance on positive/abnormal samples. Figure 10 and Table 3 give the results from a single run of each train/validation splits. The mean and standard deviations report of each validation set is available in Appendix C.  For our NTL detection problem, the characteristics of data records from so many customers are very different because the variety of types of customers. This fact can be directly reflected in the detection results shown in Figure 10. For different train/validation splits, differences in terms of ROC curves, AUC score or precision-recall curves are all non-negligible. This is totally understandable and we suppose it is due to our random selection strategy: sometimes the customers chosen for training can well cover most of the abnormal causes of NTL while sometimes the customers for training have only limited pattern of manifestation. However, all the detection results are pretty good from multiple perspectives, which demonstrates that this method has its specialties.
Our forward inference network consisted of nine convolutional layers, which is denoted Conv9 in Table 4. We justify that utilizing the unlabeled samples is essential by comparing the results of models trained with or without the semi-supervised learning strategy in Table 4. Even though we used some sampling strategy to make sure that in a training batch the ratio of positive and negative samples was kept constant throughout the experiments, results show that sole labeled training data is not enough for a model to perform well on a larger and maybe more complicated validation sample set. To guarantee that our inference network is not the cause of the poor performance of supervised model, we used a more traditional and reliable architecture, resnet-21. It turns out it still cannot address the problem that a supervised model basically classifies every sample as negative. Semi-supervised learning methods help find a more reasonable decision boundary with the presence of unlabeled data thus preventing the overfitting problem caused by the limited labeled data. Figure 11 gives the results when we change the size of labeled data in training set while keeping other settings unchanged. Generally speaking, the anomaly detection performance gradually increases as the size of labeled samples grows from 500 to 1500. There is a slight drop in recall rate when the size of labeled data changes from 500 to 1000, but the precision increased a bit. In order to prove that our network architecture is different and more suitable for the detection problem and especially for our designated data format, we did a comparison with the original mean teacher model by gradually adding our adaptations. Also, in order to justify the proposed the network structure, we trained SVMs (with balanced class weights) as the baseline classifier and chose the result with highest f1 score by ranging the penalty parameters from 0.01 to 1000 (increasing by In order to prove that our network architecture is different and more suitable for the detection problem and especially for our designated data format, we did a comparison with the original mean teacher model by gradually adding our adaptations. Also, in order to justify the proposed the network structure, we trained SVMs (with balanced class weights) as the baseline classifier and chose the result with highest f1 score by ranging the penalty parameters from 0.01 to 1000 (increasing by orders of magnitude). We used the original mean teacher model and gradually added improvements on top of it. The results tested on different models are shown in Table 5. For the general NTL detection goal, it is equivalently important have a high value on both precision and recall. Extreme result on either single index is obviously not desired. Generally, SVM does a great job in terms of recalling the NTL samples but the detection accuracy is apparently much lower. As for following the models, simpler models tend to categorize samples into the class that is in the majority. An intuitive explanation of such improvement is that triplet loss tries to minimize the pattern discrepancies among users in a same category while the role RoI pooling plays is that it highlights where the points cloud is in an image channel and naturally makes the pattern features more explicit. We can infer that our network architecture does improve the performance of detection.

Discussion
The major challenges for most of the NTL detection problems are that the anomalies can take on various forms of manifestations that are hard to capture. Worse still, the number of cases we can study from is often limited as a result of the labor-consuming on-field inspection. So it not a traditional anomaly detection problem and it is essential to make use of all the types of data available to get complete and prominent representations of the input.
Most of the previous works mainly focus on the historical energy consumption data because it has the most direct relevance to NTL detection problem. However, the types of NTL that can be detected are bound to be limited by the limited choice of measurements. [4][5][6][7][8] used sole energy consumption (EC) data. Apart from it, [32] utilized auxiliary databases and [33] made use of credit worthiness ratings. In [34], researchers tried to gather different features from various types of SM data such as the quality of measurements, electrical magnitudes, GIS data, and technological characteristics of the SM besides EC. As for our proposed method, we utilize as many types of electrical parameters we can access. There are two main advantages of our data transformation method: it focuses on the customers' long term behaviors rather than single data points; the multi-channel image input makes it possible for the deep neural networks to automatically extract the underlying correlations among these features. Besides, our data transformation approach can tolerate slight data distortion or absence. We did not make use of specific auxiliary data except the most common electrical parameters. Such transformation provides a new aspect of viewing the electrical parameters.
Furthermore, our proposed approach deals with the situation where there are only a small number of labeled samples. Table 6 shows the comparison between some of the state-of-the-art approaches and our proposed approach. All these approaches except ours deal with the problem in the supervised fashion which took advantage of the abundance of customer class information. Also, Figure 11 shows that although the detection performance may suffer with the decrease of the number of labeled samples, acceptable results can still be obtained. To our limited knowledge, our work is the first to solve the NTL detection problem with a semi-supervised method. Another advantage of our method is that the time range we need to generate a sample is rather short compared with other approaches listed in Table 6. The approach in [11] can make judgments with no delay but actually it is achieved by comparing the current consumption with the average of consumption in the past, so it is not technically real-time. Apart from [11], the detection delay of our methods is apparently shorter. This allows us to make quicker responses to the emergent NTL attacks. For a trained model, we can use a certain range of data records to form our data sample to sufficiently cut down the detection delay.
As for data privacy, the transformation we did hides the original information that SM collected. So there is nearly no original information exposed.
There are still some weaknesses or works we have not done sufficiently for now. Even though we aim to cover as many types of anomalies as possible in the detecting stage, we have not come up with a solution to explicitly indicate the exact causes of NTLs. Our detection results strongly depend on the coverage of different forms of NTL attacks used in the labeled training set.

Conclusions
This paper presented a thorough methodology for detecting non-technical losses. The transformation from the data records collected by smart meters into the super images allows us to view the consumption behaviors of a customer from a statistical perspective in the longer term. The new data format also provides a different way of extracting features and integrated analysis of more types of features would have greater potential of detecting a wider range of anomalies. Followed by the prior knowledge free semi-supervised learning strategy, our method demonstrated its superiority to the supervised learning in the situation where labeled data is in the minority of the entire data set.
Our method is trained and validated on the real data from a power grid of China. With reference to some ideas in two stage object detection models, we designed our network architecture to effectively capture features for classification. Ablation studies in the experimental section demonstrate that our method in each stage does work and the comparison with the state-of-the art methodologies proves that our result is rather competitive and our method has its advantages.