Neural Networks for Driver Behavior Analysis

: The proliferation of info-entertainment systems in nowadays vehicles has provided a really cheap and easy-to-deploy platform with the ability to gather information about the vehicle under analysis. With the purpose to provide an architecture to increase safety and security in automotive context, in this paper we propose a fully connected neural network architecture considering position-based features aimed to detect in real-time: (i) the driver, (ii) the driving style and (iii) the path. The experimental analysis performed on real-world data shows that the proposed method obtains encouraging results.

Introduction
The Internet of things (IoT) is represented by the network of physical devices, home appliances, vehicles and a plethora of devices embedded with actuators, sensors, software and, in general, with connectivity [1]. Recently, Internet of Vehicle (i.e., IoT related to automotive) started to obtain more attention [2]. As a matter of fact, there are several promising benefits deriving from the real-world implementation of the Internet of Vehicle paradigm, for instance, a vehicle can turn from an autonomous and independent system into a component of a more efficient cooperative one [3].
Nowadays, vehicles can send data between them or to an external server in order to adapt the driving style to the road condition; traffic lights can send information about the number of cars in a lane in order to suggest to other cars an alternative path to avoid traffic congestion [4].
In this scenario the proliferation of both mobile devices and info-entertainment systems in modern cars, useful to gather features from vehicles while are travelling, boosted research community to focus about safety and security in the automotive context. In this scenario, data mining algorithms can be useful to analyse features collected from vehicles with the aim to extract knowledge related to the driver behavioral analysis.
For this reason, the main research trends in this topic are ranging from driver detection to driving style recognition.
With regard to driver detection (aimed to continuously and silently authenticate the driver to his/her car), the main aim is two-fold: anti-theft system (for end users) and a system for insurance companies proposing ad-hoc policies for the single driver.
New anti-theft paradigms are necessary, considering that cars are equipped with many computers on board, exposing them to a new type of attacks [5,6]. As any software, the operating systems running on cars are exposed to bug and vulnerabilities [7].
The driver detection is of interest also for auto insurance markets that are rapidly changing: through these plethora of new data easily available, risks can be rated on an • we propose a method able to identify, exploiting the same feature set, the driver, the driving style and the kind of road traveled; • we exploit a feature set easily obtainable (it does not require the access to the sensitive vehicle CAN bus); • we designed and evaluate several fully connected neural networks and we find the optimal hidden layer number tuning for the driver identification task. Furthermore, using this configuration we demonstrate the neural network effectiveness in driving style and path recognition tasks; • for experiment replicability, we consider two real-world datasets (where each instance is manually labelled by researchers with the relative driver, driving style and road) freely available for research purposes.
The paper proceeds as follows-Section 2 introduces preliminary notions about the neural network architecture considered in the following study, Section 3 deeply describes and motivates the proposed method; Section 4 illustrates the results of the experiments, Section 5 discusses the current literature related to driver behaviour characterization and finally, conclusions are drawn in Section 6.

Background
In this section we provide preliminary notions about neural network building blocks that is, the basis of each architecture neural network based.
One of the most important concept of neural network is represented by the so-called layer, a data processing module working as a filter for data. Basically, data are the layer inputs and they come out in a more useful form. Specifically, the layers are able to extract representations out of the data fed into them, representations that should be more meaningful for the problem to solve.
The neural network training consists in the definition of the following objects: • layers: that are combined into the designed network; • input data: the feature instances with the correspondent target labels; • loss function: this function defines the feedback signal used for the learning; • optimizer: the optimizer determines how the learning proceeds.
The interaction between the described objects are depicted in Figure 1. The network aims to map the input data intro predictions and it is basically composed of layers together chained. The loss function compares the predictions with the target label, the result of this comparison is the loss value (i.e., a measure about how well the prediction of the network match was expected): this quantity is expected to be minimized during training [12]. Finally the optimizer uses the loss value to update the weights of the network: it determines how the network will be updated based on the loss function, usually it implements a specific variant of stochastic gradient descent.
As already introduced, the layer is the fundamental data structure-several layers are stateless, but there are several layers considering the state, in this case the layer exhibits weights which together contain the network knowledge. In the network design phase it is possible to choose between several layers, for instance, whether the input data are represented as vectors it is possible to use densely connected layers, while for image data 2D convolution layers are usually considered.
As an example, let us consider the following snippet: layer = layers.Dense(32, input_shape = (784,)) The instruction represents the definition of a densely connected layers accepting a 784 sized vector and will return a 32 sized vector (a dense layer with 32 output units). As a consequence, the next layers will be declared in such a way that it is able to accept as input a 32 sized vector so building the layer chain.
The purpose of the layer is to transform the data. In the following we explain how these transformations can take place: a more complete declaration of the densely connected layers, previously considered, is the following one: layer = layers.Dense(16, input_shape = (784,), activation = 'relu') The argument passed to densely connected layers is 16 in this case and it represents the number of hidden units of layers (an output 16 sized vector): formally a hidden unit is a dimension in the representation space of the layer.
In this layer the function able to "resize" the data has been specified (i.e., activa-tion='relu' and it is called activation function). The layer can be interpreted as follows: The output value is the combination of several operations: a dot product (i.e., dot) between the input vector and W (the weights of the layer) and an addition between the resulting vector and the bias (i.e., b). While the relu and the addition are element-wise operations, the dot operation combines entries (this is the operation that consents to the layer to "resize" the output data) [13]. Considering that W represents the actual value of weights for the considered layers (considering that the number of the weight is equal to the number of the neurons, for this reason whether we want to transform the data into a 16 size vector, the layer exhibits 16 weight, one weight for each neuron), while input represents the input feature vector.
Let us consider the following example in which there is a densely connected layer aimed to transform a 4 size vector into a 3 size one, where the input is I = [2, 3, 5, 6] and the weights are W = [5,6,7], the dot product between I and W is: the output of this layer is O = [80, 96,114]: the layer resized the 4 size input data into a 3 size output data.
Finally, a relu (i.e., Rectified Linear Unit) operation (i.e., relu(x) = max (x,0)) is performed. For instance, with 4 hidden units, the dot product with W will project the input data onto a 4-dimensional representation space (to this we have to add the b bias vector and the relu operation). The relu trend is shown in Figure 2. Basically the relu activation function is equal to zero when x < 0 and it is linear with slope 1 when x > 0. Considering more hidden units that is, an higher dimensional representation space, allows the network to learn more complex feature representations, but it makes the network more computationally expensive and may lead to learn unwanted data patterns (i.e., pattern that are improving the training data but not the testing ones).
From this discussion, it appears that the network designer in developing densely connected layers-based architecture have to choose: (i) how many layers to use and (ii) how many hidden layers to choose for each layer.
In addition the relu function, frequently considered as candidate for intermediate layers, another widespread activation function, the so-called softmax, is usually considered as last layer in classifiers neural network based before the output layer.
For instance, whether an instance under analysis can belong to one of the 4 label considered, it is possible to add following layer as final one: layer = layers.Dense(4, activation = 'softmax') Coherently with the previously discussed example, the input from the previous layer is resized in a 4 size vector, the only difference is that the resizing is done using the softmax function and not the relu one.
The reason why the softmax activation function is considered as last layer before the output one is that its aim is to normalize an input vector into a probability distribution: it is usual that vector can exhibit negative or greater than one values. In order to have a feature vector which elements summed are equal to one we apply the softmax function. For instance whether the output of the previous densely connected layer with the softmax activation function is equal to [0.2, 0.1, 0.4, 0.3] this is symptomatic that the instance under analysis is belonging to the #3 class (the first element of the output vector presents the probability that the instance is belonging to the first class and so on).

The Driver Behavior Analysis Approach
In this section we describe in detail the high-level architecture and the method we proposed aimed to: (i) driver identification, (ii) detect the driving style and (iii) travelled road detection. Figure 3 depicts the high-level architecture of the proposed method. We distinguish three main blocks in Figure 3: • intelligence: it represents the module able to store data and to compute the three different indicators for each driver. Once computed the indicators (related to the driver, the driving style and the road), this module is able to send suggestions to drivers for instance, in order to change the driving style to avoid traffic congestions. In addition the module is able to understand whether a vehicle is driven by the owner (to identify whether a theft is happened); • network: this module represents the communication infrastructure of the proposed architecture; • environment: this block depicts the environment, in which each vehicle is able to send a set of information related to its position. Considering that we use information related to the accelerometer sensor, the proposed method works on any type of vehicles (i.e., cars, trucks and motorcycles), without differences from the method performances point of view. Furthermore, only a mobile device (with a fixed-support) is required to obtain the information to send to the network module (alternatively it is also possible to send the accelerometer information using the info-entertainment system whether available in the vehicle).
In the following we describe in detail the proposed method to analyse the driver using data retrieved by positioning systems that is, the intelligence block in the architecture depicted in Figure 3.
We consider a set of features available from accelerometer, as shown in Table 1. The feature set depicted in Table 1 contains the data available from the inertial sensors (available on mobile phones and on info-entertainment systems) at 10 Hz (reduced from the phone 100 Hz by taking the mean of every 10 samples). With the aim to gather data, the mobile device is fixed on the windshield at the start of the travel, in this way the axes are the same during the whole trip. These are aligned in the calibration process, being Y aligned with the lateral axis of the vehicle (reflects turnings) and Z with the longitudinal axis (positive value reflects an acceleration, negative a braking). The accelerometer measurements are logged filtered using a Kalman Filter (KF) [14]. KF is an algorithm exploiting a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe: a common application is for guidance, navigation, and control of vehicles, particularly aircraft and spacecraft [15].
We considered a real-environment and not a simulated one in order to take in account all possible real-world variables, for instance, slowdowns traffic lights and all the possible variables that are not considerable from simulated environments. We highlight that the proposed feature set can also be easily collected from cheap mobile devices equipped, for instance, with the Android operating systems, but also from the info-entertainment systems available in modern vehicles. Figure 4 depicts the flow diagram of the intelligence module in detail: • the real-time feature acquisition is the module responsible to gather the feature vector from the accelerometer sensor of the positioning systems (for instance the mobile device or the info-entertainment system built-in the vehicle); • the driver detection model is the module able to test the feature vector obtained in the previous step against to the model to detect the driver; • the driving style detection model is the module able to test the feature vector obtained in the previous step against to the model to detect the driving style; • the road detection model is the module able to test the feature vector obtained in the previous step against to the model to detect the kind of road travelled; • the driver label module contains the output of the prediction obtained in the driver detection model that is, the label related to the driver; • the driver style label module contains the output of the prediction obtained in the driving style detection model, that is, the label related to the style (i.e., normal, aggressive or drowsy); • the road label module contains the output of the prediction obtained in the road detection model that is, the label related to the style (i.e., secondary or motorway).

The Classification Process
We consider supervised techniques. To build models we consider the following three main steps: (i) data instances pre-processing (data cleaning and labeling), (ii) training neural networks with an increasing number of hidden layers and (iii) classification performance assessment (Testing).
The first step (i.e., pre-processing) consists in the cleaning of the raw instances of the feature by removing incomplete and wrong instances.
The second one, the Training, is depicted in Figure 5: starting from the data (in this case the features gathered from accelerometer), these algorithms generate an inferred function. The inferred function, provided by the neural network, should be able to discriminate between accelerometer features belonging to several classes, that is, the function should define the boundaries between the numeric vectors belonging to several classes. In this study, the classes to discriminate are those related to drivers, the driver behaviours and kind of roads. One limit about the quality of the inferred function, and therefore of the supervised machine learning techniques, is related to the training data; it is important that the data used to build models are free of bias, in order to generate models able to predict the right class for unseen instances.
The third step, once generated the model is the evaluation of its effectiveness-the Testing one, is depicted in Figure 6. The effectiveness of the models is measured using well-know information retrieval metrics. It is important to evaluate instances not included into the training data in order to evaluate the (real) performance of the built models. To assure this, we consider the cross validation-we split the full dataset in two parts and we used the first part as training dataset and the second one as testing dataset. We repeat this process considering different instances for both training and testing dataset in order to evaluate all the instances belonging to the full dataset.
Different models are defined-the first one for the driver detection, the second one for the driving style detection, while the third one is related to road identification: we highlight that each classification considers the same accelerometer feature set. To train the classifiers, with regard to the driver detection, we defined T as a set of labeled traces (M, l), where each M is associated to a label l ∈ {U 1 ,. . . ,U n } (where U n represents the n-th driver under analysis that is, Driver #n, with 1 ≤ n ≤ 16). For the second classifier, for the driving style detection we defined T as a set of labeled traces (M, l), where each M is associated to a label l ∈ {U 1 ,. . . ,U n } (where U n represents a different driving style that is, Normal, Drowsy and Aggressive). Relating to the road identification classifier, we defined T as a set of labeled traces (M, l), where each M is associated to a label l ∈ {U 1 ,. . . ,U n } (where U n represents the kind of road considered that is, Secondary and Motorway).
For each M the process calculates a feature vector V f that is presented to the classifier during training.
In order to perform assessment during the training step, a k-fold cross-validation is used [16]: the dataset is subdivided into k subsets using random sampling. A subset is retained as a validation dataset to assess the trained model whereas the remaining k − 1 subsets are exploited to perform training. Such process is repeated k = 5 times-during the ten iterations, each of the k subsets has been used once as the validation dataset. To obtain a single reliable estimate, the final results are evaluated by computing the average of the results obtained during the ten iterations. The process starts by partitioning the dataset D in k slices. Then, for each iteration i, we train and evaluate the effectiveness of the trained classifier following the steps reported below: 1.
the training set T i ⊂D is generated by selecting an unique set of k − 1 slices from the dataset D; 2.
the test set T i = D − T i is generated selecting the remaining kth slice (it can be evaluated as the complement of T i to D) 3.
a classifier is trained on set T i ; 4.
the trained classifier is applied to T i to evaluate accuracy.
Since k = 5, each iteration i is performed using the 80% of the dataset D as the training set (T i ) and the remaining 20% as test set (T i ).
The last step consists of using the trained classifiers on real user data to assess their performances. The aim is to identify the driver under analysis (with the first model), the driving style (with the second model) and the road (with the third model) in real-time, the adopted features model and feeding with it the trained classifier. The classification, in this study, is also performed by using a traditional machine learning approach based on decision trees (i.e., J48) with the aim of evaluating the performance improvement of the proposed neural network classifier with respect to the existing machine learning approaches (that do not use hidden layers).

The Neural Network Architecture
To build the three models related to the driver detection, driving style detection and road detection we design a neural network architecture.
The neural network architecture we propose in detail consists of a variable number of k hidden layers, with min ≤ k ≤ max where min and max are, respectively, the minimum and the maximum number of hidden layers. We consider a different number of layers because in the evaluation section, we will experiment with several hidden layers with the aim to obtain the best performances for the driver detection task. Once the network is set up, the best architecture for the driver detection task will be used for the driving style detection and for the road detection tasks. The supervised classification approach described, aimed to generate the classifier model, is built considering a neural network architecture and is basically composed by following layers: • Input layer: this layer is used as an entry point into the network, there is a node for each considered feature; • Hidden layers: these layers are composed of artificial neurons (for instance, perceptrons). For each artificial neuron, the output is computed as a weighted sum of its inputs and passed through an activation function (that can be a sigmoid, a ReLu or a soft plus function). • Dropout layer: this layer implements a regularization technique, which aims to reduce the complexity of the model with the goal to prevent over-fitting. Basically, this layer randomly deactivates several neurons in a layer with a certain probability p from a Bernoulli distribution. The reduced network is trained on the data in each stage and the deactivated nodes are reinserted into the network with their original weights. In the training stages, the "drop" probability for each hidden node is usually in the range [0, 0.5] (for this study 0.5 was used because dropping a neuron with 0.5 probability gets the highest variance for the distribution).
• Batch Normalization: Batch normalization [17] is a method for improving the training of feed-forward neural networks. It allows to obtain speed improvements and enables the adoption of higher learning rates, more flexible parameter initialization, and saturating non-linearities. We adopted this approach since batch normalized models achieve higher accuracy on both validation and test, due to a stable gradient propagation within the network. • Output layer: this is the last layer of neurons that produce given outputs. We used a dense layer, exploiting a linear operation in which every input is connected to every output.
The MLP was trained by using cross-entropy [18] as a loss function, and stochastic gradient descent (SGD) for optimizing the loss function. Clearly, once obtained the best network configuration for the driver detection tasks (by training the network with the driver label, the same network is considered for the driving style (by considering the driving style labels) and road detection (by considering the road labels).
In Figure 7 we show the pseudocode related to the designed neural network implementation. In the pseudocode in Figure 7 from line 5 to line 7 there is the Hidden layer implementation (the one that we repeat in order to tune the network).
For the network implementation, the Python programming language in considered. Moreover, the Tensorflow (https://www.tensorflow.org/) and the Keras (https://keras. io/) libraries are exploited. TensorFlow is an open source software library for machine learning, providing proven and optimized modules, useful in building algorithms for different types of artificial intelligence tasks. From the other side Keras is an open source library for machine learning and neural networks. It is written in the Python programming language and it is designed as an interface at a higher level of abstraction than other similar lower level libraries. It supports TensorFlow libraries as a back-end.

Study Design
We design an experiment aimed to investigate whether the feature set we consider is useful to discriminate between different drivers, different driving style and different roads using the proposed neural network architecture.
In detail, the experiment is aimed at verifying whether the considered features are able to predict unseen driver instances, unseen driving style and unseen road instances. The classification is carried out by using the neural network described in the previous section with the 9 features gathered from accelerometer as input neurons.
The evaluation consists of three stages: (i) a comparison of descriptive statistics of the populations of different drivers, driving styles and roads instances; (ii) hypotheses testing, to verify whether the features have different distributions for the considered driver populations; and (iii) a classification analysis aimed at assessing whether the considered features are able to correctly discriminate between different drivers, driving styles and roads related instances.
Each layer in the neural network is used to identify a unique feature of an instance under analysis [12]. Each layer will pick out a particular feature to learn and use it to classify the instance, for this reason we do not perform a feature selection but anyway we consider descriptive statistics and hypothesis testing to respectively provide a graphical and statistical impact about the effectiveness of the proposed feature vector.
To compare the driver, driving style and road instance populations through descriptive statistics we consider boxplot, a method for graphically depicting groups of numerical data through their quartiles. Boxplots represent variation in samples of a statistical population without making any assumptions of the underlying statistical distribution.
With regards to the hypotheses testing, the null hypothesis to be tested is: H 0 : 'different drivers have similar values for the considered features'. The null hypothesis was tested with Wald-Wolfowitz (with the p-level fixed to 0.05) and with Mann-Whitney Test (with the p-level fixed to 0.05). We chose to run two different tests in order to enforce the conclusion validity.
The purpose of these tests is to determine the level of significance, that is, the risk (the probability) that erroneous conclusions be drawn: in our case, we set the significance level equal to 0.05, which means that we accept to make mistakes 5 times out of 100.

Experimental Evaluation
In this section we describe the real-world dataset considered in the evaluation of the proposed method and the results of the experiment.
To implement the designed neural network architecture, we considered Tensorflow (https://www.tensorflow.org/), an open source software library for high-performance numerical computation and Keras (https://keras.io/), a Python-based high-level neural networks API, able to run on top of TensorFlow. Furthermore the Matplot (https:// matplotlib.org/) library is considered as plotting library. We developed the network using the Python programming language.
The results of the evaluation are presented reflecting the data analysis' division in three phases discussed in previous section: descriptive statistics, hypothesis testing and classification.
The metrics that are used to evaluate the classification results are the following: Precision, Recall, ROC Area, Accuracy and Loss.
Precision has been evaluated as the proportion of the examples that truly belong to a given driver among all those which were assigned to him/her. The ROC Area is a measure of the probability that a positive example selected randomly is classified above a negative one.
The accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's true value: it is the fraction of the classifications that are The Loss is a quantitative measure of how much the predictions differ from the actual output (i.e., the label). Loss is inversely proportional to the correctness or the model.
The loss is calculated on training and validation and its interpretation is how well the model is doing for these two sets: it is a summation of the errors made for each example in training or validation sets.
The machine used to run the experiments and to take measurements was an Intel Core i7 8th gen, equipped with 2GPU and 16Gb of RAM.

The Datasets
The datasets considered in the evaluation were gathered from a smartphone fixed in the car using an adequate support [14,19].
We collected data from two different datasets: the first one [14]considering six drivers, the second one [19] related to the data of ten different drivers.
A total of sixteen drivers participated to the experimental analysis, in Table 2 for reasons of space we show details about six of the drivers involved in the experiment (the ones gathered from the first dataset) [14]). The full dataset we consider in the experimental analysis consist in more than 1000 min of naturalistic driving with its associated labels about the driver (i.e., from Driver #1 to Driver #16), the driving style (i.e, Normal, Drowsy and Aggressive) and the road (i.e., Secondary, Motorway).
The three driving styles (Normal, Drowsy and Aggressive) were performed in two different routes, one is 25 km (round trip) in a Motorway type of road with normally 3 lanes on each direction and 120 km/h of maximum allowed speed, and the other is around 16 km in a Secondary road of normally one lane on each direction and around 90 km/h of maximum allowed speed. The dataset publicly available for research purpose at the following urls: http://www.robesafe.com/personal/eduardo.romera/uah-driveset/ and https://github.com/martinacimitile/Car-Data-Mining/wiki. Each instance of the datasets is manually marked by researchers in [14,19] with three labels: the driver, the driving style and the road.

Descriptive Statistics
The analysis of boxplots helps to identify whether the features are helpful to discriminate between different drivers.
For space reasons, Figure 8 shows the boxplots related to the distributions of the considered features between the first 4 drivers involved in the experiment. For reasons space, we do not represent all the boxplots but the same consideration can be provided for the remaining features.
The "(a)" box shows the boxplot related to the F9 feature (i.e., yaw): we notice that D2 and D3 drivers exhibits a similar boxplot, while D4 boxplot is smaller if compared with the one exhibit by D1 driver.
The "(b)" box shows the boxplot for the F7 feature (i.e., roll), from the point of view of these distributions we can highlight the boxplot related to the four driver are similar, but the D4 driver exhibits an higher maximum if compared with the other drivers.
The "(c)" box is related to the boxplot for the F9 feature (i.e., yaw) for driving style identification. From this boxplot we can see that the with regard to the drowsy boxplot, this is one is higher that the remaining ones (i.e., normal and aggressive).
The "(d)" box is related to the boxplot for the F9 feature (i.e., yaw) for road detection. The motorway boxplot exhibits a bigger range of values (between the first and the third quartile) if compared to the secondary one.

Hypothesis Testing
The hypothesis testing is aimed to evaluate whether the considered features exhibit different distributions for the populations of different feature belonging to the drivers with statistical evidence.
We assume valid the results when the null hypothesis is rejected by both the performed tests.
To summarize, considering that we assume the results valid when the null hypothesis is rejected by both performed tests, all the considered features passed both the tests (p < 0.001), symptomatic that the full feature vector can be suitable to discriminate between drivers, as the classification analysis will confirm.

Classification Analysis
In the following, we present the classification results obtained from both the neural network and machine learning classification approaches (considered as baseline) on the real-world datasets we evaluate. Table 3 shows the obtained performance for the several neural network architectures, with the aim to find the best network tuning. Once found the best neural network tuning, we will consider this setting to build the models for the driving style and road detection. As shown in Table 3, we considered a classic machine learning based (i.e., ML where the J48 algorithm is considered) approach and k = 30 different neural networks. Specifically, the NN1 refers to a neural network with a single hidden layer (and a single dropout layer). Generalizing, the NN N algorithm includes N layers and N Dropout layers. We obtain the best precision and recall values with the NN25 configuration that is, the neural network with 25 hidden layers: 0.998 of precision and 0.999 of recall. Furthermore, we highlight that neural networks with more than 8 hidden layers overcomes the machine learning approach-for the machine learning classification, a precision equal to 0.874 and a recall equal to 0.879 is obtained while with the NN9 network the precision is 0.878 and the recall is 0.881.
In the follow, once obtained the best neural network architecture for driver detection (i.e., the DL25 one), we build with this setting the models for the driving style and road detection and we show the plots related to the Accuracy and the Loss metrics for the driver identification, the driving style identification and the road identification.
From the Accuracy and Loss definitions, it is expected that Accuracy and Loss should be inversely proportional-for high values of accuracy, low loss values are expected (and the opposite). Furthermore, considering that the weights and bias are initially random selected, the accuracy trend should start by exhibiting low values (and high loss value, symptomatic that the network is performing wrong predictions), but whether the network during the several "epochs" (i.e., one forward pass and one backward pass of all the training examples) is able to learn (i.e., it is able to solve the driver prediction problem), the accuracy should start to exhibit higher values in the next iterations (and consequently the loss should exhibit low values). The epoch is a parameter chosen by the network designer, usually the number of epochs chosen is such that the loss is at least and it does not get worse in the immediately succeeding epochs and, consequently, the accuracy value reached is the maximum and in the immediately succeeding epochs is not improving, symptomatic that the network has reached the stability and that further epochs would not improve performances. We set the number of epochs equal to 100, because the three networks reached the stability with a number less or equal to 100. Figure 9 shows the performances (in terms of accuracy and loss) related to Driver #1 detection. We only show some accuracy and loss plots, considering that the remaining plots are similar to the ones shown. With regard to the effectiveness of the proposed neural learning network in driver detection we obtain that the average accuracy is equal to 0.99 and the average loss is equal to 0.40. Figure 10 shows the performances (in terms of accuracy and loss) related for driving style detection. Following results are obtained: with regard to the Normal Style an accuracy equal to 0.98 and a loss of 0.18 is obtained while, for the Drowsy Style detection an accuracy of 0.92 and a loss of 0.28 is reached and, finally, relating to the Aggressive Style detection an accuracy of 0.98 and a loss of .017 is obtained.
In Figure 11 are depicted the performances (in terms of accuracy and loss) related to secondary road identification. With regard to road detection, the neural network architecture we designed following results are reached: for the Secondary Road detection an accuracy equal to 0.96 and a loss equal to 0.10 is obtained while, relating to the Motorway Road detection an accuracy of 0.91 and a loss equal to 0.26.

Related Work
Current literature related to the driver behavioural analysis in discussed in following section.
Features from accelerator and the steering wheel were analyzed by researchers in [20]. Observing these characteristics, they exploit hidden Markov model (HMM) to model drivers. They basically consider two different models for each driver under analysis, the first model is trained from accelerator features while the second model is trained from steering wheel features. The models can be used to identify different drivers with an accuracy equal to 85%.
Researchers in [21] classify a set of features extracted from the power-train signals of the vehicle, showing that their classifier is able to classify the human driving style based on the power demands placed on the vehicle power-train with an overall accuracy equal to 77%.
In order to study driver behavior questionnaire and self-reports have been explored [22] questionnaires provide a means for studying driving behaviors, which could be difficult or even impossible to study by using other methods like observations, interviews and analyses of national accident statistics. Their findings demonstrate that that bias caused by socially desirable responding is relatively small in driver behavior questionnaire responses.
Van Ly et al. [23] explore the possibility of using the inertial sensors of the vehicle from the CAN bus to build a profile of the driver observing braking and turning events to characterize an individual compared to acceleration events.
Researchers in [24] model gas and brake pedal operation patterns with Gaussian mixture model (GMM). They achieve an identification rate equal to 89.6% for a driving simulator and 76.8% for a field test with 276 drivers, resulting in 61% and 55% error reduction, respectively, over a driver model based on raw pedal operation signals without spectral analysis.
Authors in [25] design a driver identification method that is based on the driving behavior signals that are observed while the driver is following another vehicle. They analyzed signals, as accelerator pedal, brake pedal, vehicle velocity, and distance from the vehicle in front, were measured using a driving simulator. The identification rates were 81% for twelve drivers using a driving simulator and 73% for thirty drivers.
Driver behavior is described and modeled in [26] using data from steering wheel angle, brake status, acceleration status, and vehicle speed through Hidden Markov Models (HMMs) and GMMs employed to capture the sequence of driving characteristics acquired from the CAN bus information. They obtain 69% accuracy for action classification, and 25% accuracy for driver identification.
In reference [27] the features extracted from the accelerator and brake pedal pressure are used as inputs to a Fuzzy Neural Network (FNN) system to ascertain the identity of the driver. Two fuzzy neural networks are designed by authors to show the effectiveness of the two proposed feature extraction techniques.
Authors in [28] propose a method based on driving pattern of the car. They consider mechanical feature from the CAN vehicle evaluating them with four different classification algorithms, obtaining an accuracy equal to 0.939 with Decision Tree, equal to 0.844 with k Nearest Neighbor (KNN), equal to 0.961 with RandomForest and equal to 0.747 using Multilayer Perceptron (MLP) algorithm.
A hidden-Markov-model-(HMM)-based similarity measure is proposed in [29] in order to model driver human behavior. They employ a simulated driving environment to test the effectiveness of the proposed solution.
Nor and colleagues [30] adopt a set of features to recognize the emotion of the driver by using multi layer perceptron (MLP) as classifiers. The data collection was conducted in Singapore. The driver must have at least two years driving experience. They managed to collect 11 drivers including men and women, aged between 24-25 years old. The considered features are the brake and gas pedal pressures. In the experiment they state that each driver meets the accuracy level which is more than 50%: the highest accuracy is obtained from driver 10 with an accuracy equal to 71.94%, while the lowest accuracy is obtained from driver 3 with an accuracy equal to 61.65% in classification according to the other driver.
Authors in [31,32] propose a method to detect aggressive drivers. They design a scoring by employing fuzzy logic by adopting smartphone accelerometers and GPS. In order to evaluate the proposed mechanism, they collected traces from twenty vehicles equipped with an Android application developed by authors.
Castignani et alius [33,34] discuss SenseFleet, a driver profile platform that is able to detect risky driving events independently from the mobile device and vehicle. They use a Fuzzy system to compute a score for the different drivers using real-time context information like route topology or weather conditions. The method is evaluated considering multiple drivers along a predefined path, showing that SenseFleet is able to detect risky driving events and provide a representative score for each individual driver.
Researchers in [35], to control the road and driver behavior, propose a metric to measure the casualty crash for assessing the effectiveness of a PAYD (i.e., pay-as-you drive) scheme. They show that in several cases personalized feedback alone is sufficient to induce significant changes, but the largest reductions in risk are observed when drivers are also awarded a financial incentive to change behavior.
A biometric driver recognition system utilizing driving behaviors is proposed in [35]. The authors, using Gaussian mixture models, extracted a set of features from the accelerator and brake pedal pressure was then used as inputs to a fuzzy neural network system to ascertain the identity of the driver. Their experiment shows the effectiveness of the use of the FNN for real-time driver identification and verification. Jazayeri et alius [36] consider the video analysis to detect and track vehicles. They basically propose an approach with the aim to localize target vehicles in video under different environmental conditions. They train an hidden Markov Model to discern target vehicles from the background in order to track them.
Miyajima et al. [37] propose a set of characteristics gathered by exploiting spectral analysis from driving sessions building a model through a GMM. Authors obtained their evaluated dataset with both a driving simulator and a real car. Experimental results show that this approach reaches a detection rate equal to 89.6% for driving simulator and 76.8% for real vehicle.
Trasarti et alius [38] extract mobility profiles of individuals from raw GPS traces studying how to match individuals based on profiles. They instantiate the profile matching problem to a specific application context, namely proactive car pooling services, and therefore develop a matching criterion that satisfies various basic constraints obtained from the background knowledge of the application domain.
Petzoldt and colleagues [39] analyze the influence on accident involvement of younger drivers. They develop a Computer based training module to complement existing driver training programs by addressing critical cognitive skills. The developed module employs video sequences of potentially hazardous driving situations, including multiple-choice questions with adaptive feedback, in order to increase levels of elaboration and understanding. Authors concluded that Computer based training can potentially assist instruction of cognitive skills necessary for save driving.
Wu et al. [40] presents a study related to driver's voice feature selection and classification for speaker identification in vehicle systems. The designed system consists of a combination of feature extraction using continuous wavelet technique and voice classification using artificial neural network. In the feature extraction, a time-averaged wavelet spectrum based on continuous wavelet transform is proposed. Meanwhile, the artificial neural network techniques were used for classification in the proposed system. To verify the proposed system, they consider a conventional back-propagation neural network (BPNN) and generalized regression neural network (GRNN)-the experimental results obtain an identification rate equal to 92% for using BPNN and 97% for using GRNN approach.
The authors of [41] consider the identification of distracted drivers rather than focus on the driver identification. They analyse three CAN-Bus signals (i.e., vehicle speed, steering wheel angle and brake force) and three different maneuvers (left turn, right turn and lane change). In addition, as features, the authors also consider the driver and the road videos, the driver speech, the distance sensor using laser, the GPS sensor for position measurement. Their method basically adopts HMM to discover the optimum number of states for each maneuver. The driving scenarios include two different routes-residential and commercial areas, while each route is driven by each driver twice-neutral and distracted.
Carfora and colleagues [8] design an approach aimed to detect the driver driving style by considering the adoption of unsupervised machine learning techniques. They evaluate the proposed method with a real-world case study. Moreover, they discuss how the proposed method can be considered as risk indicator for car insurance companies.
Martinelli et al. [42] consider the task of driver aggressiveness detection by exploiting unsupervised classification techniques. They design a model with a set of features from the CAN bus of a real-world vehicle while traveling in urban and highway roads.
Researchers in [9] starting from a set of features gathered from the in-vehicle CAN bus, exploit several machine learning algorithms to distinguish between the car owner and impostors. Moreover, they assess the proposed machine learning models effectiveness in the evaluation of instances not evaluated in the training set.
Researchers in [19] collected a set of numerical features for driver identification and path detection. They consider supervised machine learning, in detail they exploit a timeseries classification approach based on a multilayer perceptron (MLP) network.
The authors of [43] proposed the adoption of the MLP network to build a model based on feature obtained from the CAN bus for the driver detection task, considering in the experimental analysis 10 different drivers.
The author of [44] proposed a method to automatically detect driver distraction. They consider a convolutional neural network for detecting behaviours as, drinking or adjusting radio, by analysing video streams. They obtain an average accuracy equal to 0.80.
Zhang et al. [45] proposed a driver distraction method by exploiting a deep learning (DL) approach. Their approach is related to the adoption of three main modules: multi-modal representation learning, multi-scale feature fusion and unsupervised driver distraction detection, obtaining an accuracy equal to 0.97.
Researchers in [46] exploit a convolutional neural network for distracted driver detection. They consider transfer learning, using the VGG16 pretrained model, obtaining an accuracy of 0.96.
The authors of [47] adopt a convolutional neural network for the extraction of the representation of eye and mouth fatigue from the face area detected from video frame. Their driver drowsiness detection method obtains an accuracy equal to 0.75.
Yun et al. [48] developed a clustering-based method to transform various vehicle data into cluster coordinates, to analyze data distribution, to analyze sampled cluster data and decide cluster characteristics, and to predict and monitor vehicle abnormal states. Differently from our proposal, this is work is related to safety. Table 4 presents a comparison of the proposed approach with respect to the current state-of-the-art literature.
In Table 5, we propose an overview related to the driver detection task (as highlighted in this section, the one most considered by researchers). We compare the state-of-the-art work in terms of features exploited and the detection rate obtained.
As shown from the driver detection results shown in Table 5 the only work obtaining a high detection rate by exploiting features not obtained from the CAN bus is the one we propose. Nonetheless, there are several work in literature obtaining high performances (i.e., references [10,29] but these methods require the permanent connection to the CAN bus of vehicle under analysis. Table 5. Current Literature Comparison on Driver Detection (column Research) expressed in terms of: features considered (column Features) and the obtained performances (column Detection Rate).

Features Detection Rate
Wakita et al. [25] accelerator pedal, brake pedal, vehicle velocity, distance from the vehicle in front 73% Zhang et al. [20] accelerator, steering wheel 85% Kedar-Dongarkar et al. [21] vehicle acceleration, braking, speeding index, throttle activity 77% Van Ly et al. [23] accelerator, braking, turning 65% Miyajima et al. [24] distance, velocity, gas pedal, brake pedal 89.6% Nishiwaki et al. [49] gas pedal, brake pedal 76.8% Choi et al. [26] steering wheel angle, brake status, acceleration status, speed 70% Wahab et al. [50] accelerator pedal, brake pedal 95% Meng et al. [27] acceleration, brake, steering wheel 80% Miyajima et al. [37] pedal pressure, brake pedal pressure, and steering angle, velocity, acceleration, engine speed 89.6% Nor et al. [30] brake and gas pedal pressures 71.94% Martinelli et al. [10] intake air pressure, engine soacking time, long term fuel trim bank1, torque of friction, transmission oil temperature, steering wheel speed 98% Martinelli et al. intake air pressure, engine soacking time, long term fuel trim bank1, torque of friction, trasmission oil temperature, steering wheel speed 99% Martinelli et al. [9] Intake air pressure, torque of friction 0.93 Our Approach Acceleration, roll, pitch, yaw 0.99 The main difference with regard to the elicited works is that, in this paper, we propose an architecture aimed to identify the driver, the driving style and the kind of road travelled exploiting feature related to the accelerometer sensor available from the info-entertainment system or in a mobile device fixed in the vehicle. In fact, each cited work is focused only on the driver detection or on the driving style detection, moreover several works considered a simulated environment and not the real one (and these ones require to access to sensitive information transiting on the CAN bus). This represents the first attempt to detect the driver, the driving style and the road exploiting the same feature set, overcoming the state of the art literature in terms of performances.

Conclusions and Future Work
In this paper we presented a neural network architecture aimed to provide a framework to analyse driver behaviours aimed to detect (i) the driver, (ii) the driving style and (iii) the travelled road.
We evaluated the proposed method by exploiting two real-world datasets composed by a total of 16 different drivers obtaining the analysed feature vector from the accelerometer integrated into a fixed mobile device (i.e., a smartphone). We obtained an accuracy equal to 0.99 for the driver identification, an accuracy ranging from 0.92 and 0.98 related to style detection and an accuracy ranging to 0.91 to 0.96 for road identification.
With regard to the limitations of the proposed method, one of the most recently discussed problems that afflicts deep learning techniques and artificial intelligence in general, is the lack of explainability and interpretability of the results. In fact, researchers in the field of artificial intelligence are recently proposing solutions that can explain the predictions obtained from a model. We will consider to integrate in the proposed method a way to provide the visual explanations [51,52] of the results, as already provided in other contexts for instance, in the COVID-19 detection [53].
As future work, we plan to investigate whether the application of formal methods [54][55][56] can be useful for the driver, driving style and road identification, which have been already successfully used in other domains like, for instance, malware detection [57][58][59] and in system biology [60]. Funding: This work has been partially supported by MIUR -SecureOpenNets, EU SPARTA, Cyber-SANE and E-CORRIDOR projects.

Conflicts of Interest:
The authors declare no conflict of interest.


Design of Reliable Disaster Recovery System through Integrated Server Redundancy

Even before the September 11 terrorist attacks in the United States in 2001, information systems prepared against disasters in Korea were extremely weak. However, as various domestic and foreign accident cases have occurred, it is recognized that preparations for this are necessary. Accordingly, at present, each institution has prepared and implemented various backup policies to protect the institution's information and data in case of disaster. Therefore, in this paper, we conducted a study to design a more stable and efficient disaster recovery system by building redundancy for server operating in integrated data center. To do this, we analyzed the redundancy design for the integrated disaster recovery server and designed the overall system configuration. Also, the design results were analyzed by testing web server redundancy and switch redundancy. In this paper, the proposed design method for stabilization and efficiency of disaster recovery system is the redundant construction of integrated server and switch. In other words, the disaster recovery system was composed of active storage and standby storage, and data stabilization was promoted through real-time replication of each other. In the existing disaster recovery system, there is a problem in stabilizing replication because there is no monitoring system for internal replication between storage arrays. To solve this problem, we designed a system that replicates all data in active storage to standby storage in real time and monitors the replication status. Therefore, introducing service conversion automation from the main system, which is the method designed in this paper, to the disaster recovery system, improves the stability and reliability of the service of the local governments, so that it is possible to operate a more efficient and advanced disaster recovery system.

Introduction
In recent years, Korea has been suffering from various natural disasters such as earthquakes and typhoons. Since natural disasters have no solution for humans, there is a growing sense of crisis. In particular, the damage caused by natural disasters is gradually increasing, and the damage to them is also increasing. In the future, safeguarding information assets is of paramount importance [1]. Therefore, it is necessary to minimize the loss of information assets due to natural disasters. Recently, the importance of a disaster recovery system for protecting information assets from such natural disasters and for rapid disaster recovery has been highlighted. Disaster Recovery System (DRS) is a system that recovers quickly when a system problem occurs and performs its original function. In other words, if a company's IT infrastructure such as a data center fails to perform its function due to various disasters and disasters such as natural disasters or hacking, it means to replace or recover the system so that it can perform its original function [2][3]. Of course, since they have one more system with the same structure as the existing IT system, they are often reluctant to introduce them due to construction and operating costs. However, the dependence on IT of all industries is increasing and at the same time, threat factors of IT infrastructure such as hacking, terrorism and natural disaster are also becoming more complicated and diversified. In such a situation, if the IT infrastructure system is shut down, not only will there be a huge cost loss, but also the external credibility will fall, causing tremendous damage to the enterprise [4].
Except for disasters that are extremely unlikely to occur, such as a large earthquake, flooding or flooding of fires can be a threat at any time. If these incidents occur at the enterprise computer center, it can take up to several months to physically completely restore the infrastructure. In this way, even the ultimate restoration of infrastructure and services can be understood as disaster recovery in a broad sense [5][6]. However, from the standpoint of operating a corporate IT service, it is a top priority to restart the service quickly without considering the disaster. Therefore, building a disaster recovery system in terms of IT operations is to prepare for the normal operation so that the service can be replaced with another system as soon as possible in the event of a disaster. In 2000, Dongwon Securities was flooded with data centers and paralyzed stock trading for three days. As the September 11 terrorist attacks began, awareness of the disaster recovery system began to increase. The Financial Supervisory Service recommended that financial institutions establish and operate DR centers in 2001 to recover within three hours of a disaster. Financial companies have begun building DR systems in accordance with written regulations. However, problems actually emerged when actual incidents occurred in the data center. In 2010, Citibank incurred an accident in which all electronic financial transactions were suspended due to flooding of the system due to the freezing of the cooler of the Incheon Computer Center. As a result, service interruptions lasted more than six hours. In 2014, a fire broke out in the Samsung SDS Gwacheon data center and paralyzed all Samsung Group affiliates. Although back-up equipment has been restarted to a large extent, the service has been suspended for more than a week. The damage increased further because the disaster recovery system for the area was not yet established.
In the face of these accidents, the domestic perception of the need for a disaster recovery system is improving significantly. In addition to the financial sector that must comply with regulations, the speed of recovery is different, but the willingness to build a disaster recovery system across the industry is fundamental. In recent years, DR systems are also being built in online shopping malls and electronic payments, where small obstacles lead directly to business losses, and in the medical field, which can threaten patient life in the event of system interruption. As a result, there is a growing recognition that disaster recovery systems are an essential element, and various information security laws recommend building a disaster recovery system. Therefore, in this paper, we conducted a study to design a more stable and efficient disaster recovery system by building redundancy for server operating in integrated data center. To do this, we analyzed the redundancy design for the integrated disaster recovery server and designed the overall system configuration. Also, the design results were analyzed by testing web server redundancy and switch redundancy.

DR system current status
Disaster recovery(DR) system refers to a system for minimizing the failure of IT infrastructure such as data center due to various disasters such as natural disasters and hacking. In other words, when a system fails to perform its original function, a system for replacing or restoring it to perform its original function is called a disaster recovery system [7].
As we enter the digital age, computerization of work is accelerating and dependence on information systems is growing. Therefore, the ripple effect caused by information system failure, disaster and network interruption is also increasing. In particular, when R & D is carried out using national budgets and research outputs and related data are stored in the institution's information system, the loss of data or data causes a large loss of time and money nationally. Disaster recovery systems are categorized into independent, joint, and interdependence depending on the type of deployment. In the case of independent construction, it is easy to build and operate and has excellent security by establishing an independent disaster recovery center. However, there is a disadvantage that the construction and maintenance cost is the most expensive. Mainly adopted by large institutions. In the case of joint construction, two or more organizations use the disaster recovery system jointly. It costs less than a stand alone approach, but there are many things to consider, including consensus on security and operations between organizations that use it [8][9].
Lastly, in the case of mutual construction, instead of establishing a separate disaster recovery center, if two or more organizations act as mutual disaster recovery centers, or if one organization has a number of computer centers, disaster recovery centers among computer centers It is to play a role. In the case of mutual establishment of institutions or centers, it is a method of securing a system that is reserved through a contract or cooperation system and allocating a system for disaster recovery system to another organization (center) in the event of a disaster. This approach requires that institutions or centers have similar equipment and scale, and that there is room and compatibility in the system [10]. The cost of investment is low, but the security and reliability of disaster recovery are quite low and hardly feasible. Figure 1 shows the types of disaster recovery systems. The followings must be applied in the design and construction of a disaster recovery system. First, it is necessary to secure the continuity of external services according to the spread of web services of public services. Second, according to Article 56, Paragraph 2 of the Korean Electronic Information Act, the head of the agency has to establish and operate a stable disaster recovery system because it is obliged to operate the information system stably. Third, the latest technologies should be applied such as asynchronous data replication solution of less than 5 minutes, long distance data compression transmission technology over 100km, and remote automatic disaster recovery system solution [11][12]. Figure 2 illustrates the key strategic frameworks that should be applied to the construction and implementation of a disaster recovery system.

Figure 2.
Core strategic method of disaster recovery system

Experimental setup
In this paper, we conducted a study to design a more stable and efficient disaster recovery system by building redundancy for server operating in integrated data center. To do this, we analyzed the redundancy design for the integrated disaster recovery server and designed the overall system configuration. Also, the design results were analyzed by testing web server redundancy and switch redundancy.
In this paper, the proposed design method for stabilization and efficiency of disaster recovery system is the redundant construction of integrated server and switch. In other words, the disaster recovery system was composed of active storage and standby storage, and data stabilization was promoted through real-time replication of each other. In the existing disaster recovery system, there is a problem in stabilizing replication because there is no monitoring system for internal replication between storage arrays. To solve this problem, we designed a system that replicates all data in active storage to standby storage in real time and monitors the replication status. In addition, by providing a process that is stored in the disaster recovery system through asynchronous replication of information data, was promoted stabilization. Figure 3 shows the replication status real-time monitoring system proposed in this paper. In order to realize a stable environment of disaster recovery system, the redundant design of the integrated server and the redundant design of the switch are composed of network connection. In addition, data replication was carried out in connection with disaster recovery storage. Figure 4 shows the disaster recovery system server and switch configuration in the data center. In addition, we designed an asynchronous storage environment that provides a recovery point objective (RPO) within 5 seconds for a long distance to construct a synchronization solution for disaster recovery systems. In other words, the mirror function was added to the data center and the disaster recovery system storage to replicate data in an asynchronous manner. This allows for quick resynchronization of the mirrored sites. Figure  5 shows the synchronization solution diagram of the disaster recovery system. In conclusion, an asynchronous storage-based remote replication solution was applied to provide a 3 to 5 second recovery point objective (RPO) over a long distance. In addition, the mirroring function was added to provide perfect synchronization between the disaster recovery system and the common-based system. That is, the mirroring function was added to the storage for the main center and the disaster recovery system, and asynchronous data replication was performed. In addition, fast resynchronization of the mirrored site was performed using only incremental changes. Finally, the redundancy test of the integrated DR server was conducted by removing any ports and cables among the two lines of the DR server service network. After removing any ports, the network service was tested for normality, and after removing any cables, the volume mount was tested for normalization. Figure 6 shows the screen to check the mount status after removing the port and cable.

4.
Conclusions Currently, each institution has established and implemented various backup policies to protect the institution's information and data in case of disaster. This exists in a variety of ways, from a backup method that stores and stores data on a physical medium such as a tape or disk to a data recovery in real time by building a disaster recovery center. Instead of costly storage of physical backups, there is a risk of data loss from the time the backup is completed to before the disaster occurs. However, real-time backup does not have any loss of information in the event of a disaster, but has the disadvantage that it is expensive to build and operate. Therefore, it is important to select an appropriate disaster prevention and recovery method for each institution.
In this paper, we conducted a study to design a more stable and efficient disaster recovery system by building redundancy for server operating in integrated data center. To do this, we analyzed the redundancy design for the integrated disaster recovery server and designed the overall system configuration. Also, the design results were analyzed by testing web server redundancy and switch redundancy. Especially, in case of failure of integrated disaster recovery server network and HBA port part, it is designed to automatically switch over through server redundancy configuration and maintain service availability. In particular, we confirmed the redundancy of the Unified Disaster Recovery Server Network (UTP). In addition, we checked the HBA (SAN) redundancy of the integrated disaster recovery server.
By implementing web server redundancy and switch redundancy proposed in this paper, can be realized more stable disaster recovery system environment. In particular, through the service verification system of the information work system, it is possible to establish a defect-free system of the disaster recovery system. In addition, it is possible to secure reliability by considering compatibility with existing equipment, technology, and technology to be introduced.


Undistillable: Making A Nasty Teacher That CANNOT teach students

Knowledge Distillation (KD) is a widely used technique to transfer knowledge from pre-trained teacher models to (usually more lightweight) student models. However, in certain situations, this technique is more of a curse than a blessing. For instance, KD poses a potential risk of exposing intellectual properties (IPs): even if a trained machine learning model is released in 'black boxes' (e.g., as executable software or APIs without open-sourcing code), it can still be replicated by KD through imitating input-output behaviors. To prevent this unwanted effect of KD, this paper introduces and investigates a concept called Nasty Teacher: a specially trained teacher network that yields nearly the same performance as a normal one, but would significantly degrade the performance of student models learned by imitating it. We propose a simple yet effective algorithm to build the nasty teacher, called self-undermining knowledge distillation. Specifically, we aim to maximize the difference between the output of the nasty teacher and a normal pre-trained network. Extensive experiments on several datasets demonstrate that our method is effective on both standard KD and data-free KD, providing the desirable KD-immunity to model owners for the first time. We hope our preliminary study can draw more awareness and interest in this new practical problem of both social and legal importance.

INTRODUCTION
Knowledge Distillation (KD) (Hinton et al., 2015) aims to transfer useful knowledge from a teacher neural network to a student network by imitating the input-output behaviors. The student model imitates logit outputs or activation maps from the teacher by optimizing a distillation loss. The efficacy of leveraging teacher knowledge to boost student performance has been justified in many application fields a), yielding high performance and often lighter-weight student models. Typically KD requires learning the student model over the same training set used to train the teacher. However, recent studies (Lopes et al., 2017; have demonstrated the feasibility of the data-free knowledge distillation, in which the knowledge is transferred from the teacher to the student without accessing the same training data. This is possible because the training data are implicitly encoded in the trained weights of deep neural nets. The data-free knowledge distillation is able to inversely decode and re-synthesize the training data from the weights, and then clone the input-output mapping of the teacher network. With many practical benefits brought by the KD technique, this paper looks at KD's unwanted severe side effect, that it might pose risks to the machine learning intellectual property (IP) protection. Many machine learning models will be only released as executable software or APIs, without opensourcing model configuration files or codes, e.g., as "black boxes". That can be due to multifold reasons, such as (1) those advanced models might take huge efforts and resources for the model owners to develop, who would need to keep this technical barrier; or (2) those trained models have involved protected training data or other information, that are legally or ethically prohibited to be openly shared. However, KD techniques might open a loophole to unauthorized infringers to clone the IP model's functionality, by simply imitating the black box's input and output behaviors (leaked knowledge). The feasibility of data-free KD Yin et al., 2020) eliminates the necessity of accessing original training data, therefore making this cloning more practically feasible. Even worse, those techniques point to reverse-engineering ways (Yin et al., 2020) to recover the (potentially private) training data from black-box models, threatening the owners' data privacy and security (Yonetani et al., 2017;Wu et al., 2018).
To alleviate the issue, this paper introduces a defensive approach for model owners, called Nasty Teacher. A nasty teacher is a specially trained network that yields nearly the same performance as a normal one; but if used as a teacher model, it will significantly degrade the performance of student models that try to imitate it. In general, the concept of nasty teacher is related to the backdoor attack on deep learning systems (Chen et al., 2017), which creates a model to fit "adversarial" goals in an "imperceptible" way. However, while backdoor attacks aim to manipulate or damage the performance of the poisoned model itself when triggered by specific inputs, the goal of the nasty teacher is to undermine the performance of any student network derived from it. The primary objective of constructing a nasty teacher is for model protection -a novel motivation and setting that have not been explored before. Our contributions are summarized as follows: • We introduce the novel concept of Nasty Teacher, a defensive approach to prevent knowledge leaking and unauthorized model cloning through KD without sacrificing performance. We consider it a promising first step towards machine learning IP and privacy protection.
• We propose a simple yet efficient algorithm, called self-undermining knowledge distillation, to directly build a nasty teacher through self-training, requiring no additional dataset nor auxiliary network. Specifically, the model is optimized by maximizing the difference between the nasty teacher (the desired one) and a normally trained counterpart.
• We conduct extensive experiments on both standard KD and data-free KD approaches, and demonstrate that nasty teacher trained by self-undermining KD can achieve nearly the same accuracy as their original counterpart (less than 1% accuracy gap), while the student model learned from it will degrade accuracy by up to over 10% or even diverge during training.

KNOWLEDGE DISTILLATION
Knowledge distillation aims to boost the performance of light-weight models (students) under the guidance of well-trained complicated networks (teachers). It is firstly introduced in (Hinton et al., 2015), where the student directly mimics the soft probabilities output produced by the well pretrained teacher. The following researchers explore the knowledge transferal from either intermediate features (Romero et al., 2014;Zagoruyko & Komodakis, 2016;Passalis & Tefas, 2018;Ahn et al., 2019;Li et al., 2020), or logit responses (Park et al., 2019;Mirzadeh et al., 2019;Chen et al., 2021a;b;Ma et al., 2021). Recent studies have also shown that, instead of distilling from a complicated teacher, the student networks can even be boosted by learning from its own pre-trained version (Furlanello et al., 2018;Yun et al., 2020;Yuan et al., 2020).
Several recent works also focus on data-free knowledge distillation, under which settings students are not able to access the data used to train teachers. In (Lopes et al., 2017), the author attempts to reconstruct input data by exploring encoded meta-data lying in the pre-trained teacher network. In the following work, the author of (Chen et al., 2019) proposes a learning scheme, called "Data-Free Learning" (DAFL), which treats the teacher as a fixed discriminator, and jointly trains a generator to synthesize training examples so that maximum responses could be obtained on the discriminator. The latest work "DeepInversion" (Yin et al., 2020) directly synthesizes input images given random noise by "inverting" a trained network. Specifically, their method optimizes the input random noise into high-fidelity images with a fixed pre-trained network (teacher).

POISONING ATTACK ON NEURAL NETWORK
The typical goal of poisoning attack is to degrade the accuracy of models by injecting poisoned data into training set (Xiao et al., 2015;Moosavi-Dezfooli et al., 2016). On the contrary, backdoor attack intends to open a loophole (usually unperceived) to the model via inserting well-crafted malicious data into training set (Chen et al., 2017;Gu et al., 2017;Kurita et al., 2020). The goal of back-door attack is to make the poisoned model perform normally well for most of the time, yet failing specifically when attacker-triggered signals are given.
Our proposed self-undermining knowledge distillation aims to create a special teacher model (i.e., an undistillable model), which normally performs by itself but "triggers to fail" only when being mimiced through KD. The motivation looks similar to backdoor attack's at the first glance, but differs in the following aspects. Firstly, the backdoor attack can only be triggered by pre-defined patterns, while our nasty teachers target at degrading any arbitrary student network through KD. Secondly, backdoor attack tends to poison the model itself, while our nasty teacher aims to undermine other student networks while preservingits own performance. Thirdly, our goal is to prevent knowledge leaking in order to protect released IP, as a defensive point of view, while the backdoor attack tends to break down the system by triggering attacking signals, as an attacking point of view.

PROTECTION OF MODEL IP
Due to the commercial value, IP protection for deep networks has drawn increasing interests from both academia and industry. Previous methods usually rely on watermark-based (Uchida et al., 2017;Zhang et al., 2020a) or passport-based (Fan et al., 2019; ownership verification methods to protect the IP. Nevertheless, these methods can only detect IP infringement but remain ineffective to avoid model cloning. A few recent works also explore defensive methods against model stealing (Kariyappa & Qureshi, 2020;Juuti et al., 2019;Orekondy et al., 2020). Typically, they assume attackers obtain pseudolabels on their own synthetic data or surrogate data by querying the black-box model, and train a network on the new dataset to clone the model. However, none of these defense approaches have explored the KD-based model stealing, which is rather a practical threat.

REVISITING KNOWLEDGE DISTILLATION
Knowledge distillation (Hinton et al., 2015) helps the training process of "student" networks by distilling knowledge from one or multiple well-trained "teacher" networks. The key idea is to leverage soft probabilities output of teacher networks, of which incorrect-class assignments reveal the way how teacher networks generalize from previous training. By mimicking probabilities output, student networks are able to imbibe the knowledge that teacher networks have discovered before, and the performance of student networks is usually better than those being trained with labels only. In what follows, we formally formulate the learning process of knowledge distillation.
Given a pre-trained teacher network f θ T (·) and a student network f θ S (·), where θ T and θ S denote the network parameters, the goal of knowledge distillation is to force the output probabilities of f θ S (·) to be close to that of f θ T (·). Let (x i , y i ) denote a training sample in dataset X and p f θ (x i ) indicate the logit response of x i from f θ (·) , the student network f θ S could be learned by the following: where KL(·, ·) and X E(·, ·) are Kullback-Leibler divergence (K-L divergence) and cross-entropy loss, respectively. The introduced "softmax temperature" function σ τs (·) (Hinton et al., 2015) produces soft probabilities output when a large temperature τ s (usually greater than 1) is picked, and it decays to normal softmax function σ(·) when τ s equals 1. Another hyper-parameter α is also introduced to balance between knowledge distillation and cost minimization.

TRAINING NASTY TEACHERS: RATIONALE AND IMPLEMENTATION
Rationale The goal of nasty teacher training endeavors to create a special teacher network, of which performance is nearly the same as its normal counterpart, that any arbitrary student networks cannot distill knowledge from it. To this end, we propose a simple yet effective algorithm, dubbed Self-Undermining Knowledge Distillation, while maintaining its correct class assignments, maximally disturbing its in-correct class assignments so that no beneficial information could be distilled from it, as described next.
Let f θ T (·) and f θ A (·) denote the desired nasty teacher and its adversarial learning counterpart, the self-undermining training aims to maximize the K-L divergence between the adversarial network and the nasty teacher one, so that a false sense of generalization could be output from the nasty teacher. The learning process of the nasty teacher could be formulated as follows, where the former term aims to maintain the accuracy of the nasty teacher by minimizing the cross entropy loss, and the latter term achieves the "undistillability" by maximizing KL divergence between the nasty teacher and the adversarial one. Similarly in equation 1, τ A denotes the temperature for self-undermining, and ω balances the behavior between normal training and adversarial learning.
Implementation We naturally choose the same network architecture for f θ T (·) and f θ A (·) (yet different group of network parameters) since no additional assumption on network architecture is made here. We provide a throughout study with respect to the selection of architectures in 4.3, revealing how the architecture of f θ A (·) influences the adversarial training. As for the update rule, the parameter of f θ A (·) is typically normally pre-trained in advance and fixed during adversarial training, and only the parameter of f θ T (·) is updated. Note that the selected temperature τ A does not to be necessary the same as τ s in equation 1, and we provide a comprehensive study with respect to τ s in 4.3. Once upon finishing adversarial training, the nasty teacher f θ T (·) could be released within defense of KD-based model stealing.

NASTY TEACHER ON STANDARD KNOWLEDGE DISTILLATION
To evaluate the effectiveness of our nasty teachers, we firstly execute self-undermining training to create nasty teachers based on equation 2. Then, given an arbitrary student network, we evaluate the performance of nasty teachers by carrying out knowledge distillation from equation 1 to recognize how much nasty teachers go against KD-based model stealing.

EXPERIMENTAL SETUP
Network. We explore the effectiveness of our nasty teachers on three representative datasets, i.e., CIFAR-10, CIFAR-100, and Tiny-ImageNet. Firstly, we consider ResNet-18 (teacher network) and 5-layer plain CNN (student network) as our baseline experiment in CIFAR-10, and replace the student network with two simplified ResNets designed for CIFAR-10 (He et al., 2016), i.e., ResNetC-20 and ResNetC-32, to explore the degree of impact with respect to the capacity of student networks. For both CIFAR-100 and Tiny-ImageNet, we follow the similar setting in (Yuan et al., 2020), where three networks from ResNet family, i.e., ResNet-18, ResNet-50 and ResNeXt-29, are considered as teacher networks, and three widely used light-weight networks, i.e., MobileNetV2 (Sandler et al., 2018), ShuffleNetV2 (Ma et al., 2018) and ResNet-18, are served as student networks. Following the "self-KD" setting in (Yuan et al., 2020), an additional comparative experiment, dubbed "Teacher Self", is provided, where the architectures of the student and the teacher are set to be identical.
Training. The distilling temperature τ A for self-undermining training is set to 4 for CIFAR-10 and 20 for both CIFAR-100 and Tiny-ImageNet as suggested in (Yuan et al., 2020). For the selection of ω, 0.004, 0.005, and 0.01 are picked for CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively. For the plain CNN, we train it with a learning rate of 1e−3 for 100 epochs and optimize it by Adam optimizer (Kingma & Ba, 2014). Other networks are optimized by SGD optimizer with momentum 0.9 and weight decay 5e−4. The learning rate is initialized as 0.1. Networks are trained by 160 epochs with learning rate decayed by a factor of 10 at the 80th and 120th epoch for CIFAR-10, and 200 epochs with learning rate decayed by a factor of 5 at the 60th, 120th and 160th epoch for CIFAR-100 and Tiny-ImageNet. Without specifically mentioned, the temperature τ s is the same as τ A , which is used for self-undermining training.

EXPERIMENTAL RESULTS
Our experimental results on CIFAR-10, CIFAR-100, and Tiny-ImageNet are presented in Table 1,  Table 2 and Table 3, respectively. Firstly, we observe that all nasty teachers still perform similarly as their normal counterparts by at most ∼ 2% accuracy drop. Secondly, in contrast to normally trained teacher networks, from which the accuracy of student networks could be boosted by at most ∼ 4% by distilling, no student network can benefit from distilling the knowledge of nasty teachers. It indicates that our nasty teachers could successfully provide a false sense of generalization to student networks, resulting in decreases of accuracy by 1.72% to 67.57%. We also notice that weak student networks (e.g. MobilenetV2) could be much more poisoned from distilling toxic knowledge than stronger networks (e.g., ResNet-18), since light-weight networks intend to rely more on the guidance from teachers. It terms out that KD-based model stealing is no longer practical if the released model is "nasty" in this sense. Additional experiments, dubbed "Teacher Self", are also provided here. Opposite to the conclusion drew in (Yuan et al., 2020), the student network still cannot be profited from the teacher network even if their architectures are exactly the same. The aforementioned experiments have justified the efficacy of the proposed self-undermining training.

QUALITATIVE ANALYSIS
We present visualizations of our nasty ResNet-18 and the normal ResNet-18 on CIFAR-10 to qualitatively analyze the behavior of nasty teachers. Figure 1 visualizes the logit responses of normal ResNet-18 and its nasty counterpart after "softmax temperature" function. We notice that the logit response of nasty ResNet-18 consists of multiple peaks, where normal ResNet-18 consistently outputs a single peak. We observe that the class-wise multi-peak responses might be un-correlated to the sense of generalization. For instance, the class-wise output of bird and dog might both respond actively, and it will give a false sense of generalization to student networks. We hypothesize that the multi-peak logits misleads the learning from the knowledge distillation and degrades the performance of students.
We also present the visualizations of t-Distributed Stochastic Neighbor Embedding (t-SNE) for both feature embeddings and output logits, as illustrated in Figure 2. It is observed that the feature-space inter-class distance of nasty ResNet-18 and the normal ResNet-18 behaves similarly, which aligns with our goal that nasty teachers should perform similarly to their normal counterparts. Meanwhile, we also observe that the logit response of nasty ResNet-18 has been heavily shifted, and it entails that our method mainly modifies the weights in the final fully-connected layer.

ABLATION STUDY
Adversarial Network. Instead of choosing the same architecture for both nasty teacher and adversarial network, we vary the architecture of the adversarial network to measure the consequent influence with respect to different network structures. As illustrated in Table 4, our training method shows the generality to various architectures (e.g., etc), and we also notice that weak networks (e.g., Plain CNN) might lead to less effective nasty teachers. However, although stronger networks contribute to more effective nasty teachers, we observe that the tradeoff accuracy is saturated quickly and converges to "self-undermining" ones. Thus, we consider the "self-undermining" training as a convenient fallback choice. Students Network. In practice, the owners of teacher networks have no knowledge of student's architecture, and it is possible that the student is more complicated than the teacher. As the Reversed KD in Yuan et al. (2020), the superior network can also be enhanced by learning from a weak network. To explore the generalization ability of our method, we further conduct experiments on the reversed KD. In detail, we consider the ResNet-18 as teacher, and ResNet-50 and ResNeX29 as students. From Table 5, these two sophisticated students can still be slightly improved by distilling from a normal ResNet-18 in most cases, while be degraded by distilling from a nasty ResNet-18. This implies that our method is also effective for the reversed KD setting. Weight ω. As illustrated in Figure 3, we vary the weight ω from 0 to 0.1 on CIFAR-10 and from 0 to0.01 on CIFAR-100. We show that nasty teachers can degrade the performance of student networks no matter what ω is selected. By adjusting ω, we could also control the trade-off between performance suffering and nasty behavior. In other words, a more toxic nasty teacher could be learned by picking a larger ω at the expense of more accuracy loss.
Temperature τ s . By default, the τ s , used for knowledge distillation, is the same as τ A , used in the self-undermining training. To explore the impact of temperature selection, we vary the temperature τ s from 1 to 20, and Figure 4 presents the accuracy of students after knowledge distillation with the given τ s . We show that nasty teachers could always degrade the performance of students no matter what τ s is picked. Generally, with a larger τ s , the performance of student networks would be degraded more by the nasty teacher since a larger temperature τ s usually lead to more noisy logit outputs. Note that our nasty teacher is still effective even if the student directly distills knowledge from probabilities (τ s =1). Balance Factor α. We by default set α to 0.9 as the common practice in (Hinton et al., 2015).
To explore the effect of α, we conduct the experiments by varying α from 0.1 to 1.0, and the experimental results were summarized in Figure 5(a). In general, our nasty teachers could degrade the performance of student networks regardless of what α is selected. We also observe that a small α can help student networks perform relatively better when distilling from the nasty teacher. However, a small α also makes the student depend less on the teacher's knowledge and therefore benefit less from KD itself. Therefore, the student cannot easily get rid of the nasty defense while still mincing effectively through KD, by simply tuning α smaller.   Figure 5(b), comparing to normally trained teachers, the nasty teachers still consistently contribute negatively to student networks.

NASTY TEACHER ON DATA-FREE KNOWLEDGE DISTILLATION
Instead of getting full access to all training samples, KD without accessing any training sample is considered a more realistic way to practice model stealing. To reflect the practical behavior of stealers, we evaluate our nasty teachers on two state-of-the-art data-free knowledge distillation methods, i.e., DAFL  and DeepInversion (Yin et al., 2020), where students only have access to the probabilities produced by teacher networks. For a fair comparison, we strictly follow the setting in DAFL and adopt ResNet-34 and ResNet-18 as the teacher-student pair for further knowledge distillation. The experiments are conducted on both CIFAR-10 and CIFAR-100, where nasty ResNet-34 is trained by respectively setting ω and τ A as 0.04 and 4. The experimental results with regard to DAFL are summarized in Table 6. We show that the nasty ResNet-34 largely detriments the accuracy of student networks by more than 5%, in contrast to that under the supervision of normal ResNet-34. Based on DeepInversion, we also present the visualizations in Figure 6, where the images are generated by reverse engineering both nasty and normal ResNet-34. We demonstrate that images generated from normal ResNet-34 enable high visual fidelity, while images from nasty ResNet-34 consist of distorted noises and even false category-wise features. This visualization showcases how nasty teachers prevent illegal data reconstruction from reverse engineering.

DISCUSSION
In practice, the owner of IP can release their sophisticated network defended by self-undermining training, at the cost of acceptable accuracy loss. As the aforementioned experiments show, even if a third-party company owns the same training data, they are not able to leverage knowledge distillation to clone the ability of the released model, since the performance of theirs would be heavily degraded, instead of being boosted as usual. Furthermore, we also show that stealers would suffer more than a 5% drop of accuracy if data-free knowledge distillation is performed, of which performance drop is not acceptable in highly security-demanding applications, such as autonomous driving.
To sum up, our self-undermining training serves as a general approach to avoid unwanted cloning or illegal stealing, from both standard KD and data-free KD. We consider our work as a first step to open the door to preventing the machine learning IP leaking, and we believe that more future work needs to be done beyond this current work.

CONCLUSION
In this work, we propose the concept of Nasty Teacher: a specially trained teacher network that performs nearly the same as a normal one but significantly degrades the performance of student models that distill knowledge from it. Extensive experiments on several datasets quantitatively demonstrate that our nasty teachers are effective under either the setting of standard knowledge distillation or data-free one. We also present qualitative analyses by both visualizing the output of feature embedding and logits response. In the future, we will seek other possibilities to enlarge the current gap so that the proposed concept could be generally applied in practice, for which our current work just lays the first cornerstone.