Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

In learning with noisy labels, the sample selection approach is very popular, which regards small-loss data as correctly labeled during training. However, losses are generated on-the-fly based on the model being trained with noisy labels, and thus large-loss data are likely but not certainly to be incorrect. There are actually two possibilities of a large-loss data point: (a) it is mislabeled, and then its loss decreases slower than other data, since deep neural networks"learn patterns first"; (b) it belongs to an underrepresented group of data and has not been selected yet. In this paper, we incorporate the uncertainty of losses by adopting interval estimation instead of point estimation of losses, where lower bounds of the confidence intervals of losses derived from distribution-free concentration inequalities, but not losses themselves, are used for sample selection. In this way, we also give large-loss but less selected data a try; then, we can better distinguish between the cases (a) and (b) by seeing if the losses effectively decrease with the uncertainty after the try. As a result, we can better explore underrepresented data that are correctly labeled but seem to be mislabeled at first glance. Experiments demonstrate that the proposed method is superior to baselines and robust to a broad range of label noise types.

Introduction
Learning with noisy labels is one of the most challenging problems in weakly-supervised learning, since noisy labels are ubiquitous in the real world [40,74,45,1,70]. For instance, both crowdsourcing and web crawling yield large numbers of noisy labels everyday [15]. Noisy labels can severely impair the performance of deep neural networks with strong memorization capacities [76,78,49,34].
To reduce the in uence of noisy labels, a lot of approaches have been recently proposed [43,33,35,77,80,63,64,54,37,28,38,55,69,57,22,20,17]. They can be generally divided into two main categories. The rst one is to estimate the noise transition matrix [47,52,18,14], which denotes the probabilities that clean labels ip into noisy labels. However, the noise transition matrix is hard to be estimated accurately, especially when the number of classes is large [74]. The second approach is sample selection, which is our focus in this paper. This approach is based on selecting possibly clean examples from a mini-batch for training [15,71,58,74,26,58,59]. Intuitively, if we can exploit less noisy data for network parameter updates, the network will be more robust.
A major question in sample selection is what criteria can be used to select possibly clean examples. At the present stage, the selection based on the small-loss criteria is the most common method, and has been veri ed to be e ective in many circumstances [15,19,74,60,71]. Specically, since deep networks learn patterns rst [2], they would rst memorize training data of clean labels and then those of noisy labels with the assumption that clean labels are of the majority in a noisy class. Small-loss examples can thus be regarded as clean examples with high probability. Therefore, in each iteration, prior methods [15,60] select the small-loss examples based on the predictions of the current network for robust training.
However, such a selection procedure is debatable, since it arguably does not consider uncertainty in selection. The uncertainty comes from two aspects. First, this procedure has uncertainty about small-loss examples. Speci cally, the procedure uses limited time intervals and only exploits the losses provided by the current predictions. For this reason, the estimation for the noisy class posterior is unstable [72], which causes the network predictions to be equally unstable. It thus takes huge risks to only use losses provided by the current predictions ( Figure 1, left). Once wrong selection is made, the inferiority of accumulated errors will arise [74]. Second, this procedure has uncertainty about large-loss examples. To be speci c, deep networks learn easy examples at the beginning of training, but ignore some clean examples with large losses. Nevertheless, such examples are always critical for generalization. For instance, when learning with imbalanced data, distinguishing the examples with non-dominant labels are more pivotal during training [39]. Deep networks often give large losses to such examples (Figure 1, right). Therefore, when learning under the realistic scenes, e.g., learning with noisy imbalanced data, prior sample selection methods cannot address such an issue well.
To relieve the above issues, we study the uncertainty of losses in the sample selection procedure to combat noisy labels. To reduce the uncertainty of small-loss examples, we extend time intervals and utilize the mean of training losses at di erent training iterations. In consideration of the bad in uence of mislabeled data on training losses, we build two robust mean estimators from the perspectives of soft truncation and hard truncation w.r.t. the truncation level, respectively. Soft truncation makes the mean estimation more robust by holistically changing the behavior of losses. Hard truncation makes the mean estimation more robust by locally removing outliers from losses. To reduce the uncertainty of large-loss examples, we encourage networks to pick the sample that  (Epochs 1 and 2), due to the instability of the current prediction, the network gives a larger loss to the clean example and does not select it for updates. If we consider the mean of training losses at di erent epochs, the clean example can be equipped with a smaller loss and then selected for updates. Right: uncertainty of large-loss examples. Since the deep network learns easy examples at the beginning of training, it gives a large loss to clean imbalanced data with non-dominant labels, which causes such data unable to be selected and severely in uence generalization.
has not been selected in a conservative way. Furthermore, to address the two issues simultaneously, we derive concentration inequalities [5] for robust mean estimation and further employ statistical con dence bounds [3] to consider the number of times an example was selected during training.
The study of uncertainty of losses in learning with noisy labels can be justi ed as follows. In statistical learning, it is known that uncertainty is related to the quality of data [56]. Philosophically, we need variety decrease for selected data and variety search for unselected data, which share a common objective, i.e., reduce the uncertainty of data to improve generalization [42]. This is our original intention, since noisy labels could bring more uncertainty because of the low quality of noisy data. Nevertheless, due to the harm of noisy labels for generalization, we need to strike a good balance between variety decrease and search. Technically, our method is specially designed for handling noisy labels, which robustly uses network predictions and conservatively seeks less selected examples meanwhile to reduce the uncertainty of losses and then generalize well.
Before delving into details, we clearly emphasize our contributions in two folds. First, we reveal prior sample selection criteria in learning with noisy labels have some potential weaknesses and discuss them in detail. The new selection criteria are then proposed with detailed theoretical analyses. Second, we experimentally validate the proposed method on both synthetic noisy balanced/imbalanced datasets and real-world noisy datasets, on which it achieves superior robustness compared with the state-of-the-art methods in learning with noisy labels. The rest of the paper is organized as follows. In Section 2, we propose our robust learning paradigm step by step. Experimental results are discussed in Section 3. The conclusion is given in Section 4.

Method
In this section, we rst introduce the problem setting and some background (Section 2.1). Then we discuss how to exploit training losses at di erent iterations (Section 2.2). Finally, we introduce the proposed method, which exploits training losses at di erent iterations more robustly and encourages networks to pick the sample that is less selected but could be correctly labeled (Section 2.3).

Preliminaries
Let X and Y be the input and output spaces. Consider a k-class classi cation problem, i.e., Y = [k], where [k] = {1, . . . , k}. In learning with noisy labels, the training data are all sampled from a corrupted distribution on X × Y. We are given a sample with noisy labels, i.e.,S = {(x,ỹ)}, whereỹ is the noisy label. The aim is to learn a robust classi er that could assign clean labels to test data by only exploiting a training sample with noisy labels.
Let f : X → R k be the classi er with learnable parameters w. At the i-th iteration during training, the parameters of the classi er f can be denoted as w i . Let : R k × Y → R be a surrogate loss function for k-class classi cation. We exploit the softmax cross entropy loss in this paper. Given an arbitrary training example (x,ỹ), at the i-th iteration, we can obtain a loss i , i.e., i = (f (w i ; x),ỹ). Hence, until the t-th iteration, we can obtain a training loss set L t about the example (x,ỹ), i.e., L t = { 1 , . . . , t }.
In this paper, we assume that the training losses in L t conform to a Markov process, which is to represent a changing system under the assumption that future states only depend on the current state (the Markov property) [51]. More speci cally, at the i-th iteration, if we exploit an optimization algorithm for parameter updates (e.g., the stochastic gradient descent algorithm [4]) and omit other dependencies (e.g.,S), we will have P (w i |w i−1 , . . . , w 0 ) = P (w i |w i−1 ), which means that the future state of the classi er f only depends on the current state. Furthermore, given a training example and the parameters of the classi er f , we can determine the loss of the training example as discussed. Therefore, the training losses in L t will also conform to a Markov process.

Extended Time Intervals
As limited time interval cannot address the instability issue of the estimation for the noisy class posterior well [49], we extend time intervals and exploit the training losses at di erent training iterations for sample selection. One straightforward idea is to use the mean of training losses at di erent training iterations. Hence, the selection criterion could bẽ It is intuitive and reasonable to use such a selection criterion for sample selection, since the operation of averaging can mitigate the risks caused by the unstable estimation for the noisy class posterior, following better generalization. Nevertheless, such a method could arguably achieve suboptimal classi cation performance for learning with noisy labels. The main reason is that, due to the great harm of mislabeled data, part of training losses are with too large uncertainty and could be seen as outliers. Therefore, it could be biased to use the mean of training losses consisting of such outliers [12], which further in uences sample selection. More evaluations for our claims are provided in Section 3.

Robust Mean Estimation and Conservative Search
We extend time intervals and meanwhile exploit the training losses at di erent training iterations more robustly. Speci cally, we build two robust mean estimators from the perspectives of soft truncation and hard truncation [7]. Note that for speci c tasks, it is feasible to decide the types of robust mean estimation with statistical tests based on some assumptions [8]. We leave the analysis as future work. Two distribution-free robust mean estimators are introduced as follows. Soft truncation. We extend a classical M-estimator from [7] and exploit the widest possible choice of the in uence function. More speci cally, give a random variable X, let us consider a non-decreasing in uence function ψ : R → R such that ψ(X) = log(1 + X + X 2 /2), X ≥ 0. ( The choice of ψ is inspired by the Taylor expansion of the exponential function, which can make the estimation results more robust by reducing the side e ect of extremum holistically. The illustration for this in uence function is provided in Appendix A.1. For our task, given the observations on training losses, i.e., L t = { 1 , . . . , t }, we estimate the mean robustly as follows: We term the above robust mean estimator (3) the soft estimator. Hard truncation. We propose a new robust mean estimator based on hard truncation. Specifically, given the observations on training losses L t , we rst exploit the K-nearest neighbor (KNN) algorithm [30] to remove some underlying outliers in L t . The number of outliers is denoted by t o (t o < t), which can be adaptively determined as discussed in [79]. Note that we can also employ other algorithms, e.g., principal component analysis [53] and the local outlier factor [6], to identify underlying outliers in L t . The main reason we employ KNN is because of its relatively low computation costs [79].
The truncated loss observations on training losses are denoted by L t−to . We then utilize L t−to for the mean estimation. As the potential outliers are removed with high probability, the robustness of the estimation results will be enhanced. We denote such an estimated mean asμ h . We haveμ The corresponding estimator (4) is termed the hard estimator.
We derive concentration inequalities for the soft and hard estimators respectively. The search strategy for less selected examples and overall selection criterion are then provided. Note that we do not need to explicitly quantify the mean of training losses. We only need to sort the training examples based on the proposed selection criterion and then use the selected examples for robust training.
Proof can be found in Appendix A.1.
Theorem 2. Let Z n = {z 1 , . . . , z n } be a (not necessarily time homogeneous) Markov chain with mean µ z , taking values in a Polish state space Λ 1 × . . . × Λ n , and with a minimal mixing time τ min . The truncated set with hard truncation is denoted by Z no , with n o < n. If |z i | is upper bounded by Z. For any 1 > 0 and 2 > 0, we have with probability at least 1 − 1 − 2 .
Proof can be found in Appendix A.2. For our task, let the training loss be upper-bounded by L. The value of L can be determined easily by training networks on noisy datasets and observing the loss distribution [1].
Conservative search and selection criteria. In this paper, we will use the concentration inequalities (5) and (6)  Denote the number of times one example was selected by n t (n t ≤ t). Let = 1 2t . For the circumstance with soft truncation, the selection criterion is Let 1 = 2 = 1 2t , for the situation with hard truncation, by rewriting (6), the selection criterion is Note that we directly replace t with n t . If an example is rarely selected during training, n t will be far less than n, which causes the lower bounds to change drastically. Hence, we do not use the mean of all training losses, but use the mean of training losses in xed-length time intervals. More details about this can be checked in Section 3. For the selection criteria (7) and (8), we can see that they consist of two terms and have one term with a minus sign. The rst term in Eq. (7) (or Eq. (8)) is to reduce the uncertainty of smallloss examples, where we use robust mean estimation on training losses. The second term, i.e., the statistical con dence bound, is to encourage the network to choose the less selected examples (with a small n t ). The two terms are constraining and balanced with σ 2 or τ min . To avoid introducing strong assumptions on the underlying distribution of losses [8], we tune σ and τ min with a noisy validation set. For the mislabeled data, although the model has high uncertainties on them (i.e., a small n t ) and tends to pick them, the over tting to the mislabeled data is harmful. Also, the mislabeled data and clean data are rather hard to distinguish in some cases as discussed. Thus, we should search underlying clean data in a conservative way. In this paper, we initialize σ and τ min with small values. This way can reduce the adverse e ects of mislabeled data and meanwhile select the clean examples with large losses, which helps generalize. More evaluations will be presented in Section 3.
The overall procedure of the proposed method, which combats noisy labels by concerning uncertainty (CNLCU), is provided in Algorithm 1. CNLCU works in a mini-batch manner since all deep learning training methods are based on stochastic gradient descent. Following [15], we exploit two networks with parameters θ 1 and θ 2 respectively to teach each other. Speci cally, when a mini-batchS is formed (Step 3), we let two networks select a small proportion of examples in this mini-batch with Eq. (7) or (8) (Step 4 and Step 5). The number of instances is controlled by the function R(T ), and two networks only select R(T ) percentage of examples out of the minibatch. The value of R(T ) should be larger at the beginning of training, and be smaller when the number of epochs goes large, which can make better use of memorization e ects of deep networks [15] for sample selection. Then, the selected instances are fed into its peer network for parameter updates (Step 6 and Step 7).

Experiments
In this section, we evaluate the robustness of our proposed method to noisy labels with comprehensive experiments on the synthetic balanced noisy datasets (Section 3.1), synthetic imbalanced noisy datasets (Section 3.2), and real-world noisy dataset (Section 3.3).

Experiments on Synthetic Balanced Noisy Datasets
Datasets. We verify the e ectiveness of our method on the manually corrupted version of the following datasets: MNIST [25], F-MNIST [66], CIFAR-10 [24], and CIFAR-100 [24], because these datasets are popularly used for the evaluation of learning with noisy labels in the literature [15,74,62,26]. The four datasets are class-balanced. The important statistics of the used synthetic datasets are summarized in Appendix B.1.
Baselines. We compare the proposed method (Algorithm 1) with following methods which focus on sample selection, and implement all methods with default parameters by PyTorch, and conduct all the experiments on NVIDIA Titan Xp GPUs. (1). S2E [71], which properly controls the sample selection process so that deep networks can better bene t from the memorization effects. (2). MentorNet [19], which learns a curriculum to lter out noisy data. We use self-paced MentorNet in this paper. (3). Co-teaching [15], which trains two networks simultaneously and cross-updates parameters of peer networks. (4). SIGUA [16], which exploits stochastic integrated gradient underweighted ascent to handle noisy labels. We use self-teaching SIGUA in this paper. (5). JoCor [60], which reduces the diversity of networks to improve robustness. Other types of baselines such as adding regularization are provided in Appendix B.2. Note that we do not compare the proposed method with some state-of-the-art methods, e.g., SELF [44] and DivideMix [27]. It is because their proposed methods are aggregations of multiple techniques. We mainly focus on sample selectionin in learning with noisy labels. Therefore, the comparison is not fair. Here, we term our methods with soft truncation and hard truncation as CNLCU-S and CNLCU-H respectively.
Network structure and optimizer. For MNIST, F-MNIST, and CIFAR-10, we use a 9-layer CNN structure from [15]. Due to the limited space, the experimental details on CIFAR-100 are provided in Appendix B.3. All network structures we used here are standard test beds for weaklysupervised learning. For all experiments, the Adam optimizer [23] (momentum=0.9) is used with an initial learning rate of 0.001, and the batch size is set to 128 and we run 200 epochs. We linearly decay learning rate to zero from 80 to 200 epochs as did in [15]. We take two networks with the same architecture but di erent initializations as two classi ers as did in [15,74,60], since even with the same network and optimization method, di erent initializations can lead to di erent local optimal [15]. The details of network structures can be checked in Appendix C.
For the hyper-parameters σ 2 and τ min , we determine them in the range {10 −1 , 10 −2 , 10 −3 , 10 −4 } with a noisy validation set. Here, we assume the noise level τ is known and set R(T ) = 1 − min{ T T k τ, τ } with T k =10. If τ is not known in advanced, it can be inferred using validation sets [33,75]. As for performance measurement, we use test accuracy, i.e., test accuracy = (# of correct prediction) / (# of testing). All experiments are repeated ve times. We report the mean and standard deviation of experimental results.
Experimental results. The experimental results about test accuracy are provided in Table  1, 2, and 3. Speci cally, for MNIST, as can be seen, our proposed methods, i.e., CNLCU-S and CNLCU-H, produce the best results in the vast majority of cases. In some cases such as asymmetric noise, the baseline S2E outperforms ours, which bene ts the accurate estimation for the number of selected small-loss examples. For F-MNIST, the training data becomes complicated. S2E cannot achieve the accurate estimation in such situation and thus has no great performance like it got on MNIST. Our methods achieve varying degrees of lead over baselines. For CIFAR-10, our methods once again outperforms all the baseline methods. Although some baseline, e.g., Co-teaching, can work well in some cases, experimental results show that it cannot handle various noise types. In contrast, the proposed methods achieve superior robustness against broad noise types. The results mean that our methods can be better applied to actual scenarios, where the noise is diversiform.
Ablation study. We rst conduct the ablation study to analyze the sensitivity of the length of time intervals. In order to avoid too dense gures, we exploit MNIST and F-MNIST with the    Table 3: Test accuracy (%) on CIFAR-10 over the last ten epochs. The best two results are in bold.

Experiments on Synthetic Imbalanced Noisy Datasets
Experimental setup. We exploit MNIST and F-MNIST. For these two datasets, we reduce the number of training examples along with the labels from "0" to "4" to 1% of previous numbers. We term such synthetic imbalanced noisy datasets as IM-MNIST and IM-F-MNIST respectively. This setting aims to simulate the extremely imbalanced circumstance, which is common in practice. Moreover, we exploit asymmetric noise, since these types of noise can produce more imbalanced case [47,36]. Other settings such as the network structure and optimizer are the same as those in experiments on synthetic balanced noisy datasets. As for performance measurements, we use test accuracy. In addition, we exploit the selected ratio of training examples with the imbalanced classes, i.e., selected ratio=(# of selected imbalanced labels / # of all selected labels). Intuitively, a higher selected ratio means the proposed method can make better use of training examples with the imbalanced classes, following better generalization [21].
Experimental results. The test accuracy achieved on IM-MNIST and IM-F-MNIST is presented in Figure 2. Recall the experimental results in Table 1 and 2, we can see that the imbalanced issue is catastrophic to the sample selection approach when learning with noisy labels. For IM-MNIST, as can be seen, all the baselines have serious over tting in the early stages of training. The curves of test accuracy drop dramatically. As a comparison, the proposed CNLCU-S and CNLCU-H can give a try to large-loss but less selected data which are possible to be clean but equipped with imbalanced labels. Therefore, our methods always outperform baselines clearly. In the case of Asym. 10%, our methods achieve nearly 30% lead over baselines. For IM-F-MNIST, we can also see that our methods perform well and always achieve about 5% lead over all the baselines. Note that due to the huge challenge of this task, some baseline, e.g., S2E, has a large error bar. In addition, 16 18 20 22 24 Time Interval  Table 4. The results explain well why our methods perform better on synthetic imbalanced noisy datasets, i.e., our methods can make better use of training examples with the imbalanced classes. Note that since we give a try to large-loss but less selected data in a conservative way, the selected ratio is still far away from the class prior probability on the test set, i.e., 10%. However, a little improvement of the selection ratio can bring a considerable improvement of test accuracy. These results tell us that, in the sample selection approach when learning with noisy labels, improving the selected ratio of training examples with the imbalanced classes is challenging but promising for generalization. This practical problem deserves to be studied in depth. Dataset

Experiments on Real-world Noisy Datasets
Experimental setup. To verify the e cacy of our methods in the real-world scenario, we conduct experiments on the noisy dataset Clothing1M [67]. Speci cally, for experiments on Clothing1M, we use the 1M images with noisy labels for training and 10k clean data for test respectively. Note that we do not use the 50k clean training data in all the experiments. For preprocessing, we resize the image to 256×256, crop the middle 224×224 as input, and perform normalization. The experiments on Clothing1M are performed once due to the huge computational cost. We leave 10% noisy training data as a validation set for model selection. Note that we do not exploit the resampling trick during training [27]. Here, Best denotes the test accuracy of the epoch where the validation accuracy was optimal. Last denotes test accuracy of the last epoch. For the experiments on Clothing1M, we use a ResNet-18 pretrained on ImageNet as did in [60]. We also use the Adam optimizer and set the batch size to 64. During the training stage, we run 15 epochs in total and set the learning rate 8 × 10 −4 , 5 × 10 −4 , and 5 × 10 −5 for 5 epochs each. Experimental results. The results on Clothing1M are provided in Table 5. Speci cally, the proposed methods get better results than state-of-the-art methods on Best, which achieve an improvement of +1.28% and +0.99% over the best baseline JoCor. Likewise, the proposed methods outperform all the baselines on Last. We achieve an improvement of +1.01% and +0.54% over Jo-Cor. Note that the results are a bit lower than some state-of-art methods, e.g., [73] and [54], because of the following reasons. (1). We follow [60] and use ResNet-18 as a backbone. The state-of-art methods [73,54] use ResNet-50 as a backbone. Our aim is to make the experimental results directly comparable with previous papers [60] in the same area. (2). We only focus on the sample selection approach and do not employ other advanced techniques, e.g., introducing the prior distribution [54] and combining semi-supervised learning [27,44,32].

Conclusion
In this paper, we focus on promoting the prior sample selection in learning with noisy labels, which starts from concerning the uncertainty of losses during training. We robustly use the training losses at di erent iterations to reduce the uncertainty of small-loss examples, and adopt con dence interval estimation to reduce the uncertainty of large-loss examples. Experiments are conducted on benchmark datasets, demonstrating the e ectiveness of our method. We believe that this paper opens up new possibilities in the topics of using sample selection to handle noisy labels, especially in improving the robustness of models on imbalanced noisy datasets.

A.1 Proof of Theorem 1
For the circumstance with soft truncation,μ z = 1 n n i=1 ψ(z i ). As suggested in [7], we can exploit to derive a bound forμ z . For some positive real parameter α, we de ne Let us introduce the quantity With the exponential moment inequality [13] and the C r inequality [41], we have In the same way, If we de ne for any µ s ∈ R the bounds and From [9] (Lemma 2.2), we obtain that P (r(θ) > B − (θ)) ≥ 1 − and P (r(θ) < B + (θ)) ≥ 1 − .
Letμ − z be the largest solution of the quadratic equation B − (θ) andμ + z be the smallest solution of the quadratic equation B + (θ). Also, to guarantee the solution of the quadratic equation, we assume From [9] (Theorem 2.6), we then havẽ With probability at least 1-2 , we haveμ − z ≤μ z ≤μ + z . We can choose α = n σ 2 . Then we have which holds with probability at least 1-2 .
We exploit the lower bound and let = 1 2t . Then we have where n t denotes the number of times that the example was selected in the time intervals. Here, we provide the graph of the used in uence function for the soft estimator, which explains the mechanism of the function y = log(1 + x + x 2 /2) more clearly. The illustration is presented in Figure 3. As can be seen, when x is large and may be an outlier, the in uence function can reduce its negative impact for mean estimation. Therefore, we exploit such an in uence function for robust mean estimation, which brings better classi cation performance.
A.2 Proof of Theorem 2 Lemma 1 ([48]). Let Z n = {z 1 , . . . , z n } be a (not necessarily time homogeneous) Markov chain with mean µ z , taking values in a Polish state space Λ 1 × . . . × Λ n , with a mixing time τ (υ) (for 0 ≤ υ ≤ 1). Let For some η ∈ R + , suppose that f : Λ → R satis es the following inequality: for every a, b ∈ Λ. Then for any ≥ 0, we have The detailed de nition of the mixing time for the Markov chain can be found in [48,50]. Let f be the mean function. Following the prior work on mean estimation [31,12,11,46], without loss of generality, we assume µ z = 0 for the underlying true distribution, and |z i | is upper bounded by Z. Then we can set η to 4Z/n for Eq. (23). Combining the above analyses, we can revise Eq. (24) as follows: and P max i∈[n] for 1 > 0 and 2 > 0. If we remove the potential outliers Z no from Z n . Therefore, we have which holds with probability at least 1 − 1 − 2 . For our task, we exploit the concentration inequality. Let 1 = 2 = 1 2t , and the losses be bounded by L. Next we can obtain with the probability at least 1 − 1 t . In practice, it is easy to identify the value of L. For example, we can training deep networks on noisy datasets to observe the loss distributions. Then, we exploit the lower bound such that for sample selection.

B.1 The Details of Datasets and Generating Noisy Labels
For the details of datasets, the important statistics of the used datasets are summarized in Table 6. For the details of generating noisy labels, we exploit both class-dependent and instance-dependent label noise which include ve types of synthetic label noise to verify the e ectiveness of the proposed method. Here, we describe the details of the noise setting as follows: (1). Class-dependent label noise: • Symmetric noise: this kind of label noise is generated by ipping labels in each class uniformly to incorrect labels of other classes.
• Pair ip noise: the noise ips each class to its adjacent class.
• Tridiagonal noise: the noise corresponds to a spectral of classes where adjacent classes are easier to be mutually mislabeled, unlike the unidirectional pair ipping. It can be implemented by two consecutive pair ipping transformations in the opposite direction.    Table 8: Test accuracy on F-MNIST over the last ten epochs.
(2). Instance-dependent label noise: • Instance noise: the noise is quite realistic, where the probability that an instance is mislabeled depends on its features. We generate this type of label noise to validate the e ectiveness of the proposed method as did in [64].
We use synthetic noisy MNIST as an example and plot the noise transition matrices in Figure  4. The noise rate is set to 20%.

B.2 Comparison with Other Types of Baselines
As we focus on the sample selection approach in learning with noisy labels, in the main paper (Section 3.1), we fairly compare our methods with the baselines which also focus on sample selection. Here, we evaluate other types of baselines. We exploit APL [36] and CDR [65], which add implicit regularization from di erent perspectives. The experiments are conducted on MNIST and F-MNIST. Other experimental settings are the same as those in the main paper. The experimental results are provided in Table 7 and 8, which show that the proposed methods can outperform them with respect to classi cation performance.

B.3 Experiments on Synthetic CIFAR-100
For CIFAR-100, we use a 7-layer CNN structure from [74,71]. Other experimental settings are the same as those in the experiments on MNIST, F-MNIST, and CIFAR-10. The results are provided in Table 9. We can see the proposed method outperforms all the baselines.  Table 9: Test accuracy (%) on CIFAR-100 over the last ten epochs. The best two results are in bold.

B.4 Experiments for Ablation Study
We conduct the ablation study to analyze the sensitivity of the length of time intervals. The results are shown in Figure. 5   Note that in this paper, we concern uncertainty from two aspects, i.  Figure 6: Illustrations of the hyperparameter sensitivity for the proposed CNLCU-H. The error bar for standard deviation in each gure has been shaded.
the proposed methods successful. The experiments are conducted on MNIST and F-MNIST. Other experimental settings are the same as those in the main paper (Section 3.1). Note that we employ two networks to teach each other following [15]. Therefore, when we do not consider uncertainty in sample selection, the proposed methods will reduce to the baseline Co-teaching [15].
To study the e ect of concerning uncertainty about small-loss examples, we remove the concerns about large-loss examples, i.e., the network is not encouraged to choose the less selected examples for updates. We express such a setting as "without concerning about large-loss examples" (abbreviated as w/o cl). To study the e ect of concerning uncertainty about large-loss examples, we remove the concerns about small-loss examples, i.e., we only exploit the predictions of the current network. We express such a setting as "without concerning about small-loss examples" (abbreviated as w/o cs). Besides, we express the setting which directly uses non-robust mean as Co-teaching-M.
The experimental results of ablation study are provided in Table 10 and 11. As can be seen, both aspects of uncertainty concerns can improve the robustness of models. Therefore, combining two uncertainty concerns, we can better combat noisy labels. In addition, robust mean estimation is superior to the non-robust mean in learning with noisy labels. Table 12 describes the 9-layer CNN [15] used on MNIST, F-MNIST, and CIFAR-10. Table 13 describes the 9-layer CNN [74] used on CIFAR-100. Here, LReLU stands for Leaky ReLU [68]. The slopes of all LReLU functions in the networks are set to 0.01. Note that that the 7/9-layer CNN is a standard and common practice in weakly supervised learning. We decided to use these CNNs, since then the experimental results are directly comparable with previous approaches in the same area, i.e., learning with noisy labels.   Table 11: Test accuracy (%) on F-MNIST over last ten epochs.


extfieldsofstudy:
Computer Science
Mathematics
Engineering

s2fieldsofstudy:
Computer Science
Engineering

text:
Vulnerabilities of Power System Operations to Load Forecasting Data Injection Attacks

We study the security threats of power system operation brought by a class of data injection attacks upon load forecasting algorithms. In particular, with minimal assumptions on the knowledge and ability of the attacker, we design attack data on input features for load forecasting algorithms in a black-box approach. System operators can be oblivious of such wrong load forecasts, which lead to uneconomical or even insecure decisions in commitment and dispatch. This paper is the first attempt to bring up the security issues of load forecasting algorithms to our knowledge, and show that accurate load forecasting algorithm is not necessarily robust to malicious attacks. More severely, attackers are able to design targeted attacks on system operations strategically with additional topology information. We demonstrate the impact of load forecasting attacks on two IEEE test cases. We show our attack strategy is able to cause load shedding with high probability under various settings in the 14-bus test case, and also demonstrate system-wide threats in the 118-bus test case with limited local attacks.

I. INTRODUCTION
Load forecasting plays an important role in the planning and operations of electric grids. As a cornerstone application for utilities and operators, it provides future load information which is utilized for various decision-making problems such as unit commitment, reserve management, economic dispatch and maintenance scheduling [1]. Consequently, the accuracy of forecasted loads directly impacts the cost and reliability of system operations [2].
Because of its fundamental importance, there are always strong incentives to improve short-term forecasting methods, especially under higher penetration of renewables. The driving factors of load variations are heterogeneous, including temperature, weather, temporal and seasonal effects (e.g., weekday vs. weekend) and other socioeconomic factors. Thus, load forecasting algorithms can be regarded as finding a nonlinear and complex mapping between the (potentially high dimensional) driving factors to the forecasted time series of load values. Over the past decades, a myriad of load forecasting algorithms have been proposed and adopted. See, for example, [1], [3] and the references within. Statistical and machine learning techniques, such as support vector regression [4], ARIMA [5] and neural networks [6] have been applied to short term load forecasting and implemented in practice. The recent advances in deep learning and data sciences opened the door to utilizing more input features and deeper model architectures to further improve load forecasting accuracy and provided some of the best performances to date [7]- [9]. In most of the load Y. Chen forecasting studies, forecast accuracy has been regarded as the holy grail for researchers. and proposed black-box attacks on load forecasting (b). Without knowledge about the forecast model's parameters, the attacker injects designed small, undetectable data perturbations into weather forecasts to induce abnormal system operations.

Historical Load
However, despite the critical role of forecasting algorithms and the long pursuit of forecast accuracy, the robustness and security issues have been overlooked to some extents. As the forecasting methods become more complex, they are also more susceptible to cybersecurity threats. In the previous research on cyber-security of power systems [10], [11], where state estimation [12], [13], communication [14], and electricity market [15], [16] threats and countermeasures are rigorously evaluated. However, the vulnerabilities of load forecasting algorithms are rarely discussed [17], [18], while this does not mean load forecasting is less vulnerable nor the consequences of attacks are less severe. For instance, forecasting models normally make use of weather forecasts inputs coming from external services/APIs, while such inputs can be exposed to adversarial modification and the model performance may be severely impacted by such malicious changes. Recently, there has been a hot debate on the security of machine learning models [19], and researchers found that small noises injected to the inputs can severely impact model performances [20].
In this paper, we look into the security threats in general load forecasting algorithms. By taking the perspective of an attacker and developing attack strategies on load forecasting algorithms, we conduct damage analysis of the proposed attacks. We consider the scenario where attacker adversarially injects false data into the input features of forecasting algorithms, and examine to what extent such attacks could impact the performance of load forecasting models. Specifically, we investigate false data injection attacks on the temperature data. It is an important input to load forecasting algorithms and is mostly obtained from external services/APIs. Therefore it is easier for attackers to inject perturbations on temperature data than to attack the state estimation [13] or market clearing algorithms [15]. The potential damage of load forecasting attacks can be significant, leading to increases in system operation costs and maybe even more catastrophic events such as load shedding.
In Figure 1, we show the schematic of threats and proposed attacks to systems. Our work is different from most related work in two aspects: most of the studies in forecasting lack security and robustness considerations, while most of the studies in power system security evaluate attacks with certain level of knowledge about the targeted system or unconstrained capabilities. In contrast to previous attacks research that assume full knowledge of system configuration and strong capabilities of attackers on implementing attacks (see, e.g. [12], [13], [15]), we take a restrictive setting of both the attacker's knowledge and capabilities. Under our setup, the attacker does not need to know any parameter of the targeted load forecasting algorithms, and could only inject constrained perturbations into input temperatures to avoid detection. We develop a simple data-driven attack strategy for finding the injected perturbations onto the original temperature data. Surprisingly, we find the proposed attacks significantly degrade the performance of a class of (accurate) load forecasting algorithms. With only few degrees of perturbations injected into input temperatures, the load forecasting algorithm's output deviates drastically from original accurate forecasts.
To further illustrate security issues brought by load forecasting attacks, we embed the load forecasting algorithms into canonical power system operation case studies considering network constraints and power balance. We consider both cases when attacker possesses and does not possess additional information on system topology and parameters. For the former case where the attacker can strategically inject data perturbations under certain attack budgets, we design a greedy algorithm to compromise a subset of nodal load forecasts to cause targeted damages such as uneconomical generation, infeasible line flows and generator schedules, and load shedding. Simulations based on real-world load datasets on the IEEE 14-bus and 118-bus systems demonstrate the system operation vulnerabilities by only maliciously changing the temperature by a few degrees.
This study illustrates the need to look at other properties in addition to forecast accuracy, and the need for more comprehensive analysis when developing and applying load forecasting techniques. We demonstrate that accuracy may not mean robustness, and a wrong forecast of load potentially leads to costly operation decisions or system damage. Specifically, we make the following contributions in this work: • To the best of our knowledge, this work is the first to evaluate the security issues of load forecasting procedures in power system operations. Starting with the setup for load forecasting along with its role in power system operations (Sec. II), data vulnerabilities of current forecasting methods are formulated and discussed (Sec. III). • Black-box attack algorithm gradient estimation is proposed to generate hard-to-detect, adversarial input data for load forecasting algorithms (Sec. III). • We show that the strategically designed adversarial injections upon input features could target either increased system operating costs or load shedding. The resulting optimization formulation of the attack problem maybe of independent interest (Sec. IV). • Case studies of power system operations on standard IEEE test cases using real-world load data reveal the prevalent vulnerabilities of current forecasting techniques and demonstrate potential damages on power system operations via proposed attacks (Sec. V). Compared to our prior work [18] which works on single bus vulnerabilities analysis, we bring out both the load forecasting threats and the attack strategies on power networks. Extensive numerical simulations also verify such load forecasting vulnerabilities generally exist in power networks operations. We also make our code open source as a public package for evaluating load forecasting robustness and security 1 . Due to the space limits of submission, we refer to the preprint for more details on attack implementations and thorough tests on various load forecasting algorithms [21].

II. PRELIMINARIES
In this section, we briefly describe the notations and setup for load forecasting algorithms, and illustrate how load forecasting serves as an important component of system operation in day-ahead commitment and real-time dispatch.

A. Load Forecasting
To set up and find parameters of the short-term load forecasting algorithm for a specific region, the system operator needs to collect a training dataset D tr = {(X t−H , ..., X t ); L t+k } T t=1 based on available historical data. Here L t+k ∈ [0, 1] are scalars representing scaled nodal load values and H is the history horizon [1]. The model's forecast horizon is denoted by k and ranges from one hour to one day in short-term forecasts. X t−i ∈ [0, 1] d , i = 0, ..., H are scaled, d-dimensional input feature vectors. Feature vector X t includes historical records of load, weather forecasts including temperature, weather indicators (e.g., sunny, rainy or cloudy) and seasonal indicator variables such as weekdays/weekends and hour of the day [9]. We express it as X t := {L t , X temp forecast API; X index t are a collection of indicators including weather characteristics, seasonal factors and time factors. In the task of load forecasting, one is interested to find a function parameterized by θ: f θ (·), which learns the mapping from (X t−H , ..., X t ) to future loadsL t+k . The mean absolute error (MAE) is widely used to measure the performance of f θ , which is defined by the average L 1 norm of difference between forecasted loadsL t+k and ground truth L t+k . The schematic of load forecasting model is depicted in Figure 1(a).
We note that vulnerability analysis conducted by this paper is not constrained to certain forecasting algorithms. As long as the model output is sensitive with respect to input features, our proposed attack methods would be able to alter load patterns maliciously. For the discussion hereafter, we make use of an Recurrent Neural Networks (RNN) [6]- [9], which is a widely adopted forecast algorithm to model the temporal dependencies between feature inputs and forecasted values.

B. System Operations Model
For a N -bus power network with set of loads D and set of generators G, we consider the system operation setting consisting of a day-ahead planning stage using load forecasts and a real-time operational stage using actual load.
1) A unit commitment (UC) model considering reserve margins, startup and shutdown cost, minimum up/down time constraints and ramping constraints is used to create a commitment schedule G t , t = 1, ...T based on the dayahead load forecastsL t , t = 1, ...T ; 2) For each time t, the dispatch of the scheduled units p t and the actual dispatch costs C 1 (p t ) are calculated according to a basic Economic Dispatch (ED) model [22] based on the actual load L t and generation schedule G t .
The actual daily operation costs are calculated via summing the 24-hour dispatch costs and the startup and shutdown costs. When ED based on the day-ahead commitment does not have a feasible solution, load shedding is used to maintain the balance between supply and demand. The shedded loads LS t also incur costs C 2 (LS t ). Note that under perfect forecasts, the generation schedule G t can minimize real-time dispatch costs.

III. ATTACK STRATEGIES
In this section, we first describe the objective and constraints for implementing load forecast attacks. We then illustrate how an attacker is able to design white-box attack with known load forecasting model parameters. Finally we describe under the black-box setting, how data injection attacks can be implemented even though the attacker only has limited query access to the load forecasting model.

A. Objective of Attacker
The attacker's goal is to distort the forecasted load as much as possible in a certain direction, e.g., to either increase or decrease forecasted values. Consider the task of training an accurate load forecasting models, where estimation of θ is given by minimizing the L 1 -norm of the difference between model predictions and ground truth values: where during training, ground truth of historical records on X t and L t+k are used; during testing and real-world system implementations, we are using X t which are coming from weather forecast as input features. Once the model is learned, it can be applied in a rolling-horizon fashion.
In order to distort the output forecast values from the trained model based on (1), the attacker actually has two choices of inserting attacks: to attack X t or to attack θ. While trained model θ itself is often safely kept by the operators, system operator has to use external data such as weather forecasts X temp t as input features for f θ . This actually provides a backdoor for the attacker, whose goal is to inject perturbations into the weather forecasts coming from external services. By generating adversarial input dataX temp t for f θ (·), model predictions are modified adversarially. We use γ = {−1, 1} to denote the chosen attack direction by attackers. If γ = 1 (γ = −1), the attacker tries to findX to decrease (increase) the load forecasts values. Since load values are always positive, the attacker's goal is to findX that minimizes the value of γf θ (X t−H , ...,X t ).

B. Attacker's Knowledge
We consider two attack scenarios, white box and blackbox attacks. In the white-box settings, the attacker is assumed to know exactly the model parameters θ. This is a strong assumption in the sense that load forecast model f θ (·) is fully exposed to the attacker. On the contrary, in the blackbox setting, the attacker only knows which family of load forecasting model has been applied (e.g., NN or RNN), but is blind to the forecasting algorithms and has no knowledge of any parameters of f θ . We assume the attacker only has query access to the load forecasting model 2 . That is, the attacker could query the implemented load forecasting model by using different values of input features for a limited number of times, and then try to get insights on how f θ works.

C. Attacker's Capability
From the attacker's perspective, it is necessary to construct attack injections while avoid being detected by the system operators' bad data detection algorithms. We consider several realistic constraints for attacker's capability: it could be upper bounded by the maximum number of perturbed entries in the input data, by the average deviations on all features, or by the largest deviation from the clean data. Mathematically, the attacker wants to keep ||X temp t − X temp t || p bounded, where p can take different values such as 0, 1, ∞ to express certain norm constraints corresponding to different detection countermeasures. 2 Such query access assumption is possible in many forecast-as-a-Service businesses, e.g., SAS energy forecasting and Itron forecasting.
In summary, we formulate the model of attackers as the following optimization problem: Note that there is a parallel between the forecast problem (1) and attack problem (2), where the objective's optimization directions and optimization variables are exactly in the opposite directions: forecasting model works on model parameters to minimize forecast errors, while attacker works on model inputs to maximize the errors to targeted directions. However, due to lack of model knowledge in the black-box setting, it is a challenging task for attackers to find efficient attack input X temp via (2). In the next subsection, we will show a blackbox attack method generally working with attacker's knowledge coming from query access to the forecast algorithm.

D. Black-Box Attack
Under the case of white-box where model parameters are known to the attacker, it is possible to find the attack input via solving (2). For the convenience of notations, we omit the superscript on X in some of the following paragraphs, and introduce the generalizable attack methods not only suitable for attacking temperature forecasts, but also suitable for injecting false data into other features.
Since most state-of-the-art load forecasting algorithms use complex models such as neural networks, the attacker's problem (2) is nonconvex and furthermore, there is no closedform solution forX t−H , ...,X t . Nevertheless, an attacker can still find some attack vectors iteratively by taking gradients with respect to each time step's temperature values. Even though this may not find the optimal solution to (2), because of the highly nonconvex nature of the forecasting model, a slight (suboptimal) perturbation of the input features would drastically change the forecast output.
Based on (2), we define a loss function L with respect to each time step's featureX t−i , i = 0, ..., H. Then the attacker iteratively takes gradients of L to find the adversarial input X t−i . The constraints in (2b) is included in the loss function using a log-barrier: where β is the weight of the barrier term. Since there are a large number of parameters and input features in many load forecasting algorithms, it can be computationally expensive to compute the exact gradient values for each input feature. We follow a simpler method in [19] to only update the attack features based on the sign of the gradient at each iteration j: where α controls the step size for updating adversarial temperature values. The resulting adversarial temperature vector is obtained by applying (4) for several times.
Under the black-box setting where X (j) is not able to compute, we assume attacker is able to query the load forecasting algorithm for a limited number of times, and it is still possible to construct adversarial temperature inputs by using queries to estimate the gradients. In Figure 1(b) we show the schematic on generating adversarial temperature instances via querying. For k-th dimension of the input feature at time stamp t − i,X j+1 k,t−i , the attacker needs to query the load forecasting system on each feature entry to calculate the twosided estimation of the gradient of f θ : where e k is a d-dimensional vector with all zero except 1 at k-th component, and δ takes a small value for gradient estimation. Once the gradient is estimated for each dimension of temperature features, we can follow the same method of (4) to iteratively build the adversarial features using the estimated gradient vectors: To satisfy norm constraints on the allowed perturbation of X, the attacker projects the adversarial data back into the pre-defined norms after each iterative attack step. As shown in Figure 2, the output load forecasts deviate a lot from the ground truth values, while temperature perturbations are constrained within attacker's capability. In [23], techniques on reducing number of queries are discussed for attacking an image classifier, which may also further improve the query efficiency of load forecasting attacks.

IV. ATTACKS ON SYSTEM OPERATIONS
In this section, we illustrate the attacks on load forecasting input features could further threaten power system operations, and propose a realistic attack strategy for the attacker under attack budgets on number of compromised nodal forecasts.

A. Attack Objectives
We assume the attacker could only inject constrained attacks in the day-ahead planning stage. Under the day-ahead load forecastsL t d for |D| loads in the networks, the UC problem is to find the generation schedule and dispatch for |G| generators while satisfying reserve and system operation constraints: where δ(d) is the set of lines connected to node d; u t g is the binary decision variable of the commitment status of generator g at time t, with 1 indicating g is online; p t g is the real power output of generator g at time t; all the u t g 's and p t g 's are collected together into vectors u and p; C(p) and S(u) represent the dispatch costs and startup and shutdown costs, respectively, of all the generators in all periods; the constraints are system-wide power balance constraint (7b), generation limits constraints (7c), line flow limits (7d), power balance at each node (7e), generator logical constraint (7f), minimum up time constraint (7g), minimum down time constraint (7h) and ramping constraints (7i). We also keep a fixed reserve margin throughout the simulation. Once solved, the operator gets the schedule for the set of online generators G t at each time t.
The attacker injectsX to a group of compromised load forecasts, such that the nodal load forecasts maliciously change fromL t d toL t d . The attacker is not only constrained by the average deviations uponX (Equation (2)), but also constrained on number of compromised loads N adv that are allowed to inject perturbations throughout the day. From the attacker's perspective, it is then most harmful to find a constrained adversarial day-ahead commitment scheduleG t viaL t d , which maximizes the solution for the following real-time ED: where from the system operator's perspective, ED aims to find the real power dispatch at time t, p t , that minimize the dispatch costs at time t, C(p t ), considering system-wide power balance constraint (8b), generation limits constraints (8c), line flow limits (8d) and power balance at each node (8e).

B. Attack Strategies
Under normal operating conditions, the load forecasting algorithms provide accurate forecasts on day-ahead load for system operators to solve (7). When the system is under attack, the attacker chooses a group of load buses to inject adversarial temperature forecasts, such that generation schedule coming from day-ahead planning stage is deviating from the normal schedule. Such adversarial generation schedules are likely to cause malicious operation patterns, e.g., increased system costs, load shedding, no feasible generation dispatch or violation of ramping constraints. Essentially, the attacker wants to answer the following questions to find the attacks: • Which group of load buses should be compromised to injectX? • How to generateL t j , t = 1, ..., T for compromised load bus j, such that (8) is maximized? Under the case all system parameters are known, the attacker's optimization problem is tri-level with integer constraints, which is a very challenging problem to solve. Rather than the standard approach through KKT conditions, we design a modified greedy search algorithm for the attacker to find the most vulnerable loads that cause system-level misoperations. As described in Algorithm 1, the attacker follows a modified best-first search algorithm to implement attacks on compromised nodal forecasts iteratively [24]. For each iteration, the attacker checks the lines and generators which are approaching operating limits (e.g., line flow capacity, generation capacity), and finds the most vulnerable load j based on neighboring lines and generators. Then the attacker raisesX j andX j which maximizes and minimizes the load forecasts respectively. By solving (7) using the resultingL j andL j , the attacker checks which candidate attack is more prone to make day-ahead commitment scheduleG t different from G t and keeps it asL j in this iteration, while j is added to the set of compromised loadsL.
When the attacker does not know the parameters of underlying system such as network topology, line flow limits, each generator's capacity and ramp constraints, it is not possible for the attacker to find the optimal attacks, and the attacker just randomly chooses a set of loads to attack using eitherX j orX j for j ∈L. In the next section, we will show attacks on real-world load data upon IEEE standard systems that reveal the vulnerabilities brought by either strategic or random load forecasting attacks.

Algorithm 1 Best-First Search
Input: Forecasted loads L 1 , L 2 , ...L |D| ∈ R |D|×T Initialize: Sets of Compromised LoadsL ← ∅ Initialize: Maximum Temperature perturbations Initialize: Number of compromised loads k = 0 Solve UC under normal forecasts while G t ==G t , t = 1, ..., T and k ≤ N adv do Find the most vulnerable node j and attack direction Gradient Estimation attacks to findL j L ← j Solve (7) to getG t , k+ = 1 end while Solve (8) in real-time withG t

V. CASE STUDIES
In this section, we show a detailed simulation on real-world Swiss load data, and show the threats posed by our black-box data injection attacks on both the load forecasting algorithm itself and the power system operations. Thorough evaluations on 14-bus system indicate that proposed attacks cause load shedding with very high probability, while tests on 118-bus system indicate by compromising a small portion of nodal load forecasts, the attacker can cause security threats over the whole networks.

A. Experimental Setup
Dataset Description: We collected and queried hourly Swiss actual load data from European Network of Transmission System Operators for Electricity(ENTSO-E)'s API 3 ranging from Jan 1st, 2015 to May 16th, 2017. The nominal load values are in the range of [6, 500M W, 9, 500M W ]. We followed [25] to collect day-ahead historical weather forecasts coming from major cities in Switzerland such as Zurich, Basel, Lucerne and etc. All the weather data were queried from Dark Sky API 4 . We also collected other indicator features X index , such as one-hot vectors of hour of day, weekdays and seasons. We evaluated the attack threats on split-out test data, and the forecast model parameters are kept away from the attacker throughout the black-box simulations.
Power Systems Setup: We set up the IEEE 14-bus and IEEE 118-bus to study the system vulnerabilities brought by load forecasting attacks [26]. The grid has a total capacity of 15, 500M W and 14, 949.3M W respectively, which are both over 1.5 times of yearly peak load. We set the spinning reserve requirement as 3% of the total forecasted demand based on [27]. The models of UC and ED are implemented in Python using PyPSA [28], and these two modules are directly interfaced with the load forecasting and attack algorithms using Tensorflow [29]. No load shedding occurs when clean load forecasts are used.
Model Training and Attack Implementation: We set up an RNN with 3 layers for day-ahead load forecasting, and use standard stochastic gradient descent methods for model training [30]. Once the validation error converged, our RNN model reports an 1.58% test error in mean absolute percentage error (MAPE), which are comparable to the errors reported in several recent studies on load forecasting [7], [8]. We use L ∞ constraints on the attacker's capability (2b), such that the attacker is constrained by the maximum deviation of perturbed temperature values.

B. Load Forecasting Performance
We calibrate and compare the load forecasting model performance with and without adversarial attacks on test datasets. Though the forecasting model exhibits good performances on clean test data, we inject different level of perturbations generated by gradient estimation method, and found the forecasting performance decrease drastically as the adversarial perturbations become larger. In Figure 2 we visualize the RNN's load forecasting results for 300 hours using gradient estimation algorithm with maximum perturbation on temperature of 1F and 5F respectively. The attacker tries to increase the load in the first 150 hours, and to decrease the load in the latter hours. We observe that the algorithm finds the correct attack direction to either increase or decrease the load. What's more, with only 1F deviation on temperatures, the load forecasts changes over 500MW at some time steps. When the attacker increases the perturbation to 5F , large forecasts error over 1, 200MW are observed for the Swiss load. The temperature profile before and after attack still looks similar, which could avoid system operators' security inspection (Figure 2(b)). In Figure 3 we show by either attacking to increase the forecasted loads or attacking to decrease the forecasted loads, the gradient estimation attacks could achieve similar results compared to its whitebox counterparts. More essentially, few degrees of malicious perturbations have caused large deviations on forecasted loads.

C. Impacts of Attacks on System Operations
We find small adversarial perturbations over load forecasting input features even cause severe consequences on the power system operations. We are particularly interested in the case whenG t is different from G t that causes infeasible solution for ED without load shedding. In Figure 4, we visualize different constraint violations when ED is solved.
We ran a thorough evaluation for 122 test days on the 14bus system, and evaluate if the attackers could cause load shedding under different level of knowledge and capabilities. In Figure 5, we compare the number of days attacker cause load shedding with or without system topology information. In both cases, with larger perturbations added to the temperature forecasts, it is more likely to get an infeasible commitment schedule that causes load shedding. At one extreme, when the attacker knows network topology and is able to inject a 5F perturbation strategically on compromised load based on Algorithm 1, load shedding occurs on more than 100 of the test days. At the other extreme, the attacker causes over 40 days having shedded loads by only compromising one nodal load forecasts ( Figure 5(b)). More surprisingly, when the attacker does not know network topology and just selects compromised load randomly using eitherX j orX j , the system operation is still very vulnerable to the proposed load forecasting attacks ( Figure 5(a)). To furthur evaluate the attack's threats brought upon power system operation, one-day attack example on 118-bus test case is illustrated in Figure 6. We assume the attacker is able to design greedy attacks based on known topology information in this case, and show that by only compromising a small subset of nodal load forecasts, the resulting day-ahead commitment schedule under attacks is shutting off several generators compared to that under normal forecasts. The network already observes a series of operation threats such as overflown lines, on-capacity generators and load shedding.

VI. DISCUSSION AND CONCLUSION
In this paper, we studied the potential vulnerabilities generally existing in many load forecasting algorithms. Such vulnerabilities have been overlooked by the development of most if not all forecasting techniques. We designed a data injection attack which does not require parameters of the forecast algorithms, but leads to large increase in forecast errors. The proposed attack could adversarially impact the decision making process for system operators. Experiments on real-world load datasets demonstrate such threats over power system operations. Such threats model along with damage analysis indicate that there needs more security evaluations in the design and implementation of load forecasting algorithms. In order to mitigate the damages brought by such false data injection attacks, countermeasures such as anomaly detection as well as other robust statistics are strongly recommended.

A. Details on Learn and Attack Algorithm
In addition to the gradient estimation attack introduced in the main text, we also consider the learn and attack setting, where we assume the attacker does not have access to the model parameters, and there is no query access to the model. The only knowledge the attacker has is a historical datasetD tr , which includes same features under same distributions as data set D tr used to train the load forecasting model 5 . The proposed attack algorithm consists of a training phase and an attack phase as shown in Figure 7. In the training phase, the attacker trains substitute model fθ based onD tr to minimize the training loss. In the attack phase, the attacker pretends that the substitute model is the true load forecast model and performs white-box attacks on it to find the attack vectors. This strategy is based on the assumption that the substitute model behaves similarly to the true model not only for the training data X, but also for the attack vector X. Then by injectingX into the true load forecasting model, the forecast values go to attacker's desired directions.
It is useful to evaluate the transferability of proposed attacks across different set of models f θ and fθ. The phenomenon of transferability in adversarial attacks for machine learning models have been discussed in [31], [32], where adversarial instance generated using fθ can be also treated as an adversarial instance by f θ with high probability. The theoretical understanding of why attacks transfer remains an open question and is out of scope for this paper. In Figure 8 we show such transferability also exists in the load forecasting model using same test case on Switzerland load forecasting. The temperature inputs are generated by implementing the iterative gradient update based on a substitute model under L ∞ -norm of attack perturbations, yet such adversarial temperature values also mislead the (unknown) true load forecasting model to be wildly inaccurate.  Fig. 7: The attack schematic for learn and attack approach. During the training stage, the attacker uses historical data to learn a substitute forecast model; during attack stage, the attacker finds attack vector using substitute model and transfers it to unknown targeted system.

B. Details on Best First Search Attack Selection
As described in Algorithm 1 in Section IV-B, since it is computationally inefficient and challenging for the attacker to solve a tri-level attack problem (attacker solves an adversarial UC (7) usingL t q , operator minimizes ED costs (8) using G t , attacker maximizes ED costs (8) usingG t ), we propose a modified best-first search algorithm [24] for attackers to compromise a limited number of nodal forecasts. Essentially, in order to solve the tri-level problem expressively to find attack vectors, our proposed attack on networks iteratively finds the most vulnerable node to inject attacks.
Suppose for |Q| loads in the network, the attacker can at most N adv nodal load forecasts to avoid detection by system operators. Meanwhile, the constraints on attacker's capabilities described in Section III shall hold throughout data perturbation attacks. If an attacker could cause the day-ahead UC schedulẽ G t , t = 1, ..., T usingL t q , q = 1, ..., |Q|, t = 1, ..., T shift from G t , t = 1, ..., T , then it is expected the solution of ED will change. There are several possible circumstances by usingL t q : • Increasing the load maliciously will possibly incur extra system costs, such as starting to operate redundant generators, using more expensive generation combinations and etc; • Decreasing the load maliciously will possibly incur infeasble generation schedules during real-time dispatch, since there may be fewer generators scheduled than normal conditions, which will cause generators reach capacity or line flow exceed limits; • Decreasing the peak value of load maliciously will possibly cause UC solver ignore peak values, which may cause generators reach ramping limits. In our proposed algorithm for attackers to find most vulnerable nodal forecasts and inject attack forecasts, we design an iterative search scheme. At each iteration, the attacker solves day-ahead UC (7) based on currentL, and check which generator's schedule is most prone to change. Then the attacker decides the next compromised load node j. Since it is more possible to change UC schedule by chaning the load The load forecasting algorithm is an recurrent neural networks with inputs composed of past load, regional temperature forecast values and weather indicators. The attack perturbations are generated by using the learn and attack method, and it implements load maximization strategy in the first 150 hours and load minimization strategy in the latter 150 hours. (a). Load forecasting results with temperature attack constraint of (maximum perturbations) 1F ; (b). load forecasting results with temperature attack constraint of 5F .
profile to greater extents, the attacker then proposes two attack samplesX j andX j which minimizes and maximizes the nodal forecasts respectively. Such adversarial nodal load forecast is inserted into next iteration's UC problem.
In our implementation, we simply find the prone-to-change generation schedule based on following criteria: • The generator's dispatch during day-ahead UC is either approaching to generation limit or reaching zero output; • The line flow during day-ahead UC is approaching line flow limits; And simulation results indicate such search algorithm is quite efficient to find the attack injections. Other criteria such as ramping constraints and generation on-off limits could be also utilized to find the most vulnerable nodal forecasts.
For the case when system topology and parameters are unknown, the attacker just selects a random set of compromised load and inject attacks perturbations using eitherX j orX j for the compromised nodal loads. The result shown in Fig. 5 validates that in the case without topology knowledge, compromising the same number of loads are causing fewer load shedding days than in the case when topology is known.

C. Details on Attacks Implementation
In addition to the results we have shown in the main texts using RNN as load forecasting algorithm, we added ablation study on different load forecasting algorithms, as well as attack performance and computation time analysis.
1) Forecasting Method: Feed-Forward Neural Networks A multi-layered, feed-forward neural networks (NN) has been widely used to represent the nonlinearities between input features and output forecasts [6]. For the input layer of neural networks, each neuron represents one feature of training input, and all features of past H steps (X t−H , ..., X t ) are stacked as the inputs. For each intermediate layer, NN could have a tunable number of hidden units, which represent the input feature combinations.

Recurrent Neural Networks
As described in main texts, RNN feeds each step's input X t sequentially, and outputs a hidden unit to represent the feature combination of current input and historical features. The last neuron outputs the forecasted load values in the load forecasting attack [33]. Long Short-Term Memory Long Short-Term Memory network (LSTM) is designed to deal with the vanishing gradient problem existing in the RNN with long-time dependencies [7], [34]. The major improvements over RNN are the design of "forget" gates to model the temporal dependencies and capture long time dependencies in load patterns more accurately.
2) Training and Attack Details: We set up all load forecasting models using Tensorflow [29] package in Python. Standard model architectures such as Dropout layers and nonlinear activation functions (e.g., ReLU or Sigmoid functions) are adopted in the deep learning models [35]. Since all three networks are set up to solve the load forecasting regression problem, we set the first layer having most neurons, and decrease the number of    units in subsequent layers. Table I records our load forecasting model setup. We split 80% of data as our training sets, and use the remaining 20% of data on validating and evaluating the load forecasting prediction accuracy, attack performance and case studies on market operations. Such data collection procedures could also be applied in an online fashion so that attacker could inject real-time adversarial attacks into certain load forecasting models. As shown in Figure 9, all three load forecasting algorithms' validation loss are converged during training, and we use the trained model in the subsequent planning and operation problem as well as the testbed for attack algorithms. Plots are showing the mean and variance during 3 runs.
For the substitute model training of learn and attack method, we keep the training setD same as the load forecasting model training set D. Decreasing the size ofD or using different substitute dataset could decrease the performance of learn and attack. Table II compares all three load forecasting models' performance using clean and adversarial data. For both learn and attack and gradient estimation algorithms, they distort all three load forecasting models' output and increase model's forecast error. Gradient estimation attack works generally better for all three models, and this is due to estimating the gradients via querying f θ directly is more accurate than calculating it from the substitute model and transferring to f θ .
In Figure 10, we evaluate RNN's load forecasting performance under two attack strategies: load maximization or load minimization. We observe gradient estimation attack causes similar MAPE compared to white box attack. The load decreasing attack is normally more successful than load increasing attack in terms of MAPE. Load minimization attack is more harmful results than load increasing ones, since increased forecasts only let system operators start up more generations, while adversarially decreasing the forecasted load leads to wrong generation decisions that fails to meet the larger real load.

3) Computation Time:
We recorded the computation time for neural network training and the implementation time for two proposed attack algorithms. All time are recorded on a laptop with Intel 2.3GHz Core i5-8259U 4 Cores CPU and 8 GB RAM. The training time for NN, RNN and LSTM are calculated for 20, 30 and 30 epochs respectively. The implementation time for the attacks are averaged over all test instances. We observed that learn and attack approach takes longer time than gradient estimation due to the longer time taken to calculate gradient signs over the whole neural networks; and as LSTM includes more complicated model architectures, it takes longer time to find the adversarial instance. Yet compared to the long model training time and application scenarios of day-ahead forecasts, the attacker is still efficient enough to find the adversarial perturbations. Such efficient computation enables attacker to find the most vulnerable loads or to attack the system operations in very short time.
4) Code Availability: The implementation code for forecasts, attacks, and power system operations are all available at https://github.com/chennnnnyize/load forecasts attack.