text:
Coding Techniques for Future Wi-Fi

Future wireless applications such as high definition video streaming, wireless cloud radio access networks, and cellular data offload are bandwidth-hungry, which highlights the need for Wi-Fi links exceeding 1 Gb/s. This article discusses coding schemes and, in particular, the use of Low-Density Parity-Check (LDPC) codes as Forward Error Correction (FEC) in future Wi-Fi standards. Moreover, we consider advanced design strategies such as root-check LDPC structures, computer-aided design for short blocks and high-performance decoding algorithms with low latency. We then conclude the article with a discussion of FEC challenges for future Wi-FI applications.

I. INTRODUCTION
Driven by the ever-increasing user connectivity demands, emerging and future generations of Wi-Fi, such as IEEE 802.11ac and WiGig IEEE 802.11ad, will be capable of achieving multiple gigabits per second speeds. Furthermore, the future Wi-Fi will be used to do everything from simple web browsing and peer-to-peer sharing, to multimedia streaming, real-time teleconferencing, cable replacement, and wireless docking, to name a few. Therefore, one key element will be the reliable delivery of information at the final destination. One strategy to improve the reliability of the received information is the use of Forward Error Correction (FEC). FEC relies on a mathematical mapping of messages to include a special kind of redundancy that enables the receiver to correct the errors caused by the channel [1].
There are two main types of FEC used by Wi-Fi: LDPC codes and convolutional codes.
LDPC codes belong to the class of block codes, can perform close to channel capacity and achieve excellent performance. Block codes work on fixed-size blocks (packets) of bits or January  whereas LDPC codes are preferred for higher latencies or medium to large blocks [2].
In Fig. 1 an example of a system model for a Wireless Local Area Network (WLAN) is presented. In Fig 1 a), we have a Wi-Fi communications system where a tablet, a laptop and a mobile phone exchange information with an Access Point (AP). In Fig. 1 b) a simplified structure of a Wi-Fi communication system between a transmitter (TX) and a receiver (RX), for example, a laptop and an AP is shown. In Fig. 1  Extensions to OFDM systems [3], [4] and multiple-antenna systems [5] can also be considered.
The focus of this article is on LDPC design and decoding strategies for future Wi-Fi.
Following the system model of Fig. 1  consists of m × n circulant sub-matrices of dimensions s × s with M = ms and N = ns, the resulting linear block code will be a QC code with a period of n. In such code, the n-bit shift of any codeword is another codeword. The generator matrices G of QC-LDPC codes are in systematic-circulant form with the requirement that the PCM are full rank. The memory cost for encoding QC-LDPC codes is greatly reduced and the encoder can be implemented by using simple shift registers [1].
The accumulator-based codes that were invented first are the so-called repeat-accumulate (RA) codes [1]. Despite their simple structure, they were shown to provide good performance and, more importantly, they pioneered the design of efficiently encodable LDPC codes. The key points in using IRA-LDPC codes are the simplicity in designing such codes and faster encoding than conventional LDPC methods. The Wi-Fi standard uses a type of QC-LDPC codes, whereas other standards work with a combination of QC and IRA LDPC codes known as QC-IRA LDPC codes.
There are two main advantages in QC-IRA-LDPC codes: the memory requirements to store the matrices by QC codes and the simple encoding process provided by IRA codes. For instance,   Fig. 2 is an LDPC code with code rate R = 1 2 . The rate 1 2 means that half of the codeword is the message (information) and the other half is the redundancy. The Wi-Fi standard operates with the following code rates, R = 1 2 , 3 4 , 5 8 and 13 16 with a fixed block length of L = 672 bits. For example, with a code rate 1 2 we have 336 information bits plus 336 bits of redundancy, for code rate 13 16 we have 546 information bits plus 126 bits of redundancy. Fig. 3 presents the PCM H of the Wi-Fi standard with code rate R = 1 2 and dimensions 336 × 672. The other PCMs for the rest of code rates follow a similar structure as for code rate Once a standard is defined, a design is adopted and the PCM H must be stored in all devices for compatibility issues. Whenever the standard is updated there is an opportunity to incorporate more sophisticated PCM designs which could lead to improved performance.

IV. DESIGNING HIGH-PERFORMANCE LDPC CODES
In this section we review advanced LDPC code design strategies such as Progressive Edge Growth (PEG) algorithms, Root-Check LDPC structures, QC and IRA structures.

A. PEG-Based Algorithms
Among the algorithms capable of producing LDPC codes of highest performance and efficiency for short to moderate lengths are the PEG-based algorithms [6] and QC and IRA structures.
The codes produced by the PEG algorithm exhibit improved performance compared to random construction methods [6] as PEG optimizes the underlying graph structure of the PCM. In particular, PEG strategies attempt to increase the girth of the graph and improve its general connectivity. Improvements and modifications in the original PEG algorithm have provided a better performance than the original PEG algorithm. For instance, Healy and de Lamare in [7] have proposed an algorithm to design LDPC codes which involves the use of decoder-based optimisation with the sum-product algorithm (SPA) and also considered multiple candidates to optimize the graph connections [8]. Therefore, LDPC codes for future Wi-Fi standards can take advantage if they are designed by PEG-based algorithms and several detection and decoding strategies [9], [10], [11], [12], [13], [14].

B. Structured Code Designs
A drawback of standard unstructured LDPC codes designed with conventional and PEG-based algorithms is their high encoding complexity, which impacts the cost and power consumption of devices. Fortunately, the memory and computational cost for encoding LDPC codes can be greatly reduced by adopting QC-LDPC codes because the encoder can be implemented by simple shift registers. Furthermore, IRA LDPC codes are also useful due to their simplicity in designing and faster encoding than conventional LDPC codes. In addition, designers can resort to QC-IRA LDPC codes which brings further cost reduction by combining the features of QC and IRA LDPC codes [15], using PEG-based optimization of the graphs and statistically-driven approaches [16]. These highly structured LDPC codes could play an important role in the design of future Wi-Fi standards.

C. Root-Check Codes
Another family of LDPC codes called Root-Check for block-fading channels were proposed in [17]. Root-Check codes are able to achieve the maximum diversity of a block-fading channel and have a performance near the limit of outage. Several types of Root-Check LDPC codes were developed, e.g, [18], [19], [8]. Among the Root-Check based LDPC codes the ones designed with the PEG algorithm have shown the best performance [19]. In particular, we have designed For example, consider a simple scenario where the channel is a block-fading with F = 2 fadings, BPSK modulation is used and a maximum of 20 decoding iterations are allowed.
In Fig. 4 we depict the Frame Error Rate (FER) performance of a Root-Check LDPC code versus the LDPC code from Wi-Fi standard both are with the same code rate and with the same block length. The outage curve is also plotted as reference which represents the channel capacity.
From Fig. 4, we can see that the QC-IRA Root-Check LDPC code outperforms the Wi-Fi LDPC code by 6.5dB in terms of SNR.

V. ADVANCED DECODING TECHNIQUES
In this section, we discuss advanced LDPC decoding techniques which include reweighting and scheduling approaches, and low-complexity decoding algorithms. These algorithms offer considerable advantages over standard belief propagation (BP) algorithms and can address some of its limitations with regards to performance, delay issues and computational cost.

A. Reweighting Methods
The BP algorithm, sometimes also called sum-product algorithm (SPA), is a powerful algorithm to approximately solve several inference problems which can be used for error control coding.
where L(x j ) is the a priori Log Likelihood Ratio (LLR) from the channel, ρ is the constant

B. Scheduling Methods
The studies in [22], [23] have suggested that the use of appropriate scheduling mechanisms for LDPC decoding can significantly improve the convergence speed in terms of number of iterations. In general, BP or SPA consist of the exchange of messages between the nodes of a graph. Each node generates and propagates messages to its neighbours based on its current incoming messages.
The LDPC code graph is a bi-partite graph composed by N V N variable nodes v j for j ∈ {1, · · · , N V N } that represent the codeword bits and M CH check nodes c i for i ∈ {1, · · · , M CH } that represent the parity-check equations [22]. RBP was first introduced in [24] and consists of an informed dynamic scheduling strategy that updates messages according to an ordering metric called the residual. The message with the largest residual is updated first. A residual is the norm (defined over the message space) of the difference between the values of a message before and after an update. The intuitive justification of this approach is that as iterative BP converges, the differences between the messages before and after an update diminish. Then, if a message has a large residual, it means that it is located in a part of the graph that has not converged yet. Accordingly, propagating that message first should speed up convergence.
In order to obtain a better performance, a less greedy scheduling strategy can be used. Node Wise Scheduling BP (NWBP). Fig. 6, depicts an example decoding techniques, which illustrates how they can reduce significantly the number of iterations as compared to BP. With the advent of parallel computing in the latest powerful processors, the use of scheduling methods can reduce significantly the overall number of iterations required in the decoding procedure. With the advent of parallel computing in the latest powerful processors, the use of scheduling methods can reduce significantly the overall number of iterations required in the decoding procedure.

C. Reduced Complexity Decoding Methods
Work on LDPC decoding has mainly focused on floating point arithmetic or infinite precision (BP algorithm). However, hardware implementations of decoding algorithms for LDPC codes must address quantization effects in a fixed-point realization [25]. The first approach was to adopt a logarithmic version of the BP algorithm called Log-BP. Nevertheless, Log-BP sacrifices hardware implementation due to the fact it requires many hyperbolic tangent operations.
The Log-BP algorithm can be simplified using the so-called BP-based approximation (also known as the "min-sum" approximation), which greatly reduces the implementation complexity, but incurs a degradation in decoding performance. This has led to the development of many reduced complexity variants of the BP algorithm that deliver near-optimum decoding performance. There are three main reduced complexity methods of decoding LDPC codes: BP-based, Normalized BP-based and Offset BP-based [25].
In the min-sum algorithm the key modification is on the horizontal step or the check to variable node update equation. The check to variable node update is given by where, Λ j ′ ,i are the messages sent from variable node v j to check node c i and j ′ ∈ N (i) \ j is the neighbouring set of variable nodes of c i except v j . As it can be seen in (2) there is no hyperbolic tangent operation which decreases significantly the overall computational complexity, although it causes a degradation in decoding performance.
The min-sum algorithm can be improved by employing a check node update that uses a normalization constant greater than α. The change made in (2) is on the right hand side and inside the min operation that is divided by α. This method is called Normalized BP-based.
The α parameter should be adjusted for different SNRs and iterations to achieve its optimum performance. An effective approach to determine the optimum value of α is by Density Evolution [25].
A computationally more efficient approach that captures the net effect of the additive correction term applied in each check node update operation is obtained from the BP-based decoding by subtracting a positive constant β. This is called Offset BP-based. Eq. (2) is modified such The PEG-based QC-IRA Root-Check LDPC code designed was compared to the Wi-Fi LDPC code to analyse the overall performance in terms of FER. As seen in Fig. 4 the PEG-based designed code has outperformed the Wi-Fi LDPC code by about 6.5dB. Therefore, future Wi-Fi standards would have benefits if they considered the use of Root-Check based LDPC codes for rate R = 1 2 scenarios. The use of scheduling methods to decode LDPC codes can reduce significantly the number of iterations required in the decoding procedure. Accordingly, the resulting number of iterations can increase the advantages LDPC codes over convolutional codes, which are widely used in portable devices due to their simplicity. An open problem in LDPC decoding is: Which is the most costeffective method for the Wi-Fi standard? This is an investigation that will keep designers of Wi-Fi systems busy in the next few years.
In terms of adoption of LDPC codes as FEC in Wi-Fi standards, key issues are the encoding and decoding operations. The cost of encoding can be decreased significantly by using RA based LDPC codes which are able to provide simple and low complexity encoding methods. The decoding cost is another key issue that must be addressed. One way to addressing such problem is by introducing reduced complexity decoding (RCD) algorithms, e.g., min-sum, normalized BP-based and Offset BP-based. The choice of decoding algorithm is another important design challenge, which must take into account aspects such low complexity operations, stability and performance.

VII. CONCLUSION
This article has discussed coding techniques for Wi-Fi systems. In particular, we have discussed approaches to reducing the complexity of encoding and decoding and improving the design and the decoding of LDPC codes. The main challenge is in decoding which is how to obtain an attractive trade-off between performance and computational complexity. To conclude, we advocate that future Wi-Fi standards should use LDPC codes as the main FEC to fulfil the needs of bandwidth-hungry and high-performance applications.

ACKNOWLEDGMENT
This work was partially supported by PNPD/CAPES (Brazil).


extfieldsofstudy:
Computer Science
Mathematics

text:
Improving Generalization and Robustness with Noisy Collaboration in Knowledge Distillation

Inspired by trial-to-trial variability in the brain that can result from multiple noise sources, we introduce variability through noise at different levels in a knowledge distillation framework. We introduce"Fickle Teacher"which provides variable supervision signals to the student for the same input. We observe that the response variability from the teacher results in a significant generalization improvement in the student. We further propose"Soft-Randomization"as a novel technique for improving robustness to input variability in the student. This minimizes the dissimilarity between the student's distribution on noisy data with teacher's distribution on clean data. We show that soft-randomization, even with low noise intensity, improves the robustness significantly with minimal drop in generalization. Lastly, we propose a new technique,"Messy-collaboration", which introduces target variability, whereby student and/or teacher are trained with randomly corrupted labels. We find that supervision from a corrupted teacher improves the adversarial robustness of student significantly while preserving its generalization and natural robustness. Our extensive empirical results verify the effectiveness of adding constructive noise in the knowledge distillation framework for improving the generalization and robustness of the model.

Introduction
The design of Deep Neural Networks (DNNs) for efficient real-world deployment involves careful consideration of the following key elements: memory and computational requirements, performance, reliability, and security. DNNs are often deployed in resource-constrained devices or in applications with strict latency requirements such as autonomous cars which leads to the necessity for developing compact models that generalize well. Furthermore, since the models are often deployed in dynamic environments, it is important to consider their performance on both in-distribution data as well as out-of-distribution data. This ensures the reliability of the models under distribution shift. Finally, the model needs to be robust to malicious attacks by adversaries [32].
A number of techniques have been proposed for achieving high performance in compressed models such as model quantization [61], model pruning [20], and knowledge distillation [24]. Here, we focus on knowledge distillation (KD) as an interactive learning method which is more similar to human learning. Knowledge distillation involves training a smaller model (student) under the supervision of a larger pre-trained model (teacher). In the original formulation, Hinton et al. [24] proposed mimicking the softened softmax output of the teacher which exhibits consistent performance improvement of the student compared to the model trained without teacher assistance. Despite the promising performance gain, there is still a significant generalization gap between student and teacher. Consequently, an optimal method of capturing knowledge from the larger model and transferring it to a smaller model remains an open question. While it is important to reduce the generalization gap, it is also pertinent to incorporate methods into the knowledge distillation framework to make the training and inference of the model more robust.
Noise permeates every level of the nervous system, from the perception of sensory signals to the generation of motor responses [12] and also plays an important role in neural networks training [4,41]. We therefore hypothesize that noise could help in improving the robustness and generalization of the neural networks. Inspired by trial-to-trial variability in the brain, variations in neural responses to the same stimuli, which can result from multiple noise sources, we introduce variability through noise at input level, supervision signal from teacher or target level in the knowledge distillation framework.
To test our hypothesis, we propose novel ways of injecting noise into the knowledge distillation framework as general and scalable techniques and exhaustively evaluate their effect on generalization and robustness. Our contributions are as follows: • "Fickel Teacher", a novel approach to simulate trial-to-trial response variability of biological neural networks. The method exposes student to the uncertainty of teacher which results in significant generalization improvement (from 94.28% to 94.67%).
• 'Soft-Randomization", a novel approach for increasing robustness to input variability. The method significantly increases the capacity of the model to learn robust features with small additive noise in the input. For (σ = 0.05), soft-randomization achieves 16.75% PGD-20 robustness and 92.57% test-set accuracy whereas the model trained with Gaussian data augmentation achieves only 0.41% adversarial robustness and lower generalization (92.14%).
• "Messy-Collaboration", an approach for using target variability as a strong deterrent to cognitive bias. We observe its surprising ability to significantly improve adversarial robustness (from 0.15% to 11.41%) with minimal reduction in generalization (from 94.28% to 93.96%).
• We show the effectiveness of messy-collaboration in learning with noisy labels.
• A comprehensive analysis of the effects of injecting noise in the knowledge distillation framework.

Related Work
A number of experimental and computational methods have reported the presence of noise in the nervous system and how it affects the function of the system [12]. Analogously, noise has been used as a common regularization technique to improve the generalization performance of overparameterized deep neural networks by adding it to the input data, the weights or the hidden units [1,3,16,47,48,53]. Previous studies also showed that noise is crucial for non-convex optimization [28,34,56,61]. Furthermore, a family of randomization techniques that inject noise in the model both during training and inference time are proven to be effective to the adversarial attacks [10,36,44,55]. Randomized smoothing transforms any classifier into a new smooth classifier that has certifiable l 2 -norm robustness guarantees [7,33]. Label smoothing improves the performance of deep neural networks across a range of tasks [42,49]. However, Müller et al. [40] reports that label smoothing impairs knowledge distillation. On the contrary, we show that our biologically inspired technique of injecting noise into knowledge distillation, fickle teacher, significantly improves the generalization of student. Furthermore, fickle teacher differs from the works of Bulò et al. [5] and Gurau et al. [18] in that instead of using the soft target distribution obtained by averaging Monte Carlo samples, we use the logits of the teacher model with dropout active directly as a source of uncertainty encoding noise for distilling knowledge to a compact student.

Experimental Setup
To study the effect of injecting noise in the knowledge distillation framework, we use Hinton method [24] which trains the student by minimizing the Kullback-Leibler divergence between the smoother output probabilities of the student and teacher. In all of our experiments we use α = 0.9 and τ = 4. We conducted our experiments on Wide Residual Networks (WRN) [59]. Unless otherwise stated, we normalize the images between 0 and 1 and use standard training scheme as used in [52,58] Robustness (%) Figure 1: Exposing the student to the uncertainty of the teacher significantly boosts in-distribution (CIFAR-10) and out-ofdistribution (CINIC) generalization over the Hinton method (dropout rate = 0; left). Fickle teacher increases the adversarial robustness to PGD attack only for low dropout rate (right).
To approximate the out-of-distribution generalization of our models, we use the ImageNet [31] images from the CINIC dataset [8]. For the evaluation of adversarial robustness, we use the Projected Gradient Descent (PGD) attack from Kurakin et al. [32] and conduct the attack for multiple step sizes. We report the worst robustness accuracy for five random initialization runs. Finally, we test the robustness of our models to commonly occurring corruptions and perturbations proposed by Hendrycks and Dietterich [22] in CIFAR-C as a proxy for natural robustness. We evaluate average robustness to the 19 distortions across 5 severity levels. Furthermore, we calculate mean Corruption Accuracy (mCA) over the 19 distortions. For both adversarial and natural robustness evaluation, we use a fine-grained measure of robustness, by computing accuracy on the corrupted image conditional on the clean image being classified correctly (for details of the methods, see Appendix).

Empirical study of Noises
In this section, we propose injecting different types of noise in the student-teacher collaborative learning framework and analyze their effect on the generalization and robustness of the model.

Fickle Teacher
Trial-to-trial response variability in the brain can be considerably different across stimuli, suggesting that it could also provide an important contribution to the information conveyed by the neural responses about the stimuli [45]. Similarly, in deep neural networks, dropout which randomly switches off a group of selected hidden units results in response variability and can be used to obtain principled uncertainty estimates [14]. We, therefore, propose to use dropout in the teacher model to simulate trial-to-trial variability. We first train the teacher with dropout and keep dropout active in the teacher while distilling knowledge to the student. As a result, the teacher provides variable supervision signals to the student for the same input, thereby exposing the student to its uncertainty. We systematically change the dropout rate {0.1, 0.2, 0.3, 0.4, 0.5} and show its effect on generalization and robustness (for training details, see the Appendix).
The results show that fickle teacher significantly improves both in-distribution and out-of-distribution generalization of the student compared to Hinton method (Figure 1, left). Interestingly, for CIFAR-10, even when the accuracy of the teacher decreases after a dropout rate of 0.2, the student accuracy still improves up to a dropout rate of 0.4. For the same student and teacher network architectures, fickle teacher achieves higher performance on CIFAR-10 than the state-of-the-art knowledge distillation methods [52,58]. For CINIC, as we increase the dropout rate, the student generalization gets closer to the teacher and is the highest at dropout rate 0.5. The adversarial robustness increases for dropout rate up to 0.2 and then decreases ( Figure 1, right) while the mCA for natural robustness is maintained or marginally increases with the dropout rate ( Figure 2). The results show the effectiveness of fickle teacher in improving the generalization of student.

Soft-Randomization
Pinot et al. [43] show that injection of noise drawn from the Exponential family such as Gaussian or Laplace noise leads to guaranteed robustness to adversarial attack. However, this improved adversarial robustness comes at the cost of generalization. Therefore, an important consideration for methods proposed to increase the adversarial robustness is to reduce the loss in generalization. Since knowledge distillation provides an opportunity to combine multiple sources of information, we hypothesize that combining information from a teacher with high generalization while training the student to be robust to noisy input can reduce the trade-off. To test the hypothesis, we propose a novel technique for  improving robustness to input variability in the student which uses the teacher trained on clean data, to train the student on noisy data. Here, we minimize the dissimilarity between the student's distribution on noisy data with the teacher's distribution on clean data ( Figure 3). The loss function is as follows: where δ ∼ N (0, σ 2 ) is white Gaussian noise. α and τ are the balancing factor and temperature parameter from the Hinton method. S(.) denotes the softmax output of student, S τ (.) and T τ (.) denote the student's and teacher's softmax output with raised temperature respectively.
Teacher Student +   +  Figure 3: Soft-randomization minimizes the KL divergence between the student response on noisy data and the teacher response on the clean data in addition to minimizing the cross entropy loss on the noisy data.
We train soft-randomization for both low noise intensities {0.01, 0.02, 0.03, 0.04, 0.05} and higher noise intensities {0.1, 0.2, 0.3, 0.4, 0.5}. As a baseline, we compare soft-randomization to the compact model trained alone with Gaussian data augmentation. Figure 4 shows that soft-randomization consistently achieves both higher adversarial robustness and generalization for all σ values compared to Gaussian augmentation. For lower noise intensities, soft-randomization significantly outperforms Gaussian augmentation. For σ = 0.05, soft-randomization achieves 16.75% robustness to PGD-20 attack and 92.57% CIFAR-10 generalization compared to 0.41% robustness and 92.14% generalization for Gaussian augmentation. Hence, soft-randomization significantly increases the the capacity of the student to learn robust features even with lower noise intensities. Figure 5 shows the mCA consistently improves over the Hinton method for all Gaussian noise levels. While robustness drops most notably for color distortions (brightness, fog, contrast, and saturation), robustness to noise and blurring corruptions improves significantly as the Gaussian noise intensity increases. We also observe changes in the effect at different intensities, for example for frost, the robustness increases at lower noise level and then decreases for higher intensities. Soft-randomization allows the use of lower noise intensity for increasing adversarial robustness while keeping the loss in generalization lower compared to Gaussian augmentation. This provides greater flexibility in finding a suitable trade-off between generalization and robustness based on the application.

Messy-collaboration
Human decision-making shows systematic simplifications and deviations from the tenets of rationality ('heuristics') which may lead to sub-optimal decisional outcomes ('cognitive biases') [29]. We believe, this cognitive bias is manifested in deep neural networks in the form of memorization and over-generalization, and propose a counter-intuitive regularization technique based on label noise to mitigate cognitive bias. We term this technique as messy-collaboration (MC). For each sample in the training process, with rate r, we randomly change the one-hot encoded target labels to an  Robustness (%) incorrect class. The intuition behind this method is that by randomly relabeling a fraction of the samples in each batch, we introduce target variability 2 which encourages the model not to be overconfident in its predictions and discourages memorization. There are a number of studies on improving the tolerance of DNNs to noisy labels [19,25,54]. However, to the best of our knowledge, random label noise has not been explored as a source of constructive noise for cognitive bias mitigation. Our experiments set out to dissociate the effect of injecting target variability in different stages of the knowledge distillation framework. Therefore, we study the effect of systematically increasing the noise rate r for different setups: introducing target variability while training teacher (MC-T), introducing target variability while distilling knowledge to the student (MC-S), and introducing target variability when training the teacher as well as during knowledge distillation to the student (MC-TS).
When target variability is used only during knowledge distillation to the student (MC-S), both in-distribution and out-of-distribution generalization increase over the Hinton method, even for very high noise rates. When target variability is used for training the teacher which is then used to distill knowledge to the student (MC-T and MC-TS), the generalization drops ( Figure 6). At higher noise rates (r = 0.5), all variants of messy-collaboration significantly outperform the teacher trained with target variability at the same rate. Interestingly, target variability has the remarkable effect on increasing the adversarial robustness. Figure 7 shows that the robustness of the student increases as we increase the noise rate. The increase in adversarial robustness is more pronounced when the teacher model trained with target variability is used to train the student (MC-T and MC-TS). Furthermore, messy-collaboration maintains the natural robustness of Hinton method (Figure 16 in appendix).
To understand the effect of target variability on generalization, we visualize the distribution of the magnitude of softmax probabilities. Figure 8 shows that as we increase the noise rate in messy-collaboration, it not only leads to a smoother output distribution, but also shifts the distribution from overconfident softmax probabilities to lower probabilities. The results show that target variability in messy-collaboration enforces the model not to be overconfident in its predictions and discourages memorization which results in better generalization.

Messy-Collaboration for learning with noisy labels
To utilize the vast amount of open-source data available, researchers have proposed methods to generate labels automatically using user tags and keywords. However, these techniques lead to noisy labels which affect the generalization of the model [13]. Considering the abundance of noisy labels, it is important to develop methods that can effectively learn from noisy labels. Here, we show the effectiveness of knowledge distillation in learning with noisy labels at varying rates of label corruption on CIFAR-10. We further show that messy-collaboration improves the generalization over Hinton method. Figure 9 shows that the generalization drops with increasing the corruption rate (cf. Teacher and WRN- . For label corruption rate 0.1 and higher, knowledge distillation improves the generalization performance significantly and even outperforms the teacher. The gain in generalization with Hinton method is higher as we increase the label corruption rate. Figure 10 shows the effect of varying the messy-collaboration's noise rate for the different label corruption rates. For label corruption rate over 0.1, messy-collaboration improves the generalization over the student and teacher for all noise rates. Interestingly, at a higher noise rate of 0.5, messy-collaboration improves the generalization over Hinton for all corruption rates. This shows that the target variability in messy-collaboration makes the model more tolerant to label noise which allows efficient learning with noisy labels.

Conclusion
We proposed novel ways of injecting noise in the knowledge distillation framework and rigorously studied their effect on the generalization and robustness of the model. We introduced fickle teacher which exposes the student to its uncertainty using dropout. We show that the variability in the supervision signal improves both in-distribution and out-of-distribution generalization significantly while marginally improving robustness to common and adversarial perturbations. We further proposed soft-randomization for increasing the robustness to input variability by matching the output distribution of student on noisy data to the output distribution of teacher on clean data. This significantly increases the capacity of the student to learn robust features. We showed it improves the adversarial robustness of student compared to the model trained alone with Gaussian data augmentation by a significant margin for lower noise intensities, while also reducing the drop in generalization. Finally, we introduced messy-collaboration which imposes target variability in different stages of the knowledge distillation framework. We showed the intriguing effect of target variability on increasing the adversarial robustness by an order of magnitude in addition to improving the generalization.
We further showed the effectiveness of messy collaboration in learning with noisy labels. The extensive empirical results demonstrate that injecting noises which increase the trial-to-trial variability in the knowledge distillation framework is a promising direction towards training compact models with improved generalization and robustness.

A.1 Preliminaries
In this section we provide details for the methods relevant to our study.

A.1.1 Knowledge Distillation
Hinton et al. [24] proposed to use the final softmax function with a raised temperature and use the smooth logits of the teacher as soft targets for the student. The method involves minimizing the Kullback-Leibler divergence between the smoother output probabilities: where L CE denotes cross-entropy loss, σ(.) denotes softmax function, z S student output logit, z T teacher output logit, τ and α are the hyperparameters which denote temperature and balancing ratio, respectively.

A.1.2 Out-of-Distribution Generalization
Neural networks tend to generalize well when the test data comes from the same distribution as the training data [9,21]. However, models in the real world often have to deal with some form of domain shift which adversely affects the generalization performance of the models [27,35,39,46]. Therefore, test set performance alone is not the optimal metric for evaluating the generalization of the models in test environment. To measure the out-of-distribution performance, we use the ImageNet images from the CINIC dataset [8]. CINIC contains 2100 images randomly selected for each of the CIFAR-10 categories from the ImageNet dataset. Hence, the performance of models trained on CIFAR-10 on these 21000 images can be considered as a approximation for a model's out-of-distribution generalization.

A.1.3 Adversarial Robustness
Deep Neural Networks have been shown to be highly vulnerable to carefully crafted imperceptible perturbations designed to fool a neural network by an adversary [2,50]. This vulnerability poses a real threat to deep learning model's deployment in the real world [32]. Robustness to these adversarial attacks has therefore gained a lot of traction in the research community and progress has been made to better evaluate robustness to adversarial attacks [6,15,38] and defend the models against these attacks [37,60].
To evaluate the adversarial robustness of models in this study, we use the Projected Gradient Descent (PGD) attack from Kurakin et al. [32]. The PGD-N attack initializes the adversarial image with the original image with the addition of a random noise within some epsilon bound, . For each step it takes the loss with respect to the input image and moves in the direction of loss with the step size and then clips it within the epsilon bound and the range of valid image.
where denote epsilon-bound, α step size and X original image. The projection operator ,d (A) denotes element-wise clipping, with A i,j clipped to the range [X i,j − , X i,j + ] and within valid data range. In all of our experiments, we conduct a 10, 15 and 20 step PGD attack with 0.031 epsilon with 0.03 step size and 5 random initializations.

A.1.4 Natural Robustness
While robustness to adversarial attack is important from security perspective, it is an instance of worst case distribution shift. The model also needs to be robust to naturally occurring perturbations which it will encounter frequently in the test environment. Recent works have shown that Deep Neural Networks are also vulnerable to commonly occurring perturbations in the real world which are far from the adversarial examples manifold. Hendrycks et al. [23] curated a set of real-world, unmodified and naturally occurring examples that causes classifier accuracy to significantly degrade. Gu et al. [17] measured model's robustness to the minute transformations found across video frames which they refer to as natural robustness and found state-of-the-art classifier to be brittle to these transformations. In our study we use robustness to the common corruptions and perturbations proposed by Hendrycks and Dietterich [22] in CIFAR-C as a proxy for natural robustness. Robustness (%) Figure 11: Lower level of signal-dependent noise on supervisory signal from teacher improves the in-distribution generalization and adversarial robustness but not the out-of-distribution generalization. Robustness (%) Figure 12: Signal-dependent noise maintains the natural robustness of the Hinton method.

A.1.5 Trade-off between Generalization and Adversarial Robustness
While making our model's robust to adversarial attacks, we need to be careful not to overemphasize robustness to norm bounded perturbation. We also need to rigorously test the effect of methods designed to increase adversarial robustness on model's in-distribution and out-of-distribution generalization as well as robustness to naturally occurring perturbation and distribution shift. Recent study have highlighted the adverse affect of adversarially trained model on natural robustness. Ding et al. [11] showed that even a semantics-preserving transformations on the input data distribution significantly degrades the performance of adversarial trained models but only slightly affects the performance of standard trained model. Yin et al. [57] showed that adversarially trained models improve robustness to mid and high frequency perturbations but at the expense of low frequency perturbations which are more common in the real world. Furthermore, in the adversarial literature, a number of studies has shown an inherent trade-off between adversarial robustness and generalization [26,51,60].

A.2.1 Signal-dependent Noise
Here, we add a signal-dependent noise to the output logits of the teacher. For each sample, we add zero-mean Gaussian noise with variance that is proportional to the output logits in the given sample (z i ): We systematically study the effect of increasing the intensity of signal-dependent noise (0, 0.1, 0.2, 0.3, 0.4, 0.5). Figure  11 shows that for noise levels up to 0.1, the random signal-dependent noise improves the in-distribution generalization over the Hinton method. However, it impairs the out-of-distribution generalization to CINIC-ImageNet. Figure 11 and Figure 12 show a slight increase in the adversarial robustness and natural robustness of the models.
Müller et al. [40] reported that when the teacher is trained with label smoothing, the knowledge distillation to the student is impaired and the student performs worse. On the contrary, for lower level of noise, our method improves the effectiveness of knowledge distillation process. Signal-dependent noise differs from their approach in that we train the teacher without any noise and only when distilling knowledge to the student, we add noise to its softened logits.

A.2.2 Random Swapping
To exploit the uncertainty of the teacher for a sample, we inject noise by randomly swapping softened softmax logits. With probability p, we swap the logits if the difference is below a threshold.
We propose two variants of random swapping: 1. Swap Top 2: Swap the top two logits if the difference between them is below the threshold. Robustness (%) Figure 13: Swapping the logits based on the relative logit difference improves in-distribution generalization, but impairs out-ofdistribution generalization (left). Robustness to PGD-20 is maintained except for swap Top2 with higher swapping probability (right).
These technique improves the in-distribution generalization but adversely affects the out-of-distribution generalization ( Figure 13. It does not have a significant affect on the adversarial robustness ( Figure 13), but it marginally improves mCA ( Figure 14).

A.3 Training Scheme for Fickle Teacher
Because of the variability in the teacher, the student needs to be trained for more epochs in order for it to converge and be effectively exposed to the uncertainty of the teacher.
We used the same initial learning rate of 0.1 and decay factor of 0.


extfieldsofstudy:
Computer Science
Medicine

text:
A novel gene selection algorithm for cancer classification using microarray datasets

Background Microarray datasets are an important medical diagnostic tool as they represent the states of a cell at the molecular level. Available microarray datasets for classifying cancer types generally have a fairly small sample size compared to the large number of genes involved. This fact is known as a curse of dimensionality, which is a challenging problem. Gene selection is a promising approach that addresses this problem and plays an important role in the development of efficient cancer classification due to the fact that only a small number of genes are related to the classification problem. Gene selection addresses many problems in microarray datasets such as reducing the number of irrelevant and noisy genes, and selecting the most related genes to improve the classification results. Methods An innovative Gene Selection Programming (GSP) method is proposed to select relevant genes for effective and efficient cancer classification. GSP is based on Gene Expression Programming (GEP) method with a new defined population initialization algorithm, a new fitness function definition, and improved mutation and recombination operators. . Support Vector Machine (SVM) with a linear kernel serves as a classifier of the GSP. Results Experimental results on ten microarray cancer datasets demonstrate that Gene Selection Programming (GSP) is effective and efficient in eliminating irrelevant and redundant genes/features from microarray datasets. The comprehensive evaluations and comparisons with other methods show that GSP gives a better compromise in terms of all three evaluation criteria, i.e., classification accuracy, number of selected genes, and computational cost. The gene set selected by GSP has shown its superior performances in cancer classification compared to those selected by the up-to-date representative gene selection methods. Conclusion Gene subset selected by GSP can achieve a higher classification accuracy with less processing time.

Background
The rapid development of microarray technology in the past few years has enabled researchers to analyse thousands of genes simultaneously and obtain biological information for various purposes, especially for cancer classification. However, gene expression data obtained by microarray technology could bring difficulties to classification methods due to the fact that usually the number of genes in a microarray dataset is very big, while the number of samples is small. This fact is known as the curse of dimensionality in data mining [1][2][3][4]. Gene selection, which extracts informative and relevant genes, is one of the effective options to overcome the curse of dimensionality in microarray data based cancer classification.
Gene selection is actually a process of identifying a subset of informative genes from the original gene set. This gene subset enables researchers to obtain substantial insight into the genetic nature of the disease and the mechanisms responsible for it. This technique can also decrease the computational costs and improve the cancer classification performance [5,6].
Typically, the approaches for gene selection can be classified into three main categories: filter, wrapper and embedded techniques [6,7]. The filter technique exploits the general characteristics of the gene expressions in the dataset to evaluate each gene individually without considering classification algorithms. The wrapper technique is to add or remove genes to produce several gene subsets and then evaluates these subsets by using the classification algorithms to obtain the best gene subset for solving the classification problem. The embedded technique is between the filter and wrapper techniques in order to take advantage of the merits of both techniques. However, most of the embedded techniques deal with genes one by one [8], which is time consuming especially when the data dimension is large such as the microarray data.
Naturally inspired evolutionary algorithms are more applicable and accurate than wrapper gene selection methods [9,10] due to their ability in searching for the optimal or near-optimal solutions on large and complex spaces of possible solutions. Evolutionary algorithms also consider multiple attributes (genes) during their search for a solution, instead of considering one attribute at a time.
Various evolutionary algorithms [11][12][13][14][15][16][17][18][19] have been proposed to extract informative and relevant cancer genes and meanwhile reduce the number of noise and irrelevant genes. However, in order to obtain high accuracy results, most of these methods have to select a large number of genes. Chuang et al. [20] proposed the improved binary particle swarm optimization (IBPSO) method which achieved a good accuracy for some datasets but, again, selected a large number of genes. An enhancement of BPSO algorithm was proposed by Mohamad et al. [21] by minimizing the number of selected genes. They obtained good classification accuracies for some datasets, but the number of selected genes is not small enough compared with other studies.
Recently, Moosa et al. [22] proposed a modified Artificial Bee Colony algorithm (mABC). Another study [15] proposed a hybrid method by using Information Gain algorithm to reduce the number of irrelevant genes and using an improved simplified swarm optimization (ISSO) to select the optimal gene subset. These two studies were able to get a good accuracy with small number of selected genes. However, the number of selected genes is still high compared with our method.
In recent years, a new evolutionary algorithm known as Gene Expression Programming (GEP) was initially introduced by Ferreira [23] and widely used in many applications for classification and decision making [24][25][26][27][28][29][30]. GEP has three main advantages -Flexibility, which makes it easy to design an optimal model. In other words, any part of GEP steps can be improved or changed without adding any complexity to the whole process. -The power of achieving the target that is inspired from the ideas of genotype and phenotype -Data visualization. It is easy to visualize each step of the GEP and that distinguishes it from many algorithms These advantages make it easy to use GEP process to create our new gene selection program and simulate the dynamic process of achieving the optimal solution in decision making.
A few studies applied GEP as a feature selection method and obtained some promising results [31,32] which encourage us to do further research.
GEP algorithm, based on its evolutionary structure, faces some computational problems, when it is applied to complex and high dimensional data such as microarray datasets. Inspired by the above circumstances, to enhance the robustness and stability of microarray data classifiers, we propose a novel gene selection method based on the improvement of GEP. This proposed algorithm is called Gene Selection Programming (GSP). The idea behind this approach is to control the GEP solution process by replacing the random adding, deleting and selection with the systematic gene-ranking based selection. In this paper four innovative operations are presented: attributes/genes selection (initializing the population), mutation operation, recombination operation and a new fitness function. More details of GSP are presented in the Methods section.
In this work, support vector machine (SVM) with a linear kernel serves to evaluate the performance of GSP. For a better reliability we used leave-one-out cross validation (LOOCV). The results were evaluated in terms of three metrics: classification accuracy, number of selected genes and CPU time.
The rest of this paper is organized as follows: The overview of GEP and the proposed gene selection algorithm GSP are presented in the Methods section (section 2). Results section (section 3) provides the experimental results on ten microarray datasets. Discussion section (section 4) presents the statistical analysis and discussion about the experimental results. Finally, Conclusion section (section 5) gives the conclusions and directions of future research.

Gene expression programming
Gene Expression Programming (GEP) algorithm is an evolutionary algorithm. GEP consists of two parts. The first part is characteristic linear chromosomes (genotype), which are composed of one or more genes. Each gene consists of a head and a tail. The head may contain functional elements like {Q, +, −, ×, /} or terminal elements, while the tail contains terminals only. The terminals represent the attributes in the datasets. In this study, sometimes we use the term attribute to represent the gene in microarray dataset to avoid the possible confusion between the gene in microarray datasets and the gene in GEP chromosome. The size of the tail (t) is computed as t = h (n-1) + 1, where h is the head size, and n is the maximum number of parameters required in the function set. The second part of GEP is a phenotype which is a tree structure also known as expression tree (ET). When the representation of each gene in the chromosome is given, the genotype is established. Then the genotype can be converted to the phenotype by using specific languages invented by the GEP author.
GEP process has four main steps: initialize the population by creating the chromosomes (individuals), identify a suitable fitness function to evaluate the best individual, conduct genetic operations to modify the individuals to achieve the optimal solution in the next generation, and check the stop conditions. GEP flowchart is shown in Fig. 1.
It is worth mentioning that the GEP algorithm faces some challenging problems, especially the computational efficiency, when it is applied on the complex and high-dimensional data such as a microarray dataset. This motivates us to solve these problems and further improve the performance of the GEP algorithm by improving the evolution process. The details of the proposed gene selection programming (GSP) algorithm, which is based on GEP, for cancer classification are given in the following sub-sections.

Systematic selection approach to initial GSP population
Initializing population is the first step in our gene selection method for which candidates are constructed from two sets: terminal set (ts) and function set (fs). Terminal set should represent the attributes of the microarray dataset. The question is what attributes should be selected into the terminal set. Selecting all attributes (including the unrelated attributes) will affect the computational efficiency. The best way to reduce the noise from the microarray data is to minimize the number of unrelated genes. There are two commonly used ways to do that: either by identifying a threshold and the genes ranked above a threshold are selected, or by selecting the n-top ranked genes (e.g. top 50 ranked genes). Both ways have disadvantages: defining a threshold suitable for different datasets is very difficult and deciding how many genes should be selected is subjective. To avoid these disadvantages, we use a different technique named systematic selection approach.
The systematic selection approach consists of three steps: rank all the attributes, calculate the weight of each attribute, and select the attributes based on their weight using the Roulette wheel selection method. The details of these steps are shown in the following sub-sections.

Attribute ranking
We use the Information Gain (IG) algorithm [33] to rank the microarray attributes. IG is a filter method mainly used to rank and find the most relevant genes [15,34,35]. The attributes with a higher rank value have more impact on the classification process, while the attributes with a zero rank value are considered irrelevant. The rank values of all attributes are calculated once and saved in the buffer for later use in the program.

Weight calculation
The weight (w) of each attribute (i) is calculated based on Eq (1) where sum ¼ P i r i ∀ i ∈ ts and r is the rank value, and P i w i ¼ 1.
The attributes with a higher weight contain more information about the classification.

Attribute selection
In our systematic selection approach, we use the Roulette wheel selection method, which is also known as proportionate selection [36], to select the strong attributes (i.e., the attributes with a high weight). With this approach all the attributes are placed on the roulette wheel according to their weight. An attribute with a higher weight has a higher probability to be selected as a terminal element. This approach could reduce the number of irrelevant attributes in the final terminal set. The population is then initialized from this final terminal set (ts) and the function set (fs).
Each chromosome (c) in GSP is encoded with the length of N*(gene_length), where N represents the number of genes in each chromosome (c) and the length of a gene (g) is the length of its head (h) plus the length of its tail (t). In order to set the effective chromosome length in GSP, we need to determine the head size as well as the number of genes in each chromosome (details are in the Results section). The process of creating GSP chromosomes is illustrated in Algorithm 1

Fitness function design
The objective of the gene selection method is to find the smallest subset of genes that can achieve the highest accuracy. To this end, we need to define a suitable fitness function for GEP that has the ability to find the best individuals/chromosomes. We define the fitness value of an individual chromosome i as follow: This fitness function consists of two parts. The first part is based on the accuracy result AC(i). This accuracy is measured based on the support vector machine (SVM) classifier using LOOCV.
For example, if chromosome i is +/Qa 2 a 1 a 5 a 6 a 3, its expression tree (ET) is Then, the input values for the SVM classifier are the attributes a 2 , a 1 and a 5 .
The second part of the fitness function is based on the number of the selected attributes s i in the individual chromosome and the total number t of attributes in the dataset . Parameter r is a random value within the range (0.1, 1) giving an importance to the accuracy with respect to the number of attributes. Since the accuracy value is more important than the number of selected attributes in measuring the fitness of a chromosome, we multiply the accuracy by 2r.

Improved genetic operations
The purpose of the genetic operations is to improve the individual chromosomes towards the optimal solution. In this work, we improve two genetic operations as shown below.

Mutation
Mutation is the most important genetic operator. It makes a small change to the genomes by replacing an element with another. The accumulation of several changes can create a significant variation. The random mutation may result in the loss of the important attributes, which may reduce the accuracy and increase the processing time.
The critical question of mutation is which attributes are to be added or deleted. Ideally, each deleted terminal/function in the mutation operation should be covered by some other selected terminals/functions. This requirement can be fulfilled by using our method. To clarify the GSP mutation operation, we provide a simple example in Fig. 2.
In the example, the chromosome c has one gene. The head size is 3, so the tail length is h (n-1) + 1=4 and the chromosome length is (3+4) =7. The weight table shows that the attribute with the highest weight in the chromosome is a 9 and the attribute with the lowest weight is a 1 . With the mutation GSP method selects the weakest terminal lt (the terminal with lowest weight) which is a 1 in our example. There are two options to replace a 1 : the program could select either a function such as (/) or a terminal to replace it. In the latter option, the terminal should have a weight higher than that of a 1 , and the fitness value of the new chromosome c`must be higher than the original one. This new mutation operation is outlined in Algorithm 2.

Recombination
The second operation that we use in our gene selection method is the recombination operation. In recombination, two parent chromosomes are randomly chosen to exchange some material (short sequence) between them. The short sequence can be one or more elements in a gene (see Fig. 3). The two parent chromosomes could also exchange an entire gene in one chromosome with another gene in another chromosome.
In this work, we improve the gene recombination by controlling the exchanging process. Suppose c 1 and c 2 are two chromosomes (see Fig. 4). The fitness value of c 1 = 80% and the fitness value of c 2 = 70% based on our fitness function (2). We select the "strong" gene (the one with the highest weight summation) from the chromosome that has the lowest fitness value (lc) and exchange it with the "weak" gene (the one with the lowest weight summation) from another chromosome that has the highest fitness value (hc). In general, this process increases the fitness of hc. We repeat the exchange process until we get a new chromosome (hc') with a higher fitness value than that of both parent chromosomes. The hc`has a higher probability of being a transcription in the next generation. This idea comes from the gene structure [37].
Based on the above innovative improvements for the GSP method in this section, we present the steps of GSP in Algorithm 3 with pseudocode.

Results
In this section, we evaluate the performance of GSP method using ten microarray cancer datasets, which were downloaded from http://www.gems-system.org. Table 1 presents the details of the experimental datasets in terms of diverse samples, attributes and classes.
Our experimental results contain three parts. Part 1 (Ev.1) evaluated the best setting for GSP based on the number of genes (g) in each chromosome and the head size (h). Part 2 (Ev.2) evaluated the GSP performance in terms of three metrics: classification accuracy, number of selected genes and CPU Time. To guarantee the impartial classification results and avoid generating bias results, this study adopted cross validation method LOOCV to reduce the bias in evaluating their performance over each dataset. Our gene selection results were compared with three gene selection methods using the same classification model for the sake of fair competition. Part 3 (Ev.3) evaluated the overall GSP performance by comparing it with other up-to-date models.

Ev.1 the best setting for gene and head
To set the best values for the number of genes (g) of each chromosome and the size of the gene head (h) in the GSP method, we evaluated nine different settings to show their effect on the GSP performance results. For g we selected three values 1, 2 and 3, and for each g value we selected three h values: 10, 15 and 20. We increased the values of h by 5 to make it clear to observe the effect of h values on the GSP performance, especially when the effect of increasing h is very slight. For more reliability, we used three different datasets (11_Tumors, Leukaemia 1, Prostate Tumor). The parameters used in GSP are listed in Table 2.
The average results across the three experimental datasets are presented in Table 3. AC avg , N avg and T avg represent the average accuracy, number of selected attributes and CPU time respectively for ten runs, while AC std , , N std . and T std. represent the standard deviation for the classification accuracy, number of selected attributes and CPU time respectively. Figure 5 shows the evaluation values in terms of AC avg , T avg . and N avg for three different numbers of genes in each chromosome.
It is observed from the results in Table 3 that: 1-Comparing g with h: g has a stronger effect on the results than h.   In order to evaluate the performance of our GSP algorithm objectively, we first evaluated its performance in terms of three evaluation criteria: classification accuracy (AC), number of selected attributes (N) and CPU Time (T). Then we compared the results with three popular gene selection algorithms named Particle Swarm Optimization (PSO) [48], GEP and GA [49] using the same model for the sake of a fair comparison. The parameters of the comparison methods are listed in Table 4.
The Information Gain algorithm was used in order to filter irrelevant and noisy genes and reduce the computational load for the gene selection and classification methods.
The support vector machine (SVM) with a linear kernel served as a classifier of these gene selection methods. In order to avoid selection bias, the LOOCV was used. Weka  software was used to implement the PSO and GA models with default settings, while the GEP model was implemented by using java package GEP4J [50]. Table 5 shows the comparison results of GSP with three gene selection algorithms across ten selected datasets.
The experimental results showed that the GSP algorithm achieved the highest average accuracy result (99.92%) across the ten experimental datasets, while the average accuracies of other models were 97.643%, 97.886% and 94.904% for GEP, PSO and GA respectively.
The standard deviation results showed that GSP had the smallest value (0.342671), while the average standard deviations were3.425399, 3.3534102 and 5.24038421 for GEP, PSO and GA respectively. This means the GSP algorithm made the classification performance more accurate and stable.
The GSP algorithm achieved the smallest number of predictive/relevant genes (8.16), while the average number of predictive genes was 13.8, 16.14 and 473.5 for GEP, PSO and GA respectively.
These evaluation results show that GSP is a promising approach for solving gene selection and cancer classification problems.
CPU Time results showed that GSP took almost half of the time that GEP needed to achieve the best solution. However, the time is still long compared with the PSO and GA methods.

Ev.3: Comparison of GSP with up-to-date classification models
For more evaluations, we compared our GSP model with up-to-date classification models IBPSO, SVM [14], IG-GA [35], IG-ISSO [15], EPSO [21] and mABC [22]. This comparison was based on the classification result and the number of genes regardless of the methods of data processing and classification. The comparison results on ten datasets are presented in Table 6.
It can be seen from Table 6 that GSP performed better than its competitors on seven datasets (11_Tumors, 9_Tumors, Lung_ Cancer, Leukemia1, Leukemia2, SRBCT, and DLBCL), while mABC had better results on three data sets (Brain_Tumor1, Brain_Tumor2, and Prostate).
Interestingly, all runs of GSP achieved 100% LOOCV accuracy with less than 5 selected genes on the Lung_Cancer, Leukemia1, Leukemia2, SRBCT, and DLBCL datasets. Moreover, over 98% classification accuracies were obtained on other datasets. These results indicate that GSP has a high potential to achieve the ideal solution with less number of genes, and the selected genes are the most relevant ones.
Regarding the standard deviations in Table 6, results that produced by GSP were almost consistent on all datasets. The differences of the accuracy results and the number of genes in each run were very small. For GSP, the highest AC std was 0.52 while the highest N std was 1.5. This means that GSP has a stable process to select and produce a near-optimal gene subset from a high dimensional dataset (gene expression data).

Discussion
We applied GSP method on ten microarray datasets. The results of GSP performance evaluations show that GSP can generate a subset of genes with a very small number of related genes for cancer classification on each dataset. Across the ten experimental datasets, the maximum number of selected genes is 17 with the accuracy not less than 98.88%.
The performance results of GSP and other comparative models (see Table 6) on Prostate and Brain tumor datasets were not as good as the results on other datasets. This is probably due to the fact that these models concentrated on reducing the number of irrelevant genes, but ignored other issues such as the missing values and redundancy. More effort needs to be made  on microarray data processing before applying the GSP model to achieve better results. The GSP method on datasets 11_Tumors and 9_Tumors achieved relatively lower accuracy results (99.88% and 98.88% respectively) compared with the accuracy results on other datasets. The reason was due to the high number of classes (11 and 9 respectively) which could be a problem to any classification models. We noticed from GSP performance that when the accuracy increased the number of selected genes and the processing time decreased (negative relationship). This proves that GSP is effective and efficient for gene selection method.

Conclusions
In this study, we have proposed an innovative gene selection algorithm (GSP). This algorithm can not only provide a smaller subset of relevant genes for cancer classification but also achieve higher classification accuracies in most cases with shorter processing time compared with GEP. The comparisons with the representative state-of-art models on ten microarray datasets show the outperformance of GSP in terms of classification accuracy and the number of selected genes. However, the processing time of GSP is still longer than that of PSO and GA models. Our future research direction is to reduce the processing time of GSP while still keeping the effectiveness of the method.


s2fieldsofstudy:
Computer Science
Mathematics

text:
From Key Encapsulation to Authenticated Group Key Establishment—A Compiler for Post-Quantum Primitives †

Assuming the availability of an existentially unforgeable signature scheme and an (IND- CCA secure) key encapsulation mechanism, we present a generic construction for group key establishment. The construction is designed with existing proposals for post-quantum cryptography in mind. Applied with such existing proposals and assuming their security, we obtain a quantum-safe three-round protocol for authenticated group key establishment that requires only one signature per protocol participant.

Introduction
To enable confidential communication among a group of two or more users over an insecure network, cryptography provides group key establishment procotols. Dealing with a non-trusted communication infrastructure, the question of authenticating protocol participants naturally arises in this context as well. The resulting cryptographic protocols are commonly application-agnostic and focus only on the task of establishing a shared high-entropy secret among the legitimate users, not mandating any particular usage of that secret key in subsequent applications. The resulting task, authenticated group key establishment (AGKE), is fairly well-understood, even though there is a remarkable diversity in the details of security models in use. A standard technique to derive an AGKE solution is to apply some form of protocol compiler or generic framework to a passively secure solution. If a public-key infrastructure is available, signatures provide an adequate mechanism. This results commonly in protocols with a substantial number of signatures being computed, transmitted, and verified:

•
In the Katz-Yung compiler [1], for each message sent in the original protocol, a signature has to be computed and transmitted (and verified). For instance, Apon et al. [2] propose the application of this compiler for their unauthenticated group key establishment solution.  Bohli's framework for robust group key agreement [4] targets two-round protocols, and in each round each participant sends a signed message (and verifies signatures by all other participants).
For our purposes, the KEM needs to be perfectly correct, i.e., we require that for all key pairs (pk, sk) generated by KeyGen the following correctness condition holds: if (K, C) is an output of Encaps(pk), then Decaps sk (C) = K.
The most relevant notion to capture the security of a KEM is that of indistinguishability under chosen-ciphertext attacks (also defined in [12]), which is once again similar to the usual one for PKEs, except that the adversary is now asked to distinguish between an honestly encapsulated key and a pseudorandom value. We provide a formal definition below.
Definition 2 (IND-CCA Security). A KEM is IND-CCA secure if the advantage of any probabilistic polynomial time adversary A in the game described in Figure 1, as a function in the security parameter, is negligible. Here the advantage of an adversary A is defined as Adv IND-CCA Setup: the challenger C runs the key generation KeyGen to obtain a key pair (pk, sk) and hands the public key pk to the adversary A. Queries: the adversary A is given access to a decapsulation oracle, which, when queried on an encapsulation C, returns its corresponding decapsulated symmetric key K. Challenge: the challenger C computes (K 0 , C * ) ← Encaps(pk) and selects uniformly at random K 1 from the symmetric key space. Then C selects a bit b ∈ {0, 1} uniformly at random and hands (K b , C * ) to A. Queries: the adversary A is given access to a decapsulation oracle, which, when queried on an encapsulation C, returns its corresponding decapsulated symmetric key K. The challenge encapsulation C * must not be queried by the decapsulation oracle. Guess: the adversary outputs b ∈ {0, 1} and wins if and only if b = b , and C * was not queried by the decapsulation oracle.

Signature Scheme
To formalize a signature scheme, we follow the usual approach.

Definition 3 (Signature).
A signature is a triple of polynomial time algorithms (SigKeyGen, Sign, Verify) as follows: • SigKeyGen is probabilistic. Given the security parameter , it generates a pair of public and secret keys (vk, sigk). • Sign is probabilistic. Given a secret key sigk and a message M it generates a signature σ. • Verify is deterministic. Given a public key vk, a signature σ, and a message M, this algorithm outputs 1 if the signature is valid and 0 otherwise.
We require that for all key pairs (vk, sigk) generated by SigKeyGen and for all messages M the following correctness condition holds: If σ is an output of Sign(sigk, M), then Verify vk (σ, M) = 1.
The security notion usually required for signatures is existential unforgeability under chosen message attacks, and we capture it by the following definition.
Definition 4 (EUF-CMA security). A signature scheme is EUF-CMA secure if the advantage of any probabilistic polynomial time adversary A in the game described in Figure 2 is negligible. Here the advantage of an adversary A is defined as the function Adv EUF-CMA is the event that A wins.

•
Setup: the challenger C runs the key generation SigKeyGen to obtain a key pair (vk, sigk) and hands the public key vk to A. • Queries: the adversary A is given access to a signing oracle. When queried for message m, the challenger runs algorithm Sign on input m and sigk and returns the corresponding output σ to the adversary. • Forgery: the adversary A outputs a pair (µ, σ) and wins if and only Verify vk (σ, µ) = 1 and no query for a signature on µ was asked.

Security Model and Security Goals
The security model we use follows the oracle-based approach to capture adversarial capabilities-this approach is also used by Katz and Yung [1], for instance, building on the work of Bresson et al., work [13].

Protocol Participants
Given a security parameter , we assume that the set of protocol participants U is polynomial in . We model each user U ∈ U as a probabilistic polynomial time algorithm which is allowed to execute a polynomial number of protocol instances Π s U (s ∈ N) concurrently. We assume user identities to be bitstrings of identical length and to keep notation simple, throughout we will use U referring to both the bitstring identifying a user U and the algorithm U itself. Let Π s U be a protocol instance, the following seven variables are associated with it: pid s U : stores the identities of those users in U with which a key is to be established a particular instance aims at establishing a key with, including U (this ensures being partnered is thus a reflexive relation); sid s U : stores a session identifier, i.e., a non-secret public identifier for the session key sk s U ; sk s U : stores a distinguished NULL value and after a successful protocol run holds the session key; acc s U : is set to TRUE if the session key stored in sk s U has been accepted; state s U : keeps state information needed while executing the protocol (e.g., the secret scalars used as ephemeral Diffie-Hellman keys); term s U : is set to TRUE if this protocol execution has terminated; used s U : indicates if this instance is used, i.e., currently involved in a protocol execution.

Initialization
Before actual protocol executions take place, we allow an optional trusted initialization phase without adversarial interference. During this phase, for each user public and private key pairs (pk U , ak U ) can be generated and distributed accordingly. These private and public keys can include the ones corresponding to a KEM, a signature scheme, etc. The initialization phase can also be used to issue (possibly needed) further public parameters.

Adversarial Capabilities and Communication Network
The adversary A is represented as a probabilistic polynomial time algorithm with full control over the communication network. The network is, therefore, fully asynchronous, non-private, allowing arbitrary point-to-point connections among users. More specifically, we express the capabilities of the adversary through the following oracles: Send(U, s, M) : This oracle can be used in two ways.

•
The adversary can initialize a protocol execution; sending the special message M = {U i 1 , . . . , U i r } ⊆ U to an unused instance Π s U with U ∈ M initializes a protocol run among U i 1 , . . . , U i r . After such a query, Π s U sets pid s U := {U i 1 , . . . , U i r }, used s U := TRUE, and processes the first step of the protocol. • The message M is sent to instance Π s U . The oracle returns the protocol message output by Π s U after receiving M. Reveal(U, s) : returns the stored session key sk s U if acc s U = TRUE and a NULL value otherwise. Corrupt(U) : for a user U ∈ U this query returns U's long-term secret key ak U .
Notice that, unlike Reveal, Corrupt refers to a user instead of to an individual protocol instance. We consider an adversary to be active if it has access to all of the above oracles. To capture a passive adversary, we can replace access to Send oracle with access to an Execute oracle, which returns a complete protocol transcript among the specified unused instances. An active adversary can simulate this Execute oracle using Send in the natural way.
For technical reasons, we introduce one more oracle, Test, and A must submit exactly one query of the form Test(U, s) with an instance Π s U that has accepted a session key, i.e., with acc s U = TRUE. In response to such a query, a bit b ← {0, 1} is sampled uniformly at random. For the case b = 1, the established session key stored in sk s U is returned. For b = 0 the output is a uniformly random element sampled from the space of session keys. The idea is that for a secure group key establishment protocol, no efficient adversary can distinguish between the cases b = 0 and b = 1. To turn this idea into a definition, we exclude trivial cases and focus on correct group key establishment protocols: Correctness refers to a scenario where no attack takes place; to formulate security guarantees we need to specify the circumstances under which a correct guess for the random bit used by the Test oracle constitutes a possible attack. For this we use the following notions of partnering and freshness.
Based on this notion, we can determine what a fresh instance is, i.e., an instance where the adversary does not know the session key for trivial reasons.
The following formulation allows an adversary A to reveal all secret keys without violating freshness, provided A does not send any "relevant" messages afterwards. Therefore, security in the sense of Definition 8 below implies forward secrecy: For some U j ∈ pid s i i a Corrupt(U j ) query was executed before a query of the form Send(U k , s k , * ) has taken place where U k ∈ pid s i i .

•
A query Reveal(U j , s j ) with Π s i i and Π s j j being partnered occurred.
We write Succ ke A for the event that A queries Test with a fresh instance according to Definition 7 and outputs a correct guess for the Test oracle's bit b.

Definition 8 (Semantic security).
A key establishment protocol is said to be semantically secure, if the advantage Adv ke A = |2 · Pr[Succ ke A ] − 1| is negligible for all probabilistic polynomial time algorithms A.
To make explicit that adversaries are considered to be active, i.e., have access to the Send oracle, it is common to refer to a group key establishment protocol as authenticated. On the other hand, we speak of unauthenticated group key establishment if the adversaries are passive.

Proposed Construction
The proposed generic construction for designing an authenticated group key establishment for n users U 0 , . . . , U n−1 is shown in Figure 3. It builds on an available key encapsulation mechanism and a signature scheme. The following theorem shows that the proposed construction offers a provable security guarantee, if the underlying primitives meet standard security requirements.
Set up: A pair of keys (vk i , sigk i ) for the signature scheme S is generated for each U i , which gets the secret key sigk i while vk i is publicized.

Round 1: Computation.
Each U i generates an ephemeral pair of keys for E : Broadcast. Each U i sends to his right a neighbor the message (pk i , U i+1 ) Round 2: Computation. Each U i , using the ephemeral public key he has received, computes an encapsulation for that key: Broadcast. Each U i does sends to his left neighbor the message (C i−1 , U i−1 ) Round 3: Computation. Each U i does the following: • Using his ephemeral secret key, decapsulates the received key K i ← KeyDecaps sk i (C i ); • Using the encapsulated key he sent before, computes Computes a signature σ i on pk 0 , . . . , pk n−1 , C 0 , . . . , C n−1 , X i , pid i Broadcast. Each U i broadcasts (X i , σ i ) Key Computation. Check. Each U i checks all the signatures, equality of pids, X 0 ⊕ · · · ⊕ X n−1 = 0; if any check fails, aborts. Computation. Each U i does the following: Sets the session identifier sid i to pk 0 , . . . , pk n−1 , C 0 , . . . , C n−1 , X 0 , . . . , X n−1 , pid i .; • Accepts K 0 as session key.
a The ordering can be determined by means of lexicographic ordering of user identifiers, for instance, and the users are assumed to be in a circle, i.e., U 0 and U n−1 are neighbours Figure 3. A compiler achieving authenticated group key establishment from a secure key encapsulation mechanism (KEM) and a secure signature scheme. Theorem 1. Assuming S is an EUF-CMA secure signature scheme and E is an IND-CCA secure KEM, the protocol from Figure 3 is correct and semantically secure.
Proof. We can easily verify the protocol correctness: if all the participants follow the protocol description and there is no active adversarial interference, then all checks will succeed and every participant will set the same pid and sid. Moreover every participant will receive the correct {X j } n−1 j=0 and consequently they will be able to compute the same session key K 0 .
To illustrate the security of our compiler, we use the "game hopping" technique, where we let the adversary A interact with a simulator B. We denote by Adv(A, G i ) the advantage of the adversary in Game i. The security parameter is denoted by . Further, we denote by q e and q s the maximum number of queries made by the adversary to the Execute and Send oracles, respectively.
Game 0. This game is identical to the original attack game, with all the oracles being simulated as in the real protocol. Therefore, Adv(A, G 0 ) = Adv ke A ( ).

Game 1. Let
Forge be the event that the adversary succeeds in forging an authenticated message pk 0 , . . . , pk n−1 , C 0 , . . . , C n−1 , X i , pid i , σ i of at least one party U i without having queried Corrupt(U i ) and where all the values signed involved pk 0 , . . . , pk n−1 , C 0 , . . . , C n−1 , X i , pid i , σ i were not output by a same U i 's instance. Any time this event occurs, we abort and mark this as success for the adversary.
An adversary A that can achieve Forge can be used to construct an adversary A that forges a signature in the EUF-CMA game: the given public key is assigned randomly to U i , one of the users; all other participants are initialized as the protocol indicates; afterwards all the queries in the security game are answered faithfully and when a signature by the chosen user is needed, the signing oracle of the EUF-CMA game is queried to produce it.
The probability of the adversary choosing U i when assigning the public key for the signature is at least 1/|U |, and with |U | being polynomial size, this is non-negligible: for a polynomial bound poly Forge on |U |. Game 2. This game is exactly as Game 1 except that a session t is chosen uniformly at random. If the Test query does not occur in the t-th session the game aborts, and we count it as win for the adversary. As the number of active protocol instances is polynomially bounded, we have for a polynomial bound poly Test on the number of protocol sessions activated. Game 3. This game is identical to the previous game, except that the simulation of the Send, Execute and Test oracles is modified as follows. The symmetric key K 0 output by Encaps(pk 0 ) in the Test instance Π t 0 is replaced with a random key K * chosen from {0, 1} . In order to bound the difference in the advantages between Games 2 and 3, we will build, from an adversary A ble to distinguish between both games, an adversary B attacking the key encapsulation mechanism E such that where Adv IND-CCA B ( ) denotes the advantage of a probabilistic polynomial time adversary B attacking KE M.To establish this bound, we assume that B, which runs A as an auxiliary algorithm, can access a simulation of KE M. Further, B executes the key generation algorithm of S for each user U i , thus obtaining a pair of keys (vk i , sigk i ) for the signature scheme. Adversary B also executes the key generation algorithm of KE M for user U 0 , and obtains the public key corresponding to users U 1 , . . . , U n . Our adversary B obtains a challenge (C * , K 0 , K 1 ) as described in Definition 1, and we have to describe how B answers to A's queries: • Whenever a query Corrupt(U i ) is made by A, B generates the keys for the signature and returns sigk i as answer to A. • To answer a Send query for Round 2 involving U 0 , B uses the challenge encapsulation C * . The rest of the answers are generated as in a real execution of the protocol.

•
To answer an Execute query by A, our adversaryB modifies the messages as described for the simulation of the Send oracle. • A Reveal query by A is answered in a similar way as a Send or Execute query. Notice that a Reveal query cannot be made on t or any partnered instance. To answer any other Reveal query B uses the decapsulation oracle of its IND-CCA game.

•
Finally, to answer a Test query, a bit b is chosen by B when starting the simulation. B will return the key K b received from the KEM challenger to A.
At some point A will output a bit b as a guess for b which will determine the output b of B for the KEM challenge. Specifically, B outputs b = 0 if and only if b = b . Taking into account that the view of A is identical to Game 2, if the answers of B's simulation of Test are real keys and to Game 3 if the answers of B's simulation of Test are random ones, we obtain that |Adv(A, To conclude the proof, we can see that the advantage of the adversary in Game 3 equals 0, as the session keys are chosen uniformly at random in {0, 1} κ . Collecting all the advantages, we can see that Adv ke A is indeed negligible.

Remark
We would like to point out that our scheme, as described above, meets precisely the security criteria described in Section 3. It would be possible to adjust our construction to capture additional security notions, for instance introducing contributory properties. In this case, only a minor modification becomes necessary. In fact, since all of K 0 , . . . , K n−1 are available to each legitimate user, one could choose K 0 ⊕ · · · ⊕ K n−1 as session key, rather than just K 0 . The proof would then be amended accordingly, as follows: in Game 3, after replacing K 0 with K * , produce a pseudorandom session key as K * ⊕ · · · ⊕ K n−1 . The pseudorandomness follows immediately from the fact that K * is chosen uniformly at random in {0, 1} .

Instantiation
To instantiate the above AGKE construction, there are various natural options both for KEMs and signature schemes. Some possible candidates for quantum-safe KEMs include [14][15][16][17]. Moreover, examples for candidates of post-quantum signature schemes are offered by [9,18]. The particular choice of schemes can be made taking into account their efficiency or the assumption their security relies on. Different applications may have different preferences for prioritizing, e. g., public-key size over signature size.

Conclusions
The protocol compiler presented here offers a convenient approach to systematically design authenticated group-key establishment protocols in a post-quantum scenario. Keeping the number of signatures needed low (one per user), gives a protocol designer flexibility in the type of post-quantum signature signature scheme to be deployed. With three rounds, the round complexity that is achievable with currently available key encapsulation mechanisms appears quite attractive, too.
Funding: This research was funded in part by the NATO Science for Peace and Security Programme under grant G5448 and through research project MTM2017-83506-C2-2-P by the Spanish MICINN.


Reconstruction of Inner Worm Passage of Quercus Mongolica Seeds

This paper takes Quercus mongolica as an example, In order to determine the position of insects inside the Quercus mongolica seed. The seeds of Quercus mongolica were cut at a certain thickness to obtain an internal crosssectional image of the seeds. In this paper, Canny and Hough algorithm are used to perform edge detection and line detection on unprocessed images, and determine the coordinate origin of the study area so that all study areas are in the same coordinate range. In order to obtain the position information of the insect path, based on the RGB principle, this paper designs a screening algorithm for the pixel points in the study area, and obtains the position information of the insect track. In order to observe the insect path more intuitively, this paper establishes a three-dimensional data model, and uses MATLAB to draw a three-dimensional structure diagram of the position information of the insect path, and shields the interference items such as seed germ to obtain a more accurate movement track of the insect.

INTRODUCTION
In recent years, due to the destruction of the forest ecological environment, the Quercus mongolica forest pests are more serious, affecting the natural regeneration process of the Quercus mongolica forest. In this paper, we find a rationalization solution for the Mongolian forest pests and establish a model of the position information of the insects inside the seeds. The Quercus mongolica seed was cut at a thickness of 0.1 mm to obtain an internal cross-sectional image of the seed. For coordinate unification and distance measurement, reduce the computational trouble, give the marked points A and B, and the actual distance measured between the two points of AB should be 20 mm. Involving the batch processing problem of photos, it is difficult to carry out detailed research in the case of device jitter and area acquisition too large, so it is necessary to batch cut and position the image size and position so that all research areas are in the same Within a coordinate range. It is necessary to establish a suitable model, determine the two sides of the study area, and then determine the coordinate origin of the study area. According to the coordinate origin, select the appropriate study area, and then batch-cut the study area of all images. On the basis of the above, the program is written by MATLAB software, according to the actual distance of points A and B, the image is divided into 200×200, so that each picture is divided into 40,000 pixel blocks, and a suitable model is established to perform the pixel block. Screen to determine the location of the worm. The MATLAB program is written to establish a 3D model of the insect tract, and the interference factor is judged by observing the 3D model of the wormhole, thereby shielding and optimizing.

Model 1 Establishment
First, the image is processed for observation. The rgb2gray function is used to grayscale the image, so that the edge of the image research area is more obvious, and then the appropriate algorithm is used for edge detection. Here we use the Canny algorithm.
In order to minimize the effect of noise on the edge detection results, noise must be filtered out to prevent false detection caused by noise. To smooth the image, a Gaussian filter is used to convolve with the image, which smoothes the image to reduce the apparent noise effects on the edge detector. The generating equation for a Gaussian filter kernel of size (2k+1) * (2k+1) is given by: (1) The edges in the image can point in all directions, so the Canny algorithm uses four operators to detect horizontal, vertical, and diagonal edges in the image. The edge detection operator returns the first derivative value in the horizontal Gx and vertical Gy directions, thereby determining the gradient G and the direction θ of the pixel.
Where G is the gradient strength, θ is the gradient direction, and arctan is the inverse tangent function Then use Hough algorithm to determine the upper line: The basic principle of Hough transform is to use the duality of points and lines to change the given curve of the original image space into a point of the parameter space through the curve expression. This translates the detection problem of a given curve in the original image into a peak problem in finding the parameter space. That is, the detection of the overall characteristics is converted into the detection of local characteristics. Using the Hough function to detect the upper line and the left line of the study area to determine the coordinates of A. By taking the point of Figure 1, we obtain the pixel distance between AB, and finally use the coordinates of point A and the distance of AB. Batch cropping images.

Solution of Model 1
By using the Canny algorithm and constantly adjusting the threshold, we obtained an area of significant edge research, as shown in Figure 3: Through the taking point of Figure 3, we get the distance between the marked points A and B as 800 pixels, and then use the Imcrop function to cut: pic_1 = imcrop(image,[x,y,799,799]) The cropped image we got is shown in Figure 1 and Figure 2.

Model 2 Establishment
To determine the position information of the worm tract in the study area, it is necessary to have obvious differences. By observing, we find that the color characteristics of the worm tract are different from other positions, so we use RGB to build a model to screen the wormPixel position.
Take a representative picture of the difference between the color characteristics of the insect and noninsect position. Use Photoshop software to RGB the insect and non-insect parts, and get the RGB range and non-insect path of the insect position. The RGB range, then use MATLAB to write the program, judge, obtain the position information of the pixel points in line with the RGB range of the insect channel and store it.
However, after calculation, the pixels in the study area are high. If each pixel is filtered by the above program, the amount of data is too large and the operation time is too long. Therefore, we need to reduce the resolution of the study area. On the basis of the first question (the study area is 800*800 pixels resolution, and the actual distance of AB point is 20 mm), and the thickness is 0.1 mm, we will make each picture 200*200. Splitting, each picture is divided into 40,000 pixel small blocks, each small block has a resolution of 4*4 pixels, which indirectly reduces the research area to a resolution of 200*200.
Next, we just need to determine which small blocks of pixels are on the insect track and record the position information of these small pieces of pixels. Since each pixel has 16 pixels, and considering that some small pixels are at the junction of the insect and the non-insect, we determine that if more than half of the pixels of a pixel are RGB in the worm Within the RGB range of the location, we consider the pixel block to be in the insect track and record the location information of the pixel block.

Solution of Model 2
We use Photoshop to color the RGB of ten images. Since the lower the value of RGB, the darker the color, we only need to take the highest value of RGB position ( Table  1) and The lowest value of the position of the insect ( Table  2) the data obtained is as follows:  Since the color of the germ position is darker and the RGB values are lower, we use the integrity of the position information of the RGB values. Therefore, the value of the upper limit of the RGB value determined by us is relatively high. Finally, when optimizing, the part of the germ is removed. location information. In summary, the value range of R is determined to be R<140, the range of G is G<120, and the range of B is B<80.
Divide the study area into 40,000 pixel blocks and store them in the cell array, and then extract the R, G, and B values of the elements in the cell array. Determine the R, G, and B values and store the matching locations. The location information of the pixel block.
Then merge them in the form of an array, construct a two-dimensional array K, and then give the z-axis information that meets the conditional small block, and add it to K to make it into a three-dimensional array, and put each loop the obtained K is saved in F and stored in the file. After optimizing it, it is stored in the file. We get the data about the position information of the insect path to indicate the position of the insect path.

Model 3 Establishment
According to the data obtained by Model 2, we draw the data in a three-dimensional image, and can intuitively obtain the three-dimensional structure of the insect track. However, because the value of RGB in question 2 is slightly higher, there may be position information of the germ. And the emergence of other interference information.
By visually observing the 3D structure diagram, we use MATLAB to write programs to remove germ position information and other interference information for optimization.

Solution of Model 3
The initial data obtained by the model 2 is subjected to 3D composition, with A as the coordinate origin and AB as the y-axis, and the following figure is obtained:

III. CONCLUSION
The Canny and Hough algorithms can accurately determine the coordinate origin of the study area. The RGB design screening algorithm accurately identifies the pictures with large color differences, which greatly reduces the complexity of the optimization program and facilitates further research. The established image segmentation model reduces the amount of data processing, optimizes the computational efficiency of the RGB filtering algorithm, reduces the program running time, and increases the practicability of the program. This paper adopts a more reasonable method to describe the position information of insects, which can reflect the harm of insects to seeds with more comprehensive information. Using it to simulate the algorithm, it has a certain method to resist the pests, which helps people to innovate the resistance to forest pests.