extfieldsofstudy:
Computer Science
Mathematics

text:
MINT: Deep Network Compression via Mutual Information-based Neuron Trimming

Most approaches to deep neural network compression via pruning either evaluate a filter's importance using its weights or optimize an alternative objective function with sparsity constraints. While these methods offer a useful way to approximate contributions from similar filters, they often either ignore the dependency between layers or solve a more difficult optimization objective than standard cross-entropy. Our method, Mutual Information-based Neuron Trimming (MINT), approaches deep compression via pruning by enforcing sparsity based on the strength of the relationship between filters of adjacent layers, across every pair of layers. The relationship is calculated using conditional geometric mutual information which evaluates the amount of similar information exchanged between the filters using a graph-based criterion. When pruning a network, we ensure that retained filters contribute the majority of the information towards succeeding layers which ensures high performance. Our novel approach outperforms existing state-of-the-art compression-via-pruning methods on the standard benchmarks for this task: MNIST, CIFAR-10, and ILSVRC2012, across a variety of network architectures. In addition, we discuss our observations of a common denominator between our pruning methodology's response to adversarial attacks and calibration statistics when compared to the original network.

I. INTRODUCTION
Balancing the trade-off between the size of a deep network and achieving high performance is the most important constraint when designing deep neural networks (DNN) that can easily be translated to hardware. Although deep learning yields remarkable performance in real-world problems like medical diagnosis [1]- [3], autonomous vehicles [4]- [6], and others, they consume a large amount of memory and computational resources which limit their large-scale deployment. With current state-of-the-art deep networks spanning hundreds of millions if not billions of parameters [7], [8], compressing them while maintaining high performance is challenging.
In this work, we approach DNN compression using network pruning [9]. There are two broad approaches to network pruning, (a) unstructured pruning, where a filter's importance is evaluated using weights [9] or constraints like the l 1 norm [10] on them, without considering the overall structure of sparsity induced, and (b) structured pruning, where the objective function is modified to include structured sparsity constraints [11]. Most structured pruning approaches ignore the dependency between layers and the impact of pruning on downstream layers while unstructured pruning methods force the network to optimize a harder and more sensitive optimization objective. The underlying common theme between both approaches is the use of filter weights as a proxy for their importance.
Evaluating a filter's importance purely from its weights is insufficient since it does not take into account the dependencies between filters or account for any form of uncertainty. These factors are critical since higher weight values do not always represent its true importance and a filter's contribution can be compensated elsewhere in the network. Consider the example shown in Fig. 1, where a simple weight-based criterion suggests the removal of small valued weights. However, the mutual information (MI) score, which we use to measure the dependency between pairs of filters and emphasize their importance, values the smaller weights over the large ones. Pruning based on the MI scores would ensure a network where the retained filters pass on as much information as possible to the next layer.
To overcome these issues, we propose Mutual Informationbased Neuron Trimming (MINT) as a novel approach to compression-via-pruning in deep networks which stochastically accounts for the dependency between layers. Fig. 2 outlines our approach. In MINT, we use an estimator for conditional geometric mutual information (GMI), inspired by [12], to measure the dependency between filters of successive layers. Specifically, we use a graph-based criterion (Friedman-Rafsky Statistic [13]) to measure the conditional GMI between the activations of filters at layer l and l + 1, denoted by F  Fig. 2: Illustration of the experimental setup highlighting the components of MINT. Between every pair of filters in consecutive layers (l, l + 1) we compute the conditional geometric mutual information (GMI), using the activations from each filter, as the importance score. The total number of filters in each layer is defined by N (l) and N (l+1) . The conditional GMI score indicates the importance of a filter in layer l's contribution towards a filter in layer l + 1. We then threshold filters based on the importance scores to ensure that we retain only filters that pass the majority of the information to successive layers. Finally, we retrain the network once to maintain a desired level of performance evaluating all such dependencies, we sort the importance scores between filters of every pair of layers in the network. Finally, we threshold a desired percentage of these values to retain filters that contribute the majority of the information to successive layers. Hence, MINT maintains high performance with compressed and retrained networks. Through MINT, we contribute a network pruning method that addresses the need to use dependency between layers as an important factor to prune networks. By maintaining filters that contribute the majority of the information passed between layers, we ensure that the impact of pruning on downstream layers is minimized. In doing so, we achieve state-of-theart performance across multiple Dataset-CNN architecture combinations, highlighting the general applicability of our method.
Further, we empirically analyze our approach using adversarial attacks, expected calibration error, and visualizations that illustrate the focus of learned representations, to provide a better understanding of our approach. We highlight the common denominator between its security vulnerability and decrease in calibration error while illustrating the intended effects of retaining filters that contribute the majority of information between layers.

II. RELATED WORKS
Deep network compression offers a number of strategies to help reduce network size while maintaining high performance, such as low-rank approximations [14]- [16], quantization [17]- [20], knowledge distillation [21], [22], and network pruning [9], [23], [24]. In this work, we focus on network pruning since it offers a controlled set up to study and compare changes in the dynamics of a network when filters are removed. We broadly classify network pruning methods into two categories, unstructured, and structured, which we describe below. Also, we highlight common strategies to calculate multivariate dependencies and how they vary from our method.
A. Network Pruning 1) Unstructured pruning: Some of the earliest in this line of work used the second-order relationship between the objective function and weights of a network to determine which weights to remove or retain [25], [26]. Although these methods provide deep insights into the relationships within networks, their dense computational requirements and large runtimes made them less practical. They were surpassed by an alternative approach that thresholded the weight values themselves to obtain a desired level of sparsity before retraining [9]. Apart from the simplicity of this approach, it also highlighted the importance of re-training from known minima as opposed to from scratch. Instead of pruning weights in one shot, [27] offered a continuous and recursive strategy of using miniiterations to evaluate the importance of connections using their weights, remove unimportant ones and train the network. Similarly, [10] proposed pruning filters using the l 1 -norm of their weight values as a measure of importance. By melding standard network pruning using weights with network quantization and Huffman coding, [28] showed superior compression performance compared to any individual pipeline. However, the direct use of weight values across all these methods does not capture the relationships between different layers or the impact of removing weights on downstream layers. In MINT, we address this issue by explicitly computing the dependency between filters of successive layers and only retaining filters that contribute a majority of the information. This ensures that there isn't a severe impact on downstream layers.
A subset of methods uses data to derive the importance of filter weights. Among them, ThiNet [23] posed the reconstruction of outcomes with the removal of weights as an optimization objective to decide on weights. More recently, NISP [29] used the contribution of neurons towards the reconstruction of outcomes in the second to last layer as a metric to retain/remove filters. These works represent a shift to data-driven logic to consider the downstream impact of pruning, with the use of external computations (e.g., feature ranking in [29]) combined with the quality of features approximated using weights from a single trial. Compared to the deterministic relationship between the weights and feature ranking methods, our method uses a probabilistic approach to measure the dependency between filters, thereby accounting for some form of uncertainty. Further, our method uses one single prune-retrain step compared to multiple iterations of fine-tuning performed in these methods.
2) Structured Pruning: The shift to structured pruning was based on the idea of seamlessly interfacing with hardware systems as opposed to relying on software accelerators. One of the first methods to do this extended the original brain damage [25] formulation to include fixed pattern masks for specified groups while using a group-sparsity regularizer [30]. This idea was further extended in works that used the grouplasso formulation [11], individual or multiple l n norm constraints [31] on the channel parameters and in works that balanced individual vs. group relationships [32], [33] to induce sparsity across desired structures. These methods explicitly affect the training phase of a chosen network by optimizing a harder and more sensitive objective function. Further, sideeffects like-stability to adversarial attacks, calibration, and difference in learned representations have not been fully quantified. In our work, we optimize the standard crossentropy objective function and characterize the behaviour of the original network and their compressed counterparts.
Hybrid methods combine the notion of a modified objective function with the measurement of downstream impact of pruning by enforcing sparsity constraints on the outcomes of groups [34], using a custom group-lasso formulation with a squared dependency on weight values as the importance measure [35] or an adversarial pruned network generator to compete with the features derived from the original network [24]. The disadvantages of these methods include their multi-pass scheme and large training times as well as those inherited from modifying the objective function.

B. Multivariate Dependency Measures
The accurate estimation of multivariate dependency in highdimensional settings is a hard task. The first in this line of work involved Shannon Mutual Information which was succeeded by a number of plug-in estimators including Kernel Density Estimators [36] and KNN estimators [37]. However, their dependence on density estimates and large runtime complexity means they are not suitable for large scale applications including neural networks. A faster plug-in method based on graph theory and nearest neighbour ratios [38] was proposed as an alternative. More solutions that use statistics like KL divergence like [39] or Renyi-α [40] were proposed to help bypass density estimation fully. Instead, in this work, we focus on a conditional GMI estimator, similar to [12], which bypasses the difficult task of density estimation and is nonparametric and scalable.

III. MINT
MINT is a data-driven approach to pruning networks by learning the dependency between filters/nodes of successive layers. For every pair of layers in a deep network, MINT uses the conditional GMI (Section III-A) between the activations of every filter from a chosen layer l and a filter from layer l + 1, given the existence of every other possible filter in layer l to compute an importance score. Here, data flows from layer l to l + 1. Once all such importance scores are evaluated, we remove a fixed portion of filters with the lowest scores. This induces the desired level of sparsity in the network before retraining the network to maintain high accuracy. The core algorithm is outlined and explained in the sections below.

A. Conditional Geometric Mutual Information
In this section, we review conditional GMI estimation [12] and use a close approximation to their method to calculate multivariate dependencies in our proposed algorithm.
Definition We first define a general form of GMI denoted by I p : For parameters p ∈ (0, 1) and q = 1 − p consider two random variables X ∈ R dx and Y ∈ R dy with joint and marginal distributions f (x, y), f (x), and f (y) respectively. The GMI between X and Y is given by Considering the special case of p = q = 1/2 in Eqn. 1 we obtain, The conditional form of this measure is proposed in [12] as, and each filter in the previous layer l. In each ρ() computation, the highlighted filters in layer l represent the conditioned variables while the arrows indicate the filters whose actual dependence we compute. These steps are repeated for every possible combination of filters across every pair of consecutive layers in the network

B. Approach
Setup In a deep network containing a total of L layers, we compute the dependency (ρ) between filters in every consecutive pair of layers. Here, the layer l is closer to the input while layer l+1 is closer to the output among the chosen pair of consecutive layers. The activations for a given node in layer l + 1, are computed as, where x ∈ R m×d , m is the total number of samples, d is the feature dimension and x is the input to a given layer used to compute the activations. σ() is an activation function, w ∈ R N (l) , and b are the weight vector and bias. • For a given pair consecutive of layers (l, l + 1), we compute the dependency between every filter in layer l + 1 in relation to filters in layer l. The main intent of framing the algorithm in this perspective is that the activations from layers closer to the input have a direct effect on downstream layers while the reverse is not true for a forward pass of the network. • Using the activations F () for the selected filters, j ) we compute the conditional GMI between Algorithm 1: MINT pruning between filters of layers (l, l + 1) for Every pair of layers (l, l + 1), l ∈ 1, 2, . . . , them given all the remaining filters in layer l, as shown in Fig. 3. This dependency captures the relationship between filters in the context of all the contributions from the preceding layer. Since the activations of layer l + 1 are the result of contributions from all filters in the preceding layer, we need to account for this when considering the dependence of activations between two selected filters.
j ), the contribution of filters from the previous layer is either retained/removed. We define a threshold δ for this purpose, a key hyper-parameter.
i stores the indices of all filters from layer l that are retained for a selected filter F (l+1) i . The weights for retained filters are left the same while the weights for the entire kernel in the other filters are zeroed out. In the context of fully connected layers, we retain or zero out specific weight values in the weight matrix.
Group Extension While evaluating dependencies between every pair of filters allows us to take a close look at their relationships, it does not scale well to deeper or wider architectures. To address this issue, we evaluate filters in groups rather than one-by-one. We define G as the total number of groups in a layer, where each group contains an equal number of filters. We explore in detail the impact of varying the number of groups in Section IV-D. Although there are multiple approaches to grouping filters, in this work we restrict ourselves to sequential grouping, where groups are constructed from consecutive filters. There is no explicit requirement for a pre-grouping step before our algorithm so long as a balanced grouping of filters is used.
Finer Details MINT is constructed on the assumption that the majority of information from the preceding layer is available and the filter in consideration can selectively retain contributions for a subset of previous filters. This allows us to work on isolated pairs of layers with minimal interference on downstream layers since retaining filters with high MI will ensure the retention of filters that contribute the most information to the next layer. By maintaining as much information as possible between layers, the amount of critical information passed to layers further on is maintained.

IV. EXPERIMENTAL RESULTS
We breakdown the experimental results into three major sections. Section IV-C focuses on the comparison of our method to state-of-the-art deep network compression algorithms, Section IV-D highlights the significance of various hyper-parameters used in MINT, and finally, Section IV-E characterizes MINT-compressed networks w.r.t. their response to adversarial attacks, calibration statistics and learned representations in comparison to their original counterparts. As a prelude to all these three parts, we describe the datasets, models, and metrics used across our experiments. We restrict the implementation details to the appendices.

A. Datasets and Models
Our experiments are divided into the following Dataset-Architecture combinations in order of increasing complexity in dataset and architectures, MNIST [ Fig. 4: (a) An increase in the number of groups per layer allows for finer grouping of filters which in turn leads to more accurate GMI estimates and thresholding. Thus, there is a steady increase in the number of parameters that can be removed to achieve > 98.50 performance. (b) Keeping G = 20, we observe that increasing the number of samples per class improves the GMI estimate accuracy which in turn allows for better thresholding and an increase in the parameters pruned. The values on top of the bar plots are the recognition accuracies this value by using SciPy [50] to convert weight matrices to CSR format and storing them in "npz" files.

C. Comparison against existing methods
As a first step to showcasing MINT's abilities, we compare it against state-of-the-art baselines in network pruning. The baselines in Table I are arranged in ascending order of the percentage of parameters pruned. Our algorithm clearly outperforms most of the SOTA pruning baselines across the number of pruned parameters while maintaining high accuracy and reducing the memory footprint of the network. We note that while some of the pruning baselines listed use multiple prune-retrain steps to achieve their result, we use only a single step to match and outperform them.
In Fig. 5 we take a deeper look at how the overall compression % is spread throughout the network. Comparing Figs. 5a, 5b and 5c, we can establish the strong influence of datasets and network architecture on where redundancies are stored. In the cases of VGG16 and ResNet56, training with CIFAR-10 leads to the storage of possibly redundant information in the latter portion of the networks. The early portions of the network are extremely sensitive to pruning. ResNet50 when trained on ILSVRC2012 forces compression to be more spread out across the network, possibly indicating the spread of redundant features at different levels of the network.

D. Hyper-parameter Empirical Analysis
We take a closer look at two important hyper-parameters that help MINT scale well to deep networks, (a) number of groups in a layer G, and (b) the number of samples per class, m, used to compute the conditional GMI. Below, we look into how each of them impacts the maximum number of parameters pruned to achieve > 98.50% accuracy for MNIST on MLP.  Group size G directly corresponds to the number of filters that are grouped together when computing conditional GMI and thresholding. More groups lead to lesser filters per group, which allows for more fine-grained computation of multivariate dependency and thereby, more precise pruning. In this experiment, m = 250. Results in Fig. 4a match our expectation by illustrating the increase in the upper limit for parameters pruned to achieve the desired performance..

E. Characterization
The standard metric used to compare deep network compression methods is the percentage of parameters pruned while maintaining recognition performance close to the baseline. However, the original intent of compressing networks is to deploy them in real-world scenarios which necessitate other characterizations like robustness to adversarial attacks and an ability to reflect true confidence in predictions.
To understand the impact of pruning networks in the context of adversarial attacks we use two common adversarial attacks, Iterative FGSM [52], which doesn't exclusively target a desired class, and Iterative-LL [53], which targets the selection of the least likely class. Fig. 6 shows the response of other the original and MINT-compressed networks to both attacks. We clearly observe that MINT-compressed networks are more vulnerable to targeted and non-targeted attacks.
While the core idea behind MINT is to retain filters that contribute the majority of the information passed to the next layer, in using a subset of the available filters we remove a certain portion of the information passed down. Fig. 7 compares the portions of the image that contribute towards a desired target class, between the original (top row) and MINT-compressed networks (bottom row). We observe that the use of a subset of filters in the compressed network has Calibration statistics [54], [55] measure the agreement between the confidence provided by the network and the actual probability. These measures provide an orthogonal perspective to adversarial attacks since they measure statistics only for indomain images while adversarial attacks alter the input. Fig. 8 highlights the decrease in Expected Calibration Error (ECE) for the MINT-compressed networks when compared to their original counterparts. The plot illustrates that the histogram trend is closer to matching the ideal trend indicated by the linear red curve. After pruning, the sparse networks seem to behave similarly to a regularizer by focusing on a smaller subset of features and decreasing the ECE. On the other hand, the original networks contain many levels of redundancies which translates to overfitting and having higher ECE.

V. CONCLUSION
In this work, we propose MINT as a novel approach to network pruning in which the dependency between filters of successive layers is used as a measure of importance. We use conditional GMI to evaluate importance and incorporate stochasticity in our algorithm in order to retain filters that pass the majority of the information through layers. In doing so, MINT achieves better pruning performance than SOTA baselines, using a single prune-retrain step. When characterizing the behaviour of MINT-pruned networks, we observe that it behaves like a regularizer and improves the calibration of the network. However, a reduction in the number of filters used and redundancies makes pruned networks susceptible to adversarial attacks. Our future direction of work includes improving the robustness of compressed networks to adversarial attacks as well as detailing the sensitivity of layers in various architectures to network pruning.
ACKNOWLEDGMENT This work has been partially supported (Madan Ravi Ganesh and Jason J. Corso) by NSF IIS 1522904 and NIST 60NANB17D191 and (Salimeh Yasaei Sekeh) by NSF 1920908; the findings are those of the authors only and do not represent any position of these funding bodies. The authors would also like to thank Stephan Lemmer for his invaluable input on the calibration of deep networks.

A. Practical Implementation Hyper-parameter
In Section 3.2 of the main paper, we describe MINT in detail. In the description, we use δ as a threshold on ρ() values to help retain only those dependencies that contribute a majority of the information to the next layer. However, in practice, since the ρ() values are not normalized w.r.t. each other, there are cases when certain layers can be fully removed when the δ parameter is large. In order to protect the network from such behaviour, we use an additional threshold called γ to cap the maximum percentage of filters that can be removed from a selected layer. If the pruning ratio within a layer exceeds the value of γ, then γ is used to re-evaluate δ such that it falls within the desired pruning ratio. The new temporary value of δ is computed using the ρ() values within the pair of layers considered, and is only used prune filters in the selected pair of layers. Here, we observe a relatively steady increase in the maximum number of parameters pruned as the number of groups (G) increases, when no γ is provided or when a linear/quadratic dependence on G is used to compute γ. However, when γ is fixed at 0.8, we observe irregular behaviour, where the maximum percentage of parameters removed decreases with an increase in G. We posit that this behaviour arises from two factors, (a) the varied sensitivity of different layers is not compensated for by a fixed γ value, and (b) the difference in G values leads to different GMI estimates which account for more uncertainty in measurement.

B. Experimental Setup
In the following section we outline the hyper-parameters used during each stage of our experimental pipeline, training, pruning and retraining. The values are provided here in an effort to help users replicate our experiments. We plan to release our code soon. Training Setup: Table B-1 outlines the setups used to obtain trained models from which mutual information (MI) estimates are computed. A standard pretrained model from PyTorch zoo is used for the ResNet50-ImageNet experiments. Pruning Setup: For the conditional geometric mutual information (GMI) estimate we provide a set of parameters describing the number of groups per layer, G as well as the number of samples per class, m. We use the average activation across a filter's dimensions as the samples for GMI computations. MLP -MNIST: Between FC 1 and FC2, G is set to 250 and 100 while between FC 2 and FC 3 it is set to 300 and 10. 650 samples per class are used to compute the conditional GMI estimates. VGG16 -CIFAR10: All the layers between Convolution 4 and Linear 1 use G = 64 while m = 650. γ = 0.45 is used for Convolution layers 5, 6, 7, and 8.


An Optimized Hybrid Deep Learning Model to Detect COVID-19 Misleading Information

Fake news is challenging to detect due to mixing accurate and inaccurate information from reliable and unreliable sources. Social media is a data source that is not trustworthy all the time, especially in the COVID-19 outbreak. During the COVID-19 epidemic, fake news is widely spread. The best way to deal with this is early detection. Accordingly, in this work, we have proposed a hybrid deep learning model that uses convolutional neural network (CNN) and long short-term memory (LSTM) to detect COVID-19 fake news. The proposed model consists of some layers: an embedding layer, a convolutional layer, a pooling layer, an LSTM layer, a flatten layer, a dense layer, and an output layer. For experimental results, three COVID-19 fake news datasets are used to evaluate six machine learning models, two deep learning models, and our proposed model. The machine learning models are DT, KNN, LR, RF, SVM, and NB, while the deep learning models are CNN and LSTM. Also, four matrices are used to validate the results: accuracy, precision, recall, and F1-measure. The conducted experiments show that the proposed model outperforms the six machine learning models and the two deep learning models. Consequently, the proposed system is capable of detecting the fake news of COVID-19 significantly.

Introduction
A novel coronavirus  was discovered in Wuhan, China, at the beginning of December 2019.
e World Health Organization (WHO) has announced that the COVID-19 outbreak is a global pandemic on 11 March 2020 [1]. Due to the panic from COVID-19 disease, people started posting fake news and misinformation about the coronavirus on social media networks. e posts, tweets, and comments contain misleading statements. Recently, the researchers have had a particular interest in utilizing sentiment analysis to distinguish the fake news about COVID-19 [2]. Social media has the biggest contribution for spreading COVID-19 fake news due to the huge number of people's posts having panic expressions. erefore, the governmental authorities start to launch official websites for COVID-19 announcements to stop circulating fake stories about COVID-19 [3]. Consequently, the researchers began to pay attention to COVID-19 misleading information by analyzing social media contents and applying advanced AI technologies (i.e., machine learning and deep learning) to profiling the COVID-19 fake news [4]. As a result of the research direction in content analysis, the research organizations start raising funding to provide novel solutions to combat COVID-19 in terms of analyzing the misleading information about the COVID-19 pandemic [5][6][7][8][9][10]. Recently, machine learning and deep learning are playing a vital role in different areas such as sentiment analysis [11,12]; Alzheimer detection [13]; prediction cancer [14], and others [15,16]. e researchers have utilized the collected datasets related to COVID-19 through social media to evaluate their proposed approaches [17]. In this work, we have proposed an optimized hybrid model to detect the fake news on COVID-19 on social media. e core idea of the proposed model is the hybridization of using CNN and LSTM.
Our main contributions in this work are as follows: (i) Development of a hybrid model integrating CNN and LSTM to detect fake news about COVID-19 is done. (ii) e proposed model is optimized using a Hyperopt optimization technique to select the optimal values of parameters (iii) e proposed model, CNN, LSTM, and regular ML algorithms are applied to three COVID-19 fake news datasets (iv) e experimental results demonstrated that the proposed model had achieved the best performance compared with other models e rest of this paper is structured as follows. Section 2 presents the related work. Section 3 describes the architecture of the proposed system of COVID-19 fake news detection. Section 4 describes the experimental results. Finally, the paper is concluded in Section 5.

Related Works
Recently, researchers have been actively working to detect fake news about COVID-19. Wani et al. [18] used CNN, LSTM, and bidirectional encoder representations from transformers (BERT) to detect fake news about COVID- 19. ey used Contraint@AAAI 2021 Covid-19 Fake news detection dataset [19]. Elhadad et al. [20] proposed a voting ensemble classifier using 10 ML algorithms with seven feature extraction techniques to identify misleading information related to the COVID-19 outbreak. ey tested their proposed classifier to 3,047,255 tweets about COVID- 19. e best results are obtained from the NN, DT, and LR classifiers. Müller et al. [21] proposed transformer model COVID-Twitter-BERT (CT-BERT), on large a large corpus of Twitter messages about COVID-19. In [22], authors created an annotated dataset about COVID-19 fake news tweets. ey proposed a multilingual bidirectional encoder (mBERT) to extract the textual features from the dataset. e results show the mBERT has achieved the highest performance compared with SVM, RF, and a multilayer perceptron. In [17], two stages are developed to detect fake news. e first stage uses a novel fact-checking approach to retrieve the most relevant facts about COVID-19, while the second stage verifies the level of truth by computing the textual entailment. In addition, the authors used pretrained transformer-based language models to retrieve and classify fake news in a particular domain of COVID-19 using BERT and ALBERT. ey used a dataset that consists of more than 5000 COVID-19 false claims. In [23], authors gathered COVID-19 news articles from two data sources, and Poynter and Snopes then crawled the sources' textual contents. ey classified the articles into 11 various categories. Also, they applied ML algorithms on annotated the articles to detect misinformation about COVID-19. Al-Rakhami et al. [24] proposed an ensemble-learning-based framework to classify the tweets into credible or noncredible. ey applied the framework to a large dataset of tweets carrying news about COVID-19.
eir framework obtained high accuracy. Hossain et al. [25] introduced a benchmark dataset, COVIDLIES, which contains known COVID-19 misconceptions. ey classified each tweet in the dataset into three categories Agree, Disagree, or express No Stance. Patwa et al. [19] created and annotated the dataset that includes 10,700 posts and articles of real and fake news on COVID-19. Four ML baselines: DT, LR, Gradient Boost, and SVM, have been applied to an annotated dataset to classify posts as fake or real. SVM has obtained the best performance with the testing set.

The Proposed System of Detecting COVID-19 Fake News
In this section, the proposed system of detecting COVID-19 fake news is introduced. Figure 1 depicts the workflow of the proposed system showing in a set of steps which are (1) data collection, (2) data cleaning, (3) feature extraction, (4) hyperparameter optimization, and (5) evaluation models.

Data Collection.
ree datasets of COVID-19 fake news are used, which are described as follows.
3.1.1. Dataset 1. Dataset 1 was collected from Facebook posts, a far-right website which called Natural News (https://towar dsdatascience.com/explore-covid-19-infodemic-2d1ceaae2306). Also, another medicine website is used called orthomolecular.org. Although some data sources are removed from the Internet and social media, they are able to reach by the Internet's Archives.

Dataset 2.
Dataset 2 which is COVID-19 fake news data was collected from Internet (https://https://www. researchgate.net/publication/346036811_COVID-19_Fake_ News_Data). e dataset 2 consists of a set of COVID-19 fake news. Based on the dataset 2 attributes, the headings used later as labels have the binary attribute. In particular, it has 0, which indicates that the news is fake, while one indicates that the news is true.

Dataset 3.
Dataset 3 which is COVID-19 fake news data was collected from Internet (https://www.researchgate.net/ publication/349517903_COVID-19_Fake_News_Dataset). Dataset 3 was collected by Webhose.io and then was manually labeled. e dataset consists of three types of news which are (1) false news, which is called fake, (2) true news, and (3) partially false news. For simplification, false news and partially are considered false, labeled as 0, while the real news is labeled as 1.

Data
Cleaning. In this phase, we have applied five steps described as follows: (i) Text Parsing. In this step, the tokenization functions are used to divide the text within the datasets for further analysis. (ii) Data Cleaning. In this step, regular expressions methods are used to extract English alphabets, numbers, and their combination. is step is applied to eliminate any noisy data within the datasets texts. (iii) Part of Speech (PoS) Tagging. is step has marked each word in the text with its root, including a verb, adjective, and noun. (iv) Stop Words Removal. All the common words are removed from the text in this step, such as "a" and "the." (v) Stemming. In this step, we have applied a replacement method to replace each word with its root to eliminate the redundancy within the text. In addition, English stemmer is used, which reduces the text by 40-50% concerning the original text within the three datasets.

Feature Extraction Methods.
In this phase, two subphases have been done, which are applying Term Frequency-Inverse Document Frequency (TF-IDF) and word embedding for regular ML and DL models, respectively.
(i) For regular ML models, we have applied the TF-IDF method to assign weights to the prepossessed text of the three datasets [26]. e key idea of the TF-IDF is to determine the word frequencies within the text. (ii) For DL models, word embedding is a mechanism to represent words into vectors where the words with the same meaning have similar vectors. us, every word within the text is represented in dense vectors. Glove is one of the more popular word embedding techniques [27].

e Proposed Model Description.
In this section, the architecture of our proposed model is described. Figure 2 depicts the set of layers of the proposed model, which are sequentially working to detect fake news. In particular, the following layers are used: an embedding layer, a convolutional layer, a pooling layer, an LSTM layer, a flatten layer, a dense layer, and an output layer. In addition, we have applied Hyperopt optimization methods to optimize the proposed model further to select the best values for the proposed model's parameters. e layers are described as follows: (i) Embedding Layer. Each new has been represented into vectors mapped to each word. At the technical level, we have implemented this layer using Keras library [28]. e Keras library has three parameters Computational Intelligence and Neuroscience 3 which are input-dim, output-dim, and inputlength. e input-dim is used to configure the vocabulary size, while the output-dim is used to configure the size of the embedded words. e input-length is used to configure the length of input sequences. According to our implementation, the input-dim, output-dim, and input-length are 20000, 200, and 32, respectively. (ii) Dropout Layers. ese layers are used to prevent overfitting and reduce the complexity of the model [29]. As stated earlier, this layer receives its input from the embedding layer output. For configuration, we have set the value of the dropout as range between 0.1 and 0.9. (iii) Convolutional Layer. e convolutional layer receives input from the dropout layer. e convolutional layer has two main parts, which are filter and feature map as a kernel. In the first part, a filter is used to apply filtering on the input word matrix. e filtering process is useful for providing a map of features that indicates the pattern of the input data [30]. e ReLU activation function is used to identify the features within the news. (iv) e Pooling Layer. is layer uses the max operation for feature reduction within the feature mapping process. In particular, configuring a high value will significantly help capture the essential features, which reduces the computation for the next layer. (v) LSTM Layer. LSTM is a type of recurrent neural network which is used for prediction based on learning long-term dependencies. According to this work, we have used LSTM to build the hybrid model. (vi) e Flatten Layer. e text was converted to a 1dimensional array by the flatten layer, which was then input to the following layer. (vii) Dense Layer. It is a deeply connected neural network layer. It has some parameters which are input, kernel, bias, and activation. e input parameter represents the input data. e kernel represents the weight data. And the activation is used to represent the activation function.
(viii) e Output Layer. is layer is used to take the output of the flattened layers to generate the model's final output, real or fake news. We have used ADAM optimizer [31] and sigmoid activation function [32].

Different
Models. Different models have been compared with the proposed model: six regular ML models, CNN, and LSTM.
(i) Six regular ML models such as DT [33], LR [34], KNN [35], RF [36], SVM [13], and NB [37] were used to compare with proposed model. (ii) e long short-term memory (LSTM) model has five layers which are (1) an embedding layer, (2) hidden layer, (3) dropout layer, (4) flatten layer, and (5) an output layer. e embedding layer is the first layer that has been designed similarly to the proposed model layers. e hidden layer is used, which is LSTM [38,39]. In particular, the L2 weight regularization technique is used with reg_rate value for l2. e third layer is the dropout layer which is used to eliminate the overfitting and simplify the model [29]. It is configured by setting the dropout value as a range (0.10.9). e fourth layer is the flattened layer which aims to convert the entire text into a vector of features. Finally, the output layer takes the flatten layer output to generate the final output, which classifies text as the whole in terms of real or fake news. (iii) Convolutional neural network (CNN) consists of an embedding layer, a convolutional layer, a pooling layer, a flatten layer, a dense layer, and an output layer.

Hyperparameter Optimization.
Hyperparameter optimization aims to optimize the hyperparameters for ML and DL models automatically. Hyperparameter tuning is utilized to pass various parameters into the model to select the best values of parameters to achieve the best performance.  (i) For optimization ML models, we have used grid search. We have defined a set of initial values for each hyperparameter. e model checks these values and then selects the best value for each hyperparameter for obtaining the highest accuracy. Also, K-fold cross-validation (CV) is used where the dataset is equally divided into K-fold. K-1 folds are used for training, and the rest part is used for testing. e dividing process is repeated until the model reaches that each fold has been used for testing. Finally, the classifier is evaluated based on the average of accuracy within the 10-fold.
(ii) We have used Hyperopt, which is distributed asynchronous hyperparameter optimization built-in python library, for optimization of DL algorithms. Furthermore, Hyperopt is an open-source library for large-scale AutoML and HyperOpt-Sklearn (https:// github.com/hyperopt/hyperopt). A set of parameters for the proposed model is configured as shown in Table 1. Also for the LSTM model, a set of parameters is as shown in Table 2. Similar to LSTM, a set of parameters of the CNN model is configured as shown in Table 3.

Evaluation Metrics.
Four standard machine learning metrics are used, which are accuracy, precision, recall, and F1-score. Equations (1)-(4) describe the formulas for calculating these metrics. Accordingly, TP stands for true positive, TN for true negative, FP for false positive, and FN for false negative.
(i) Accuracy is the popular metric used to perform ML and DL models. It measures the percentage of correctly predicted observations. e accuracy calculation formula is as follows: (ii) e second metric is a precision which indicates the ratio of true positives to all true events predicted. e precision calculation formula is as follows: (iii) Recall is the third metric used to indicate the total number of positive classifications out of true class. e recall calculation formula is as follows: (iv) e fourth metric is F1-score which shows the trade-off between precision and recall. It shows the weighted average of precision and recall. e F1score calculation formula is as follows:

Experiment Setup.
e experiments were conducted using a Google Colab RAM 25 GB, Python 3, and GPU. e three comparable models, the proposed model, LSTM, and CNN models, are implemented using the Keras library. e regular ML models have been implemented using the sci-kitlearn package. For optimization, grid search is used for ML, and the Hyperopt library is used for DL. For the embedding layer, a 200-dimensional word vector is used for the Glove set pretrained. For the datasets, three datasets of COVID-19 fake news are used. Each dataset is divided into 80% for training and 20% for testing. Each experiment has been repeated ten times. e result of cross-validation (CV) and testing performance has been registered.     Table 6. e optimal values for CNN's parameters are shown in Table 7.

Results
e optimal settings of the LSTM parameters are shown in Table 8.            Table 16. e optimal values for CNN's parameters are shown in Table 17. e optimal settings of the LSTM parameters are shown in Table 18.

Discussion.
e section will be present the best models for each dataset. Figure 3 illustrates a comparison between the best models for CV and test performance for dataset 1 Overall, the proposed model has achieved the best performance for CV and testing. Figure 4 illustrates a comparison between the best models for CV and test performance for dataset 2. For CV performance, the proposed model has the best performance ( Overall, the proposed model has achieved the best performance for CV and testing. Figure 5 illustrates a comparison between the best models for CV and test performance for dataset 3. For CV performance, the proposed   Overall, the proposed model has achieved the best performance for CV and testing.

Conclusion
In this paper, the hybrid model based on CNN and LSTM has been proposed to detect COVID-19 fake news. Some layers for the proposed model were developed (i.e., an embedding layer, a convolutional layer, a pooling layer, an LSTM layer, a flatten layer, a dense layer, and an output layer. ree datasets about COVID-19 fake news were used to evaluate the proposed model. e experimental results have proved the superiority of our proposed model to detect the fake news of COVID-19 among six machine learning models (i.e., DT, KNN, LR, RF SVM, and NB) and two deep learning models (i.e., CNN and LSTM).

Conflicts of Interest
e authors declare that they have no conflicts of interest.


Virtual Sample Generation and Ensemble Learning Based Image Source Identification With Small Training Samples

Nowadays, source camera identification, which aims to identify the source camera of images, is quite important in the field of forensics. There is a problem that cannot be ignored that the existing methods are unreliable and even out of work in the case of the small training sample. To solve this problem, a virtual sample generation-based method is proposed in this paper, combined with the ensemble learning. In this paper, after constructing sub-sets of LBP features, the authors generate a virtual sample-based on the mega-trend-diffusion (MTD) method, which calculates the diffusion range of samples according to the trend diffusion theory, and then randomly generates virtual sample according to uniform distribution within this range. In the aspect of the classifier, an ensemble learning scheme is proposed to train multiple SVM-based classifiers to improve the accuracy of image source identification. The experimental results demonstrate that the proposed method achieves higher average accuracy than the state-of-the-art, which uses a small number of samples as the training sample set.

INTRoDUCTIoN
Nowadays, the digital images generation is popular and easier, which makes it possible for some individuals to upload unsuitable images for their interests or to steal images of others for commercial purposes. Therefore, image source identification is very important in the judicial field, which can offer help to bring evil men to justice. The issue of image source identification is usually modeled as a classification problem, which means decent results are expectant with enough training samples. However, it is well known that obtaining a large number of sufficient training samples may be very difficult, and the classifiers perform very poorly in this scenario of small training samples. Therefore, it is always a big challenge when there are only a small set of labeled images used as references in the practical forensic application.
In recent years, many methods are proposed for the small training sample problem, which are mainly divided into three categories. The first category is active learning and semi-supervised learning based methods, but they usually require a large number of unlabeled samples as auxiliary information, and it is sometimes unrealistic in practical forensic applications; the second category is the methods based on gray prediction model, such as BGM (Chang, Li, Huang, & Chen, 2015), GBM (Wang, Wang, Sun, & Zhang, 2014), ANGM (Chang, Li, & Chen, 2014), which is used to deal with raw samples. However, these methods usually ignore the internal mechanism, and then make the generated virtual samples unsuitable; the third category is consist of the methods based on virtual samples generation, which is proposed by Poggio andVetter in 1992 (Poggio &Vetter, 1992). Considering the insufficient training samples, the appropriate virtual samples are generated under the condition of the training samples' prior information to increase the number of training samples. By obtaining the virtual samples, the training set is supposed to be expanded to effectively improve the generalization ability of the classifier.
In recent years, there are many kinds of researches respect to virtual samples generation. In order to improve the energy prediction accuracy of small training samples problem, He et al. (He, Wang, Zhang, Zhu, & Xu, 2018) propose nonlinear interpolation virtual samples generation method based on the highly nonlinear characteristics of input data and output data. After the virtual samples generation, the images are classified by the extreme learning machine (ELM) (Huang, Zhu, &Siew, 2004) and the experimental results are promising. Li et al. (Li & Fang, 2009) propose a nonlinear virtual sample generation technique (NVSG) and receive an average classification accuracy of 76% for camera models in the Iris data set. The methods of virtual sample generation based on the original samples' distribution are also widely used. Yang et al. (Yang, Yu, Xie, & Zhang, 2011) assume that the samples obey the Gaussian distribution and calculates the mean and variance of the Gaussian distribution from the original training set. Experiments on the Iris data set show that the classification accuracy increases 18%.
In this paper, a MTD based virtual sample generation method is introduced to identify the image source when the training samples are small. By box plot based MTD and sample attributes correlation based method, a reasonable virtual samples generation range is obtained and the virtual samples are generated based on average distribution. Considering the randomness of virtual sample generation, multiple groups of samples are obtained and combined with the original training samples. Multiple weak classifiers based on SVM are trained and integrated to obtain the classifier.
The rest of this paper is organized as follows: Section 2 describes the related work: LBP features and virtual sample generation method; the virtual sample generation and ensemble learning based method are proposed in Section 3; Section 4 demonstrates the experimental design and the discussion of the results and finally, the paper is concluded in Section 5.

LBP
LBP is a local operator describing the texture features of an image. The uniform gray-scale invariant LBP (Tan, Wang, Li, Guo, Kong, & Shi, 2015) operator is donated by LBP P R , and defined as follows: where P represents the number of pixels in the neighborhood and R represents its radius. g c is the center pixel value, and the values of the neighborhood pixels in a circle with a radius R is g p . We set P = 8 , R = 1 . The threshold function s x ( ) is defined as: Refer to (1), (2), the difference between the gray-level value of the center pixel and its neighborhood pixels can be calculated, and there are 256 different patterns of combination. As what is defined in (Tan, Wang, Li, Guo, Kong, & Shi, 2015), ''uniform'' local binary patterns are more likely to appear than other patterns which are called ''non-uniform'' local binary pattern, and the authors integrate all ''non-uniform'' local binary patterns into one pattern. Thus the dimension of features is reduced from 256 to 59.
The whole procedure of LBP feature extraction is shown in Figure 1. The original image is estimated by the prediction function to obtain the predicted image, and the prediction-error image is obtained by subtracting a predicted image from the original one. We finally extract LBP features from the red and green channels of the original image, the 1st-level diagonal wavelet sub-band, and prediction-error 2D array mentioned in (gardenfors, 2004). Thus a total dimension of 59 3 2 354 × × = improved LBP features are obtained.

MTD
The MTD method proposed by Li et al. (Li, Wu, Tsai, & Lina, 2007) is based on the technique of global fuzzification and information diffusion, which is used to fill the blank caused by incomplete samples. It can augment the training set by generating virtual samples based on the distribution of the existing small training samples. Regarding the training samples as a whole, the upper and lower limits of the specific boundary for virtual samples generation are determined as: where L and U are the lower and upper bounds of the virtual samples diffusion, represents the variance of the real samples, N L and N U represent the actual number of real samples whose value are less than and greater than the center value, respectively.
x i and x represent the real samples and their mean, ˆ/ s N x L 2 represents the real samples' diffusion coefficient, n represents the number of the real samples, u min max set = + ( ) / 2 represents the real samples' center value, min and max are the real samples' minimum and maximum value, respectively.
In order to estimate the trend of the real samples while extending the real samples' range, they asymmetrically spread the real samples and consider the skewness of the real samples, the equations are: Where Skew L and Skew U are the left and right skewness of the real samples point relative to the center of the real samples. Set ϕ ϕ ( ) ( ) a b = = − 10 20 in Equation (1) and (2), the upper and lower boundaries after diffusing the final sample set are: then the virtual samples are generated on the uniform distribution within this range.

Box Plot Based MTD
Box plot, also known as boxer map, is widely used to describe the sporadic degree of the data by five statistics: minimum, first quartile, median, third quartile and maximum. It can be unaffected by the outliers when measuring the characteristics of the given data, which is the biggest advantage of the box plot. The core of the box plot based MTD virtual samples generation method (Li, Chen, & Chang, 2012) is to reasonably spread the real samples' range. It is quite essential to accurately estimate the boundary and the skewness of the data set and box plot is proficient at that. The box plot is established as shown in Figure 2, and the data situation can be judged according to its position in the box plot.
After establishing the box plot, we can get the upper and lower limits of the extended data based on the real samples. Different from the MTD method, the diffused upper and lower limits of the training samples are obtained according to the real samples' box plot (Lin, Li, & Pan, 2016), as shown in the following Equation (7) and (8). Therefore, the final determined virtual samples' range is [ L ,U ].
Considering that the randomness of generating virtual samples based on the uniform distribution within the range [ L ,U ], a triangular membership function is utilized to estimate can the occurrence possibility of the generated virtual samples. According to the fuzzy theory, they draw an asymmetric triangle membership function ( MF ) (Li, Lin, Chen, Chen, & Lin, 2018), as shown in Figure 3.

Figure 2. A right-skewed distribution drawn in a box-and-whisker plot
In Figure 4, MF represents the distribution of the training samples, and tv is the virtual samples' value which is randomly generated in the uniform distribution within [ L ,U ]. Based on the generated tv value, the corresponding MF value is calculated according to the following Equation (9), and the MF value is used as the occurrence probability to evaluate whether the randomly generated virtual sample tv is the most suitable. The class label for the generated virtual sample is the same as the original sample. Through the above steps, we can generate sufficient virtual samples to expand our training set for images source identification when the number of training samples are small.

Sample Attributes Correlation Based MTD
Compared with the box plot based MTD method in Section 2.2.3, sample attributes correlation based MTD method considers the correlation between sample attributes when determining the diffusion range of the real samples. Li et al. consider the correlation of sample attributes when calculating the range of generated virtual samples (Gole & Bhme, 2010). Taking the correlation of sample attributes into account, the real samples' data center is calculated by Equation (10), which is used to estimate the samples' distribution.
As shown in the Equation (10), x nth represents the real sample. Considering the mean is susceptible to outliers, Li et al. decide to use the real samples' median instead of the mean to calculate the correlation of sample attributes, because the median can better reflect the deviation of the data center than the mean. The correlation is replaced with the TSA (trend similarity between attributes): where g h i j ( ) , represents degree of similarity between the two attributes' dimensions of real samples in the same class. If g h i j ( ) , = 1 , the similarity trend between the two attributes' dimensions of m and n is extremely high, that is, both are on the same side of the center point CL. S i j , represents the similarity between different attributes' dimensions of the real samples, and k represents the number of samples in the same class. In order to avoid using mean which is susceptible to outliers, we use Euclidean distance instead of sample standard deviation to measure the dispersion as shown in (12). Incorporating the correlation of sample attributes, the ultimate boundary for virtual sample generation after diffusion is calculated as:

PRoPoSED METHoD
According to the above analysis, virtual samples generation is a random procedure and the virtual samples generated by MTDBOX and MTDRELATION are not stable enough to obtain a promising identification classifier, which will drag down the identification accuracy. But on the other hand, instability also means diversity and that is quite suitable for ensemble learning. So in this paper, we incorporate virtual samples generation with ensemble learning to identify the image source when the training samples are small, the framework is shown in Figure 5. In our proposed method, we follow the basic idea of the Bagging in ensemble learning rather than use a specific ensemble learning method. Different from the previous ensemble learning methods, we don't directly operate on the training samples or the features of training samples, instead we generate virtual samples by MTDBOX and MTDRELATION and add these virtual samples to the image set of real samples in order to increase the number of training samples. And then train the base classifier by SVM on the expanded training set.
Ensemble learning is a form of "expanding others" by constructing multiple weak classifiers and combining them into a powerful classifier to effectively accomplish their tasks, it comprehensively determines the learning results by combine multiple learners obtained through training. A weak classifier has a slightly better classification effect than a random guess. Throughout the experiment, the weak classifiers obtained through the initial training are assembled, because the classification results of a single base classifier usually can't reach the ideal standard. By learning and complementing each other, the classifiers can ultimately improve the accuracy of the classification and achieve the desired experimental results.
There are many strategies for integrating classifiers, such as averaging, voting, and learning. For the prediction of classification problems, the voting method is popular, so we use it in this paper. The voting method is also "Majority rule", which means if there are n classifiers, the class obtained by a large number of classifiers is the final class of the image.
In this paper, we firstly extract small training samples' LBP features, and then use the MTD based method to generate virtual samples. The generated virtual samples share a class label with the corresponding original samples. Because the virtual samples generated by the MTD based method are random and the number of generated virtual samples is variable, the MTD based method can be used to train multiple base classifiers. Finally, the results of multiple base classifiers are operated by ensemble learning -the relative majority voting method, and the final sample's label is obtained.

EXPERIMENTS Experiments Set Up
In order to evaluate the proposed algorithm in this paper, the well-known "Dresden Image Database"(Gole & Bhme, 2010) is used. The 16 different camera models from 11 camera brands in the experiments are listed in Table 1. The training samples range from 5 to 20 and there are 180 test samples for each camera.
In our experiments, the box plot based and sample attributes correlation based MTD methods are used to generate virtual samples respectively. Combining the two virtual sample generation methods with SVM classifier directly, we get two baseline algorithms named as MTDBOX and MTDRELATION for comparison. The proposed method using ensemble learning is called MTDEM in the following. In our experiment, we randomly extract 5, 10, 15, and 20 samples from each camera (a total of 16 cameras) as training samples. For each set of samples, 10 virtual samples are supposed to be generated. In order to reduce the randomness of sampling and generating virtual samples, we repeat the experiment 10 times for each class of samples and take the average accuracy as the results of the image source identification.

RESULTS AND DISCUSSIoN
The experimental results are all shown in Table 2.
Without the virtual samples, the accuracy of 39.64%, 63.19%, 70.94%, and 76.32% is achieved based on the image source identification model trained with the raw database of 5, 10, 15, and 20 training samples, respectively. Combing the 10 virtual samples generated by box plot based MTD, the accuracy of image source identification comes to 47.51%, 64.25%, 71.86%, and 77.03%. And also, the virtual samples from sample attributes relation based MTD help the SVM classifier to improve 2.32%, 0.76%, 0.87% and 1.25% in the case of 5, 10, 15 and 20 raw training samples. From the results without virtual sample, MTDBOX and MTDRELATION based virtual sample, an important observation is that the high quality virtual samples do help the classification model improve the performance, while the improvement seems limited. In another word, the virtual samples provide benefits for constructing weak classifiers.
For the proposed method, we obtain a decent accuracy of 53.93% in the case of 5 raw training samples, and high accuracy of 83.28% when the number of training images increases to 20. The highest identification accuracy identify the effectiveness of the ensemble learning used in the proposed method.

CoNCLUSIoN
In this paper, virtual sample generation based algorithms combined with ensemble learning is proposed to identify the image source in the scenario of limited training samples. Two different virtual sample generation methods, MTDBOX and MTDRELATION are used to obtain virtual samples as the training sample. Considering the weak classifiers based on the two virtual sample generation methods, the MTDEM algorithm, which introduces the ensemble learning to improve the performance of the classifier, is proposed in this paper. The experimental results demonstrate that the effectiveness of the proposed method and show that the performance of MTDEM is superior to existing methods when identifying the image source with small training samples.

ACKNoWLEDGMENT
This work was supported by the National Natural Science Foundation of China (No. U1936117, No. U1736119, and No. 61772111).


Improved Image Denoising Methodology using Deep CNN Bilateral Filter Compared to Additional Methods

Image Denoising techniques are widely used to remove the noise from the images. Due to the ease of the bilateral filter, it is used very often to remove the noise from the images. In this paper, a novel approach has been proposed to enhanced bilateral filter in conjunction with CNN as a booster to eliminate Gaussian noise from Grey images. Studies reveal that standard CNN using a bilateral filter is the best technique to eliminate Gaussian noise from images along with high PSNR values. This paper also performs a comparative study of the various existing techniques for image denoising with the CNN technique and the applied Bilateral filter Method as a de facto to improve the results in terms of enhanced PSNR values. ECNDNet (Enhanced CNN) applied to noisy images with standard deviation σ = 15 gives PSNR values up to 32.81 In comparison to this when both bilateral filter and deep CNN applied, in conjunction produces improved PSNR values up to 34.73 along with the equivalent standard deviation. The results in this work reveal better performance in terms of PSNR as compared to other methods. The test result proves that the bilateral filter Method along with CNN can improve the quality of restored images significantly better.

INTRODUCTION
Image Denoising is a traditional technique; it has been required in many fields. Images acquired through the sensor of cameras are always corrupted by some noise. The existence of random discrepancy of intensity affects color information images usually can be the feature of electronic noise. The denoising model is the process to eliminate unwanted information from an image, different images are denoised by using different technologies. This research work explains and explores different methodologies for noise reduction. Bilateral filtering technique smooth images while edgepreserving. Its filtering work in 1995 by Aurich and Weule [1] on linear Gaussian filtering.

Advantages of Bilateral Filter:
• bilateral filtering edge-preserving, while removing noise and causes smoothing of images.
• It reduces color blurriness while working with color images.
• It is a very simple and non-iterative scheme to implement.
The rest of the paper organize in such a manner that, section 2 will briefly introduce related work, section 3 discuss the image denoising techniques, Section 4 will discuss the proposed model, section 5 introduce experiment and data, and also will explain regarding data set Section 6 will explain the result and section 7 will conclude the future scope and conclusion.

II. RELATED WORK
In 2017, Kai Zhang et al. [11] presented by exploring the creation of feed-forward denoising with CNN (convolutional neural network). Specifically, this technique is applied to residual learning and batch normalization. It helps speed up the training process. This approach influences training a single DnCNN model. Such as Gaussian denoising JPEG image DE blocking.
In 2017, N. Senthilkumaran, J. Saromary et al [14] proposed an Adaptive Bilateral filtering method which consisted Switched bilateral filter and a bilateral filter to eliminate Gaussian Noise from Gray images and found that bilateral filtering is the best procedure to remove Gaussian noise from images with high PSNR values. The proposed technique gives the best results than all other techniques in terms of assessment parameters such as PSNR, MSE, and NCD.
In 2018 [15] Yapeng Tian et.al presented CNN has achieved great success for image restoration also hierarchical features. This approach to generating visually pleasing high-quality measurement. This paper proposed local and global features and obtaining very powerful representation ability.
In 2018, [16] Chunwi Tian, et al. proposed the CNN technique for image denoising by enhancing the quality of various images experimentally and analyzing the PSNR values on different standards deviation. Work demonstrated was looked to the effects of dilated convolution. Its uses for ECND Net to enlarge the receptive fields and improve the performance.
Image restoration [17] techniques developed in the past years and discuss future developments. Learn a CNN to predict the residual between the initialized high-resolution images and the ground truth image. Deep CNN can be very efficient in the testing stage. Image restoration methodologies do not scale well to larger images, while efficient alternative bigot.

III. IMAGE DENOISING TECHNIQUE
Bilateral filter denoising [2] improved the images to remove Gaussian noise from the grayscale. It is found the standard Bilateral filter is the best method to eliminate Gaussian noise from images with high PSNR value. Bilateral filter methods are more effective than popular denoising methods. One of the most popular advantages to preserving edges while removing noise causes smoothing of images and reduces color blurriness. It is a simple and iterative scheme to contrivance. Which is a Stimulating research area that has conventionally use statistical image features. Bilateral has shown recent promise for this task and further improvements can be realized via the use of the PSNR ratio, which allows the use of a full reference quality applied to extracted pixel and original images.
Image restoration [17] essential difficulty in image processing and low-level visualization. Objective reconstructed to latent high-quality image from its ruined reflection. The acquired images are a ruined reflection of the unknown latent images, while the ruin comes from various factors such as noise corruption object motion, resolution limit hazing rain, a combination of them. Introduce the major IR techniques developed in the past years and discuss future developments. Learn a CNN to predict the residual between the initialized high resolution images and the ground truth image. Deep CNN can be very efficient in the testing stage. Image restoration approaches do not scale well to larger images, while efficient alternative. "Goal of this paper is to develop an effective approach to image restored offers both computational efficiency and high restoration quality".

Figure 3.1: Image denoising method
Enhanced CNN [19] used for removing noise in pixel quality better PSNR. This paper used the dilated convolutional networks to increase the perspective information and reduce the computational cost. Enhanced convolutional neural denoising network exceeds the state-of-the-art methods for image denoising. Convolutional neural networks are used to increase network performance. It is used to obtain an uncontaminated image from the residual image and noisy image.

Comparison of Denoising Methods
This section briefly describes the denoising methods bilateral filtering is reasonable with popular methods suchasBM3D, WNNM, EPLL, CSF, TNRD, FFD, and ECN. WNNM [7] weight nuclear norm minimization problems are evaluated under different conditions. However, this impressively confines its proficiency and flexibility in allocating with many concrete denoising problems. This paper analyzed the weighted nuclear norm minimization problem, where the curious values are allocated different weights. WNNM algorithm outperforms many states of the art denoising algorithm.
EPL [8] expected patch log-likelihood method has been introduced, enforced the concluding reconstructed image patch. In the perspective of the Gaussian mixtures model, this idea has been lead to a state of the art in image denoising and de-blurring. This paper represents a method that essentially concludes these thresholds to improve the image evaluation. It"s accomplished a similar performance to that of EPLL with GMM in denoising. TNRD [10] trainable nonlinear reaction-diffusion describes a knowledge agenda built on the perception of nonlinear reaction models for various image restoration problems. This methodology is appropriate for a variety of image restoration tasks by including suitable reaction force. We determine its proficiencies with the representative application, Gaussian image denoising single image super restoration, and JPEG de-blocking. BM3D was developed by Ahn B, et al [13] is the foremost prevalent denoising strategy. BM3D is a two arrange non-locally collaboration sifting strategy within the change space. It utilizes 3d shapes, which are stacked into a 4Dchart. ffD Net [16] is to the fast implication and respectable performance, however, this method mostly learn a specific model for each noise level, require multiple models for denoising images with the different noise level. Fast and flexible denoising network works on accomplishing a good trade-off between implication speed and denoising performance. FFD Net is effective and efficient, making it a highly attractive practical denoising application. CSF [18] shrinkage operation bears a connection to wavelet methodologies, but is used here in a random field context computational proficiency is accomplished by creation concluded the use of convolutional high renewal quality is attained through loss-based training of all models parameters and the use of cascade architecture. We study approaches that can be revised easily to a different trade-off between efficiency and image quality.

IV. PROPOSED DENOISING METHOD
According to the previous analysis, we all know that the denoising methodology will be expressed as Y = x+µ. F(y), because the objective performance of learning is as follows:- (1) is an objective function to train the denoising model, where represent the ℎ noisy image patch and represent the ℎ level image patch and learn more features. Learn a lot of possibility as a result of these considerations, we tend to propose a unique network know as Bilateral with CNN furthermore residual learning builds this network more practical for image denoising the deserves of the projected strategies have fielded it in 17 th layer network and residual learning to stop the issues of vanishing a further CNN performance convergence and build the network easier to trained knowledge. Moreover, the use of BN and residual learning make this network more effective for image denoising. In addition to bilateral filter with CNN enhanced performance BN techniques are used to convergence and make the network easier to trained data.  3 At last, the images which denoised Bilateral filter using CNN technique. This approach is very convenient for robustness and helps in improving the PSNR to survive against different types of techniques, also by using CNN. The extraction process involves increasing the pixel quality of denoised images.

Application of Bilateral Filtering:
• Bilateral filter, wherever it found a broad application that medical imaging pursuit film restoration and a lot of we tend to discuss helpful extension referred to as the bilateral filter. • A bilateral filter could be a feature formula the discrepancy will be used to extend the visibility of alternative pictures searching filters that train to scale back noise a large style of ultimate. Edge sharpening filters company by enhancing high frequency. • Bilateral filtering application to decompose many pictures into parts so they single output image that inherits selected visual properties from every picture. • We will additionally utilize the assorted technique like texture and illumination separation tendency creating and residency management for bilateral filtering.

V. DATASET & IMPLEMENTATION:
This paper technique is enforced Matlab-19 and varied performance metrics are unit taken into thought peak signal noise magnitude relation a good picture quality of denoising images. This study aims to evaluate the working, performance, and efficiency of various denoising techniques. This study helps in knowing different types of techniques used to remove noise in original images. Data set used for experimental purpose as follows:

Figure 5.1:
For other experimental, this paper used various techniques such as a bilateral filter using CNN. The experimental results of this technique are well tried to point out that its capable of image denoising single image superresolution and JPEG in image artifacts removal. Above all the images throughout Gray pictures, sp compared to the bilateral filtering methodology and bilateral with CNN preexisting denoising methodology result. lts promising within the sense that was ready to recover images details in structures within the actual ensuring image was a lot of visual images.
We tend to noise magnitude relation PSNR of each image. The BN and residual learning are effective for image denoising. We all know that the bilateral filter with CNN is useful for image denoising.
Bilateral for Grey images denoising: We choose 12 pictures with the dimensions of 180*180 for mathematicians denoising the format of pictures is png. In line with BM3D, WNNM, EPPL, CSF, TNDR, IRCNN, ECNDNet, and bilateral with CNN to verify the performance of gray pictures. Denoising to check the strength of our project methodology for low-level and high-level noise. That use we choose σ = 15, σ = 25 and σ = 50 to conduct comparative experiments.

Run Time:
To appraise the concert metrics of image denoising methods PSNR is used as a representative quantitative measurement. Give a ground truth image x , the PSNR of denoised images x᷃ is defined by PSNR(x, x᷃ ) = 10. 10 ( PSNR and runtime of processing every image are two important factors of image denoising. The performance of the proposed methods has been verified in the gray image section. The PSNR is a different Dimension an image is tested for Grey images denoising as follows (table 2). We utilize noisy image sizes of 256*256 and 512*512 with different standard deviations such as σ =15, σ = 25, and σ = 50 to test the PSNR of different images for the Bilateral method. Specifically, we use Matlab-2019 to test the run process of Bilateral with CNN Table (3) we know that our Bilateral using CNN is modest with popular methods such as BM3D, WNNM, EPLL, CSF, TNRD, IRCNN, ECNDNet, and Bilateral with CNN.

VI. RESULT AND DISCUSSION
We have taken various images for experimental purposes and we have seen that on different standards deviation such as 15, 25, and 50 obtained results have been mention below as under table 2.

Performance Evaluation
According to the preceding research, we know that the denoising method can be articulated. It is being used to enhance the performance of the noise level and the increased PSNR. We consume noisy image sizes of 256*256, 512*512, 1024*1024 with σ= 15, σ =25, and σ=50 to test the quickness of different methods for an image. Specifically, we use MATLAB to test CNN and bilateral filtering.
Our proposed method is a better PSNR for image denoising. We know that our bilateral filtering is competitive with popular methods such as BM3D, WNNM, EPLL, CSF, TNDR, and CNN. We will demonstrate the efficiency of these methods for image denoising. The performance of the recommended method has been proved the processing an image is tested for gray image denoising. I have accompanied some testing to see the performance of the proposed framework. I virtual noisy images by adding White Gaussian noise with various standard deviations to some standard test images. These noisy images were then denoised using several PSNR results were considered real images for each test image, three noisy versions were created by adding white Gaussian noise with standard deviations 15, 25, and 50.

VII. FUTURE SCOPE AND CONCLUSION FUTURE SCOPE
Future expansion of this approach is to be with proposed technique the proposed filtering method work very properly for noise care of different filtering approach. We will combine model-based Optimization and Discriminative gaining knowledge of strategies to get rid of the noise from the real noise images. For future work, first, explore how to covenant with different sorts of noise mainly these current in actual life. Secondly, the technique besides the use of image fairs instills an open problem. Besides the methodology of pictures denoising can also be improved in two other applications.

VIII. CONCLUSION
This paper compares the most widely used image denoising different techniques and it concludes that based on the data set we have to find the best performance and increasing PSNR value for the bilateral filter using the CNN technique, for image denoising. We have to find a better PSNR value for the image denoising technique.