extfieldsofstudy:
Medicine
Computer Science

text:
Detection of dynamic protein complexes through Markov Clustering based on Elephant Herd Optimization Approach

The accessibility of a huge amount of protein-protein interaction (PPI) data has allowed to do research on biological networks that reveal the structure of a protein complex, pathways and its cellular organization. A key demand in computational biology is to recognize the modular structure of such biological networks. The detection of protein complexes from the PPI network, is one of the most challenging and significant problems in the post-genomic era. In Bioinformatics, the frequently employed approach for clustering the networks is Markov Clustering (MCL). Many of the researches for protein complex detection were done on the static PPI network, which suffers from a few drawbacks. To resolve this problem, this paper proposes an approach to detect the dynamic protein complexes through Markov Clustering based on Elephant Herd Optimization Approach (DMCL-EHO). Initially, the proposed method divides the PPI network into a set of dynamic subnetworks under various time points by combining the gene expression data and secondly, it employs the clustering analysis on every subnetwork using the MCL along with Elephant Herd Optimization approach. The experimental analysis was employed on different PPI network datasets and the proposed method surpasses various existing approaches in terms of accuracy measures. This paper identifies the common protein complexes that are expressively enriched in gold-standard datasets and also the pathway annotations of the detected protein complexes using the KEGG database.

As PPI network continuously transforms with respect to the environment, time and various phases of the cell cycle, the clustering analysis on static PPI does not emulate these dynamic attributes and it is far from optimal solution. Thus, in recent times, various attempts on the clustering process of dynamic PPI network has been initiated along with the gene expression data to enhance the protein complex detection. Also, many evolutionary approaches were employed for analysing the clustering process of the PPI network such as ant colony optimization ACC-DPC 1 , ACO-MCL 10 , cuckoo search optimization (CSO) 11 , BiCAMWI using genetic algorithm 12 , Soft Regularized-MCL 13 , particle swarm optimization (PSO-MCL) 14 and artificial fish school algorithm (AFA-MCL) 15 . The firefly optimization was employed along with Markov Clustering (F-MCL) on the dynamic PPI network for predicting complexes. The execution time for F-MCL is higher as all the fireflies (proteins) in the population (network) tries to reach the optimal solution (cluster). There are few proteins that are not eligible to come under the cluster and take more iterations to reach the cluster, which may take a long time 16 .
The above-mentioned approaches were effective, but they do not promise a global outcome since they suffer from the effect of unwanted clusters which leads to time consuming. In order to discard the drawbacks of the above-mentioned approaches, a novel approach was proposed to detect the dynamic protein complexes through Markov Clustering based on Elephant Herd Optimization Approach. One of the most important advantages for EHO is that it is the most computationally efficient and has less time consuming compared to F-MCL and other approaches. This is because the unwanted noisy data (unclustered proteins) will be removed from the clan separating operation of EHO approach. The remaining sections of this paper is ordered as follows: Section 2 discusses briefly about the methodology of the proposed approach. Section 3 illustrates the experimental results with various performance measures, Section 4 deliberates about the implementation and discussion of the proposed method in detail and finally Section 5 concludes the paper and recommends for the future enrichments.

Methods
For detecting the protein complexes, initially, the proposed method divides a static PPI network into a sequence of subnetworks below diverse time points by combining gene expression data to form dynamic model. In order to build a dynamic model, the static PPI network is integrated with gene expression data, which declare the level of gene expression, as well as protein expression. As a protein does not always becomes active at a cell cycle, it is assumed that a protein was active at the time points with its highest expression level 17 . The expression level of a protein will be increased before its expression and will be decreased once the protein has completed its function, and the time points are identified with its expression level, which are higher than a threshold.
Given is a static PPI network P P = (P ver , P Edg ), where P ver , is a set of proteins and P Edg , is a set of interactions between these proteins. In gene expression data, there is a series of T time stamps coming with |P ver | × (T * TR) matrix M, where TR is the number of repetitions of the time series. Each element M(P ver , j) of this matrix represents the level of gene expression.
The three-sigma principle is employed to determine if a gene is expressed in a single stamp. For each gene P ver , the gene expression is defined as given in the following Eqs (1)(2)(3)(4)(5) = ∑ + × − = Ev(P ) M(P , i T (tr 1)) TR (1) i ver tr 1 TR ver where Ev i (P ver ) is the mean of the expression value of gene P ver at timestamp i, UE(P ver ) is the mean of its expression values over times ranging from 1 to T, σ(P ver ) is the standard deviation of its expression values, Fl(P ver ) is used to show fluctuation of the expression curve of gene P ver . Suppose that the gene expression data is governed by a normal distribution, then S 1 (P ver ) and S 2 (P ver ) are the associated mean and three-sigma value, that is S 1 (P ver ) = UE(P ver ) and S 2 (P ver ) = UE(P ver ) + 3σ(P ver ). In virtue of three-sigma principle, the probability that a value greater than S 2 (P ver ) is not an active point is less than 0.1%. AT(P ver ) is the active threshold of gene P ver . Consider the gene (P ver ) at timestamp i. If Ev i (P ver ) > AT(P ver ), then the gene P ver is expressed and the gene product exists 16 .
In the clustering procedure of every subnetwork, the proposed method starts with constructing the initial protein clusters depending on the protein complexes attained at the prior time point. The initial clusters constructed in the first generation have three steps The procedure for constructing initial clusters has three steps: seed node selection, attachment nodes addition and finally refining 1 . To clearly demonstrate the three steps, a 1. Selecting seed nodes: This step first computes the clustering coefficient of every node. Then it selects the nodes whose clustering coefficients are greater than a given threshold λ c as seed nodes, and puts them into the set of seed nodes at the current time point t, denoted by S t . The seed nodes are considered as the candidate clustering centers and represent different clusters of protein complexes. The clustering coefficient of any node i is defined in Eq. (6): where Neigh(i) = {j є P ver t |(i · j) є P Edg t } represents the neighbor nodes of node i, and |Neigh(i)| is the number of neighbor nodes of node i, n i t is the number of links between neighbour nodes of i at the time point t. 2. Attachment nodes addition: For any seed node i (i є S t ) of current time point t, if it is also the seed node of previous time point (t − 1), then the nodes which are in the cluster i at the previous time point (t − 1) and also exists in the subnetwork P p t at the current time point t are put into the cluster i of current time point t. In this way, initial clusters are built. However, some clusters may be too sparse since that not all proteins of previous time point (t − 1) exist at the current time point t. Thus, a refining step is needed to be carried out on the initial clusters. 3. Refining: For any initial cluster of protein complex c i t at the current time point t, if its density is smaller than a given threshold λ d all the nodes in c i t are sorted in a descending order according to their clustering coefficients, and the node with the smallest clustering coefficient is iteratively removed until the density of cluster c i i t is not smaller than the given threshold λ d . The density of a protein complex c i t is computed by Eq. (7): where n i and l i are number of nodes and edges in cluster c i t respectively 1 . Now, the clustering analysis of the remaining generations is employed by utilizing the Markov Clustering technique along with the EHO algorithm on every subnetwork. The matrix is constructed that depicts the probabilities of transition of a Markov Chain (random walk) based on the graph. The MCL procedure comprises of two activities such as expansion and inflation, which was applied to the matrix that was constructed. The construction of matrix M at for a graph description and the process of Markov clustering method is briefly described 18 .
Let P p = (P ver , P Edg ), where P ver , is a set of proteins and P Edg , is a set of interactions between these proteins. Denote a node in P ver by p vi and an edge between p vi and p vj in P Edg by (p vi , p vj ), in which i and j are the indexes of the corresponding nodes 16 . W(p vi , p vj ) is the weight of edge (p vi , p vj ), which represents the confidence level of the interaction in a weighted PPI networks. Adj is the adjacency matrix of a weighted graph given as Eq. (8) A canonical flow matrix M at is an n × n (n = |P ver |) matrix that shows the probabilities of transition of a random walk defined on the graph. M at (i, j) represents the probability of a transition from node p vi to p vj . The transition probability from p vi to p vj is referred to as the stochastic flow from p vi to p vj . All the elements in each column of M at will sum up to 1 and the matrix is expressed as given in Eq. (9) at k 1 n

= ∑ =
The three crucial parameters of MCL are inflation constant (ic), balance (b) and penalty proportion (P p ), where ic defines the size of each cluster, b defines the user-specific balance constant that is employed for penalizing higher-propensity neighbours and P p defines the penalty ratio of the protein nodes, which is also user-specified 16 . The clustering process using EHO algorithm is briefly explained here for clustering protein complexes. The overall flowchart of the proposed method is shown in Fig. 1. elephant herd optimization. One of the contemporary swarm intelligence technique is the elephant herd optimization which was projected in 2016 19 . This algorithm was stimulated by the herding characteristics of elephants. In general, elephants are social mammals with the composite social group comprising of numerous clans under the guidance of a matriarch. A clan comprises of one or more female elephant with their calves. Female desires to live in domestic clusters while male elephants prefer to live alone and they will exit from the clan when they grow with each generation 20 . The characteristics of the clans signifies exploitation and leaving elephants signifies the exploration of the population. (2019) 9:11106 | https://doi.org/10.1038/s41598-019-47468-y www.nature.com/scientificreports www.nature.com/scientificreports/ The characteristics of an elephant are measured using two main operators, namely clan updating and clan separating operators that are used for producing better clustering of proteins. Here, the elephant population is referred to as the static PPI network, each clan is referred to as the dynamic PPI subnetwork, and the elephants inside each clan is represented as proteins.
Clan updating operator. The static PPI is initially separated into k dynamic PPI. Each dynamic PPI is headed by the individual protein, which represents the best solution of the dynamic PPI. In each generation, protein e of dynamic PPI cl i moves towards the p best cl , i which has the best fitness in dynamic PPI cl i . The fitness of the dynamic PPI is computed by employing the accuracy values of the protein complex. For new protein e in dynamic PPI cl i , the position is updated by following Eq. (10).
new cl e cl e b est cl c l e , , , , , is the new position of protein e in dynamic PPI cl i and p cl e , i denotes the position in previous generation. p best cl , i signifies dynamic PPI cl i which has the best fitness, α is the scale factor that determines the influence of best fitness and rand is the random variable employed to enhance the diversity of the populations and defined in the range (0, 1).
The movement of a protein e for best fitness can be updated using Eq. (11).
p p (11) best cl e c enter cl , , , where β belongs to (0, 1) which is a scale to regulate the effect of p center cl is the centre of dynamic PPI cl i and for the di th dimension it can be computed using the Eq. (12).
The centre of the dynamic PPI cl i is computed through DI computations using Eq. (12). The pseudocode for the dynamic PPI updating operator is depicted in Algorithm 1.
Clan separating operator. To enhance the search capacity of the proposed method, the unclustered proteins and clusters with the lowest fitness will exit in every generation as given in Eq (13) 19 .
where p max and p min are the upper and lower bound of the single protein.
is the protein or complex with the lowest fitness. The rand is the random variable that has stochastic and uniform distribution in the range (0, 1). The pseudocode for the clan separating operator is given in Algorithm 2.
Depending on the clan updating and separating operator, the module of the proposed algorithm is framed as given in Algorithm 3.  www.nature.com/scientificreports www.nature.com/scientificreports/ The relationship between the DMCL-EHO and the protein complex is given in the Table 1.

experimental Results
Datasets. In this experiment, the datasets which consists of interactions for both Saccharomyces cerevisiae and Homo Sapiens are DIP 21 , BioGRID 22 and STRING 23 . The benchmark PPI datasets employed only for Saccharomyces cerevisiae are Gavin2 and Gavin6 24 , Krogan-core and Krogan-extended 25 , Collins 26 , and WI-PHI 27 . The Gavin + Krogan dataset was generated by merging Gavin and Krogan Core datasets. The PPI datasets employed only for Homo Sapiens are HPRD 28 , HPID 29 and PIPs 30 . Table 2 shows the list of datasets used in this experiment.
The gene expression data used in this study for Saccharomyces cerevisiae (GSE3431) 31 and Homo Sapiens (GSE3933) 32 are taken from the GEO database.
The predicted complexes are compared to gold standard benchmark databases such as CYC2008 33 , MIPS 34 , SGD 35 for Saccharomyces cerevisiae organism and PCDq 36 benchmark dataset for Homo sapiens organism. The percentage of overlapping interactions among the datasets in Gavin2 is 32%, Gavin6 is 53%, Krogan-core is 46%, Collins is 56%, HPRD is 23%, PIPs is 57%, DIP is 2%, BioGRID is 55% and STRING is 47% 37,38 . performance measures. To evaluate and compare the clustering results of predicted protein complexes, the generated complexes were compared and matched with the gold standard benchmark protein complexes. Assume P r (V Pr ,E Pr ) and B e (V Be ,E Be ) be the set of vertices (proteins) and edges (interaction) of a predicted protein complex and benchmark protein complexes.
Complex similarity score (CSS). CSS is defined as the closeness of two protein complexes namely predicted (P r ) and benchmark (B e ) protein complexes and they are computed based on Eq. 14.  where V pr and V Be denotes the set of proteins in predicted and benchmark protein complexes. If CSS(P r , B e ) is equal to 0, it denotes that the predicted and benchmark protein complexes do not have any common protein complexes. On the contradictory, if CSS(P r , B e ) is equal to 1, then the predicted complex P r (V Pr , E Pr ) has the same equal nodes as the benchmark complex B e (V Be , E Be ). Here, if CSS(P r , B e ) > 0.2, it is considered as the predicted and benchmark protein complexes are identical 39 .
Recall. Recall is defined as the accuracy of benchmark protein complexes that are identical to the predicted complexes. If the recall value is high, it indicates that the predicted complex has a good number of coverage of the proteins in the gold standard complexes. The recall of the protein complexes is computed based on Eq. (16).
where N Pc is denoted as the number of predicted complexes which match at least one recognized benchmark complex, N Bc is denoted as the number of recognised benchmark complexes which match at least one predicted complex, Predicted set is denoted as the set of complexes predicted by the proposed approach and Known set is denoted as the set of recognised gold standard benchmark protein complexes.
Coverage ratio (CR). CR is defined as the fraction of proteins in benchmark complex V Be found in predicted complex V pr and they are computed based on Eq. (17).

Elephant
The temporary proteins in dynamic subnetwork.
where V Be is denoted as the set of proteins in benchmark protein complexes. T i, j is denoted as the common number of proteins between V pr and V Be .
F-Measure. F-Measure is defined as the harmonic mean, i.e., a rational mixture of both precision and recall and it is computed based on Eq. (18).     Number of Clusters. The number of clusters is defined as the total quantity of clusters formed from the PPI network after the clustering process has been completed.
The performance measures such as coverage ratio, the number of clusters, precision, recall, f-measure and accuracy of the proposed method for Saccharomyces cerevisiae are compared with various datasets and existing algorithms against CYC2008 benchmark database and the graphical representation of the comparison is depicted in Figs 2-7. Also, the performance measures such as coverage ratio, the number of clusters, precision, recall,     www.nature.com/scientificreports www.nature.com/scientificreports/ From Figs 2 and 8, it is inferred that the number of clusters in the proposed method is less when compared to FOCA, AFA-MCL and ACO-MCL as they try to get solution from all the proteins in the network. These methods will not discard the undesirable proteins which may result in false positives. But in the proposed method, the clusters which has less than three proteins are discarded. Hence the precision, recall, F-Measure and accuracy are high for the proposed method.
From Figs 3 and 8, it is observed that the proposed method has more coverage ratio than the existing methods since it employs the iterated clustering approach. This enhances the coverage of proteins in the network as the proteins in the benchmark complexes are highly found in the predicted complexes. From Figs 4-7 and 9 it is observed that the precision, recall, F-Measure and accuracy shows fluctuations for PSO-MCL, ACO-MCL, AFA-MCL, F-MCl, FOCA and EHO-MCL. The mean of these measures for all the datasets shows that the proposed method performs better than the existing methods because it has employed the dynamic PPI along with EHO.

implementation and Discussion
The computational issue of attaining a solution with a high accuracy solution for protein complex detection from dynamic PPI is still a challenging task. In this paper, the elephant herd optimization algorithm along with Markov clustering technique is combined to solve the protein complex detection problem. The proposed method provides an enhancement of the results compared to all the other popular existing methods. This work was executed on 2.00 GHz Intel i3 with 8GB of memory running on Windows 10.
The number of clusters is small in an average when compared to other existing methods, due to the deletion of proteins without interactions. Here, the minimum number of proteins inside a cluster should be three or more and that are considered as a protein complex. The protein cluster with less than three proteins are removed. The proposed method was evaluated based on the removal of noise, insertion and deletion of random protein interactions, large PPI network, namely WI-PHI, various parameter analysis, statistical significance and finally with biological significance. evaluation by noise removal. The PPI networks are obtained from high-throughput experiments, the large coverage of the PPI network comprises of noise in the format of false positive interactions and redundant data. The main challenge of clustering these PPI networks is present in the PPI networks itself. In this method, after the clustering process is accomplished, the proteins that do not present in any of the clusters is also considered as a noise. These solitary proteins that do not interact with any other proteins will not provide any valuable information. The minimum number of proteins inside the cluster is set to be three in this work. Thus, the isolated proteins and clusters with below three proteins are considered as a noise and they are removed by the clan separating operator by the elephant herd optimization method. Many evolutionary approaches are inheriting the undesirable proteins from one generation to another which may lead to loss of accuracy, but EHO approach will discard the undesirable proteins from the population in the clan separating operator that leads to the optimal solution. The comparison of EHO with other existing methods is depicted in  evaluation by adding and removing random protein interactions. The testing of the proposed method is accomplished by inserting and deleting the random interactions of the PPI network to evaluate its performance. The noise can also be any missing information (false negatives) or added noise (false positives) in the PPI network. The DIP dataset is used for evaluation of adding and removing random interactions. The missing information of PPI network is processed by removing the proportion of edges randomly (0%, 20%, 40%, 60%,   ) and the false positive information of PPI network is processed by adding the proportion of edges randomly (0%, 20%, 40%, 60%, 80%, 100%). The performance of the proposed method by adding and removing the random interactions are depicted in Figs 10 and 11. From the Figs 10 and 11, it is observed that even though the random insertions and deletion of the protein interactions are employed on the dataset, the proposed method performs better than other existing approaches.    www.nature.com/scientificreports www.nature.com/scientificreports/ evaluation by parameter analysis. Generally, every metaheuristic approach is based on certain stochastic dissemination. Hence, diverse runs will produce various diverse results. This work implements 500 independent runs in order to score optimal solution. In general, 20 numbers of clans were employed as per literature. The execution process will be terminated, if the best result generated in each iteration remains interchangeable for 100 successive iterations or the maximum number of generations is attained. The assignment of parameter values was adjusted based on the experimental results. It was identified that the parameters of the proposed method that has values of α = 0.5 and β = 0.1 produced better solution among different values and hence were allocated. It was observed that the optimal solution was identified after 315 th generation. For all the performance measures, there were fluctuations during the first 10 runs of the experiment and in the future runs reliability was observed. Figures 2-9 shows the average outcome of performance measures for the above parameter values of the proposed method. Table 3 shows the various parameter values for the proposed approach and the other existing approaches of protein complex detection. evaluation by statistical significance. The proposed method was also assessed by utilizing non-parametric test such as, Wilcoxon Matched-Pair Signed-Rank Test among each pair of approaches that produces the statistical consequence. The discrepancy between the F-Measure and Accuracy for every entry in Figs 6 and 7 was tested based on the confidence level of 1% (p-value < 0.01). The p-value less than 0.01 are assumed as highly significant and the values greater than 0.01 are assumed as insignificant values. The scores of F-Measure and Accuracy is alone considered as they are computed based on precision and recall. The Statistical Significance  of the proposed and existing approaches based on F-Measure and Accuracy is depicted in Table 4. The scores of upper right positions of the table are attained from F-Measure of proposed and various existing algorithms based on DIP dataset against CYC2008 benchmark database. The scores of lower left positions of the table are attained from Accuracy of proposed and various existing algorithms based on DIP dataset against CYC2008 benchmark database. From Table 4, it is shown that the proposed method is statistically significant in nature compared to all the existing methods.
Evaluation by biological significance. Many of the existing methods solve the protein complex detection problem based on the topological similarity. But to obtain some useful biological information, the computational methods should be biologically significant in nature. This proposed method is evaluated in the biological significance test. The gold standard benchmark databases are manually annotated based on the information from biologically experimental analysis. Thus, the detected protein complexes obtained from the proposed method is compared and matched with the benchmark databases. Few benchmark databases such as CYC2008, MIPS, SGD databases for Saccharomyces cerevisiae and the PCDq database for Homo sapiens are employed for assessing the proposed method. Table 5 displays the common protein complexes between the CYC2008 benchmark database and the proposed method for DIP and Krogan-extended. Also, the common biological process, molecular function and the cellular component of the obtained protein complexes are displayed. Correspondingly, the common pathway annotations of the predicted protein complexes are obtained from the KEGG database are displayed.
The predicted complex gene ontology and KEGG pathway enrichment analysis were predicted by using the DAVID gene function classification online tool. The overall predicted complex enrichment score and the respective gene ontology elements and KEGG pathway enrichment scores are displayed in Table 5. The pictorial representation of the common RNA Polymerase KEGG Pathway of the predicted protein Complex on Krogan-extended dataset and common Oxidative Phosphorylation KEGG Pathway of the predicted protein complex on DIP dataset is exhibited in the supplementary information. The RNA polymerase is essential for nucleolar assembly and for high polymerase loading rate. Oxidative phosphorylation is the metabolic pathway in which cells use enzymes to oxidize nutrients, thereby releasing energy which is used to produce adenosine triphosphate (ATP) [40][41][42] . The pictorial representation of the Top 5 common protein complexes, gene ontology functions and KEGG pathways of the predicted complexes of proposed method is given as Venn diagram in Figs 12 and 13. execution time. Besides the accuracy, the time required to detect the dynamic protein complexes is also an important factor. Processing the various benchmark datasets with various numbers of proteins and different interactions requires more time complexity due to stochastic optimization methods. Subsequently, not all methods were available under the same platform, the execution of many of the approaches were done on virtual machines, which prohibited us from accomplishing an exact comparison of their relative execution times. Thus, here the average execution time of SR-MCL, ACO-MCL, PSO-MCL, AFA-MCL, F-MCL, FOCA AND EHO-MCL is displayed in Fig. 14.
From Fig. 14, it is observed that in this research, the proposed algorithm has less execution time when compared to other algorithms, due to the clan separating operator of EHO approach. It is inferred that the proposed EHO-MCL is efficient for detecting dynamic protein complexes. In future, the EHO-MCL can be further optimized in multicore CPU.

conclusion
The Protein Complex detection is an exposed problem for scientists. The solution for the complex problem should be recurrently improved as they are important in the analysis of the biological process. The volume of PPI networks has also been increased due to high-throughput experiments, the lack of accurate computational model for protein complex detection exists. Many of the existing researches were employed on the static PPI data that do not provide accurate biological results. Thus, in this proposed method initially, the static PPI data is converted into dynamic PPI data by integrating the gene expression data. Later, every dynamic subnetwork was clustered based on the popular clustering technique MCL along with the elephant herd optimization method for exploring and exploiting the better solution. The proposed method was employed on various 11 widespread datasets and the predicted complexes were compared with 4 different benchmark databases. Also, the proposed method was evaluated based on noise removal, insertion and deletion of random protein interactions, using the large PPI dataset, various parameter analyses, statistical significance and biological significance. On every evaluation phase, the proposed method was outperforming all other existing approaches and identified the common protein complexes, Gene Ontology functions and KEGG pathways of predicted protein complexes. As a future work, additional information on the unknown protein complexes predicted by the proposed method is to be addressed with the help of biological experts. The proposed method can also be applied and analyzed on weighted PPI networks. Also, various other diseased databases can be used to experiment.


text:
Deep Feature Selection and Causal Analysis of Alzheimer’s Disease

Deep convolutional neural networks (DCNNs) have achieved great success for image classification in medical research. Deep learning with brain imaging is the imaging method of choice for the diagnosis and prediction of Alzheimer’s disease (AD). However, it is also well known that DCNNs are “black boxes” owing to their low interpretability to humans. The lack of transparency of deep learning compromises its application to the prediction and mechanism investigation in AD. To overcome this limitation, we develop a novel general framework that integrates deep leaning, feature selection, causal inference, and genetic-imaging data analysis for predicting and understanding AD. The proposed algorithm not only improves the prediction accuracy but also identifies the brain regions underlying the development of AD and causal paths from genetic variants to AD via image mediation. The proposed algorithm is applied to the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset with diffusion tensor imaging (DTI) in 151 subjects (51 AD and 100 non-AD) who were measured at four time points of baseline, 6 months, 12 months, and 24 months. The algorithm identified brain regions underlying AD consisting of the temporal lobes (including the hippocampus) and the ventricular system.

INTRODUCTION
Alzheimer's disease (AD) causes progressive brain atrophy and memory loss, is a progressive, irreversible degenerative disease of the brain, and is the most common neurodegenerative disease in the world (Struyfs et al., 2015;Zhuang et al., 2017;Liu et al., 2018a,b). AD is an increasingly prevalent disease affecting an estimated 5.4 million Americans and more than 30 million people in the world. It is estimated that these numbers will be tripled by 2050. AD is the sixth leading cause of death in the United States (Alzheimer's Association, 2016; Leandrou et al., 2018).
Diagnosis and prediction of AD via clinical and psychometric assessments are challenging (Leandrou et al., 2018). The AD patients cannot obtain early and accurate diagnosis through clinical dementia rating and cognitive tests. A final diagnosis of AD is confirmed by histological examination at postmortem biopsy. However, the histological examination of the brain for the living patients is infeasible. Individually varying brain structure, function, and pathological effects can be measured by images. Therefore, imaging plays an important role in improving diagnosis and prediction of AD. According to the recommendation by the National Institute of Neurological and Communicative Disorders and Stroke-AD and Related Disorders Association (NINCDS-ADRDA) Work Group, the clinical classification of AD should explore the image markers: magnetic resonance imaging (MRI), diffusion tensor imaging (DTI), positron emission tomography (PET), amyloid-PET, tau-PET, and abnormal neuronal cerebrospinal fluid (CSF) markers (tau and/or Aβ) (Dubois et al., 2007;Leandrou et al., 2018;Liu et al., 2018a,b).
As the size of the imaging datasets increases, manual analysis of imaging data is tedious and time-consuming. Computeraided diagnosis (CAD) of AD that combines computational models and analytical tools for high-dimensional imaging data analysis is emerging as one of the major tools for diagnosis and prediction of AD (Dimitriadis et al., 2018;Leandrou et al., 2018). The widely used machine learning (ML) methods in CAD include discriminant analysis (DA), logistic regression (LR), random forest, neural networks, and support vector machine (SVM) (Lorenzi et al., 2017;Sarica et al., 2017;Dimitriadis et al., 2018;Leandrou et al., 2018). Deep learning, a rapidly resurging subfield of ML, outperforms many classical ML approaches and is emerging as a major analytic platform in ML (Esteva et al., 2019). Deep learning with massive amounts of computational power has produced a revolution in driverless cars, speech recognition, and imaging analysis (Waldrop, 2019) and demonstrated great potential for the diagnosis and predictive power in tuberculosis (Heo et al., 2019), cancer (Esteva et al., 2017;Haenssle et al., 2018;Ghatwary et al., 2019;Ladefoged et al., 2019), diabetic retinopathy (Gulshan et al., 2016), chronic kidney disease (Ravizza et al., 2019), AD (Payan and Montana, 2015;Hosseini-Asl et al., 2016;Sarraf and Tofighi, 2016;Ju et al., 2017;Ding et al., 2018;Wada et al., 2018;Spasov et al., 2019), and conversion from mild cognitive impairment (MCI) to AD (Choi et al., 2018;Spasov et al., 2019). There is a growing interest in the application of deep learning to health care and medicine.
Despite its great progresses in computer vision, natural language processing, control, decision making, diagnosis, and early detection of complex diseases, deep leaning is also well known as a "black box" owing to its low interpretability to humans and still has a serious opacity problem (Waldrop, 2019). Overcoming the limitation of the lack of transparency and interpretation remains a great challenge for deep learning (Dubois et al., 2007). In this paper, we develop a novel general framework that integrates deep leaning and causal inference for image classification. The new framework for image analysis consists of two stages: (1) develop convolutional neural networks (CNN) to classify AD status on the basis of DTI and use of occlusion map to find image regions that are most distinctive for disease status and (2) the use of state-of-the-art causal inference tools to determine if the selected image regions are causal for AD.
Brain anatomy, structural connectivity, and physical connection between brain regions that are characterized through water molecular diffusing within white matter tracts can be measured by DTI. The imaging signals provide intermediate endophenotypes. Genetic variants will influence brain microstructure, function, and disease development. Understanding the role that genetics has in imaging and disease variation is key to understanding the causal chain of complex diseases (Jahanshad et al., 2013;Bycroft et al., 2018;Elliott et al., 2018). Therefore, to further cover the genetic bases of brain structures and function, and mechanism of AD, a joint analysis of the genetic brain images and AD will be carried out. We will assess both association and causal relationships among genetic variants, brain regions, and AD.

Materials
The DTI images used in this study are downloaded from the Alzheimer's Disease Neuroimaging Initiative (ADNI); the size of each image was 91 × 109 × 91. ADNI is a longitudinal multicenter study designed to develop clinical, imaging, genetic, and biomedical biomarkers for the early detection and tracking of AD 1 (Alzheimer's Disease Neuroimaging Initiative, 2019). DTI images were recorded for every participant from different time points in which they joined the research study. The diagnostic results were normal control (NC), MCI, and AD. In this study, DTI images of 151 individuals from NCs (100 images) and AD (51 images) groups were chosen from four different diagnostic time points: baseline, 6 months, 12 months, and 24 months.

Image Preprocessing
To make sure that all the images for this analysis are comparable, we register all the DTI image data for every subject at every time point to the common template, which can be downloaded from the McConnell Brain Imaging Centre 2 . We utilized a strategy of combination of linear and non-linear registration algorithm to map each individual DTI data to the common template. During the linear image registration procedure, we first map the image data to the common template to make sure all the images are within the standard brain region by using FLIRT (FMRIB's Linear Image Registration Tool) from FSL (FMRIB software library) image analysis suite 3 . Then we further applied non-linear registration algorithm, which is implemented in RNiftyReg to map the image details within the standard brain. The linear image registration process helps us restrain each individual DTI image to a standard template, and the non-linear image registration helps us to make sure that the registered image maintains the structures details as the original data.

Genetic Data Preprocessing
We performed quality control (QC) in both individual level and single-nucleotide polymorphism (SNP) level QC in the plink binary format. For the individual level QC, the following steps were applied to the data: 1. Individuals with discordant gender information were removed from the data. 2. Individuals with missing rate >10% were removed from the data.
3. Individuals with heterozygosity rate of more than three standard deviations from the mean were excluded from the data. 4. Individuals with identity by descent (IBD) > 0.185 were excluded from the data.
After the individual level QC was conducted, the following steps for SNP level QC were further applied to the data: 1. SNPs with missing genotype rate >10% were excluded from the data being analyzed. 2. SNPs with P-value for Hardy-Weinberg equilibrium (HWE) test <1E-6 were excluded from the data. 3. SNPs without polymorphism were removed from the data.
Then pre-imputation QC tool from McCarthy Groups was further applied to check the data against 1000G reference data. The imputation of the genetic data was conducted under the SHAPEIT + IMPUTE2 framework in the internal computational clusters. The 1000G reference data were used as the reference panel for imputation. After the imputation, the SNP level QC steps were applied again to the data to produce the final genetic data for analysis. Finally, a total of 1,589,061 common SNPs in 36,480 genes genotyped in 151 individuals were included in analysis.

Architecture of Convolutional Neural Network
The CNN model Visual Geometry Group (VGG) that won the first and second places in the localization and classification tracks, respectively, in the ImageNet Challenge 2014 was chosen for image classification and prediction (Simonyan and Zisserman, 2014). To improve the classification accuracy, the VGG utilized smaller receptive window size and increased the depth of the network. Furthermore, to prevent overfitting and improve the image region recognition ability of the networks, global average pooling (GAP) layer was used as a structure regularizer and localizer in the model to identify the complete extent of the object and exactly which regions of an image are being used for classification (Zhou et al., 2016).
As is shown in Figure 1, the network contained five max pooling layers, followed with a GAP layer before a fully connected softmax layer with two nodes.
Three-dimensional (3D) whole brain images with 109 × 91 × 91 size were input into CNN. DTI measures microscopic random motion of water molecules, which uncovers the orientation of surrounding tissues, and provides tract information on brain structure. Convolution of an image with different filters can perform operations that capture various types of features and directional information of DTI images and can preserve tract of DTI and the relationship between pixels. 3D CNNs (3D-CNN) with five convolutional layers and three fully connected layers were used for AD prediction. A 3D filter was applied to the dataset, and the filter moves in three directions (X, Y, Z) to calculate the low-level feature representations. Specifically, 3D filters were arranged as in Table 1.
To overcome the small sample size limitation of medical images, image augmentation techniques were used (Aderghal et al., 2017). The first technique we applied was Gaussian filters to blur the image to mimic the possible variations in the original images. A filter size of 3 × 3, 5× 5, and 7× 7 were used with spread parameters of 0.7, 0.7, and 0.6, respectively. The second augmentation technique we used was translation, where we shifted the images by ±1 pixel in each dimension. This imitates the possible variations in registration process where the images were aligned with the template. Finally yet importantly, the images were flipped horizontally because some regions of the brain (e.g., the hippocampus) are symmetrical to enlarge our sample size. To balance the data, we randomly duplicated some images from the under-sampled category. Data augmentation and class balancing produced over 20 times more data than the original dataset.
The model was trained in the Texas Advanced Computing Center (TACC) Maverick2 with NVIDIA GTX 1080 Ti GPUs.

Deep Feature Selection for Diffusion Tensor Imaging Images
Prediction difference analysis for visualizing the response of CNN to a specific input was used to select features for DTI image classification (Zintgraf et al., 2017). Specifically, prediction difference analysis estimates the importance of input pixels by calculating the effect of removing information from the imaging on the class prediction precision (Zeiler and Fergus, 2014).
A sliding window (patch) of 3 × 3 × 3 was applied to each image. The imaging signals contained in the sliding window were taken as a feature. Each one 3 × 3 × 3 patch was replaced by randomly sampled values from multivariate normal distributions. The resulting new image where the imaging feature (information) was removed was input into a previously trained CNN model to obtain probability p 1 for predicting AD. Let p 0 be the probability of predicting AD using the original images [without removing the feature (information)]. The relative importance of the feature was evaluated by Zintgraf et al. (2017).
The sliding window moved across the entire image and a relevance matrix, W of the same size as the whole image was generated, which reflected the relevance importance of all image pixels. A positive value indicated that the pixel contributed evidence for the classification of AD, whereas a negative value showed that the pixel contributed against the classification of AD. For details, please see Zintgraf et al. (2017).

Conditional Generative Adversarial Network and Classifier Two-Sample Tests for Causal Discovery
Three-dimensional functional principal component (FPC) scores were used to summarize the imaging signal information of the brain region (Xiong, 2018). Similarly, 1D FPCs can be used to summarize genetic information in the gene. Conditional generative adversarial networks (CGANs) will be used to discover causal relationships between the brain neuroimaging region and AD and causal relationships between the brain neuroimaging region and gene as well (Goodfellow et al., 2014;Lopez-Paz and Oquab, 2017) (Figure 2). Specifically, consider two variables X and Y, which can be binary disease status or continuous FPCs summarizing imaging signals in the brain region or genetic variation in the gene. If X causes Y, denoted by X → Y, then we have where f Y is a non-linear function and realized by CGAN where a neural network is used to approximate the non-linear function f Y (X, N Y ), and N Y is a noise random variable and is independent of cause X. Similarly, if Y causes X(Y→X), then we have where f X is a non-linear function and N X is a noise random variable and is independent of cause Y. Assume that n subjects are sampled.
We define dataset D w = {u i , v i , i = 1, ... , n}. We assign label 0 to dataset D u = {u i , i = 1, ... , n} and 1 to dataset D v = {v i , i = 1, ... , n}. Let P be the distribution of u i , i = 1, ... , n and Q be the distribution of v i , i = 1, ... , n. We use the K nearest neighbor (KNN) as a binary classifier to classify two datasets and define the test statistic t as the classification accuracy to test the null hypothesis of equal distributions of two datasets P = Q. Let z be a random variable.
The procedures for bivariate causal discovery using CGAN are summarized as follows (Lopez-Paz and Oquab, 2017): 1. Use a CGAN from X→Y to generate the dataset D X → Y = { (x i ,ŷ i = f y (x i ,z i )), i = 1,. . . ,n}.
2. Use a CGAN from Y→X to generate the dataset D Y→ X = { (x i = f X (y i ,z i ),y i ), i = 1,. . . ,n}.
3. Divide the total samples into training samples and test samples. 6. Calculate the test statistic T =t X→Y −t Y→X . Under the null hypothesis of no causal relationship or test inconclusive, the statistic T is asymptotically distributed as  N(0,σ 2 ), where σ 2 = 0.5 n test − 2cov(t X→Y ,t Y→X ) and n test is the number of subjects in the test set.
Association is defined as measuring the dependence or correlation between two variables and to use these dependencies for prediction that is not dealing with causal problems. Almost all currently used statistical methods in imaging genetics [such as sparse canonical correlation analysis (SCCA), sparse reduced rank regression (SRRR), and parallel independent component analysis (ICA)] are association analysis methods. These methods can detect association between genetic variation and imaging signals. It is well known that correlation or association analysis does not imply causation. The signals identified by association analysis may not have specific pathological relevance to diseases. Association signals provide limited information on the causal mechanism of diseases. Most genetic and imaging analysis questions to uncover the mechanism of the disease are causal in nature. Causation analysis is essential to the genetic analysis of complex phenotypes yet ignored for a long time.
Distinguishing causation from association is an ageold problem. Intuitively, causation implies that changes in one variable will directly make changes in the other. The essential distinction between association and causation relies on what the response will be if we intervene in the system (Lattimore and Ongv, 2018).
There are two types of causal inference: interventional causal inference and observational causal inference. Interventional causal inference learns the effect of taking an action directly via experiments, for example, randomized controlled trials. Interventional experiments are a gold standard for causal inference. However, because in human genetics we cannot change the genetic materials of human subjects, experimental interventions are unethical and infeasible. Therefore, it is essential to develop statistical methods and algorithms to predict the outcomes of an intervention from passive observation.
The additive noise models (ANMs) assume one causal direction X → Y but no reversible causal direction Y → X. Causation is asymmetric. However, the association of X and Y can be (1) X → Y, (2) Y → X, and (3) X → Y, Y→ X. Association is symmetric.
Additive noise models are based on the independence of cause and mechanism (ICM) principle. ICM assumes that causes and mechanisms are chosen independently by nature, which is a recently proposed principle for causal reasoning and causal learning (Peters et al., 2017). ICM assumes that the mechanism that generates effect from its cause contains no information about the cause, which implies that X and N Y in the ANMs are independent. However, X and N Y in the non-linear regression model Y = f Y (X, N Y ) may be dependent.
In summary, association is studied by observed conditional distribution, and causation is investigated by interventional distribution where causal effect is determined by the effect of hypothetic manipulation of an input on an output. In other FIGURE 3 | Visualization of the brain regions with relative importance values at the baseline, 6 months, 12 months, and 24 months. The deeper the red color of the brain region, the more important for AD prediction. AD, Alzheimer's disease.
words, association is investigated by seeing, and causation is investigated by doing.

Alzheimer's Disease Classification and Prediction
The VGG network with 3D filters was used for classification and prediction of AD using 3D whole brain DTI images at four different time points: baseline, 6 months, 12 months, and 24 months. We consider two classes: AD and NC. AD prediction accuracy using VGG is listed in Table 2, and its sensitivity and specificity are shown in Table 3, where the first and second values in the brackets represent sensitivity and specificity, respectively. Tables 2, 3 demonstrate that the prediction accuracy, sensitivity, and specificity of VGG using the training dataset at baseline to predict AD in the test datasets at baseline, 6 months, 12 months, and 24 months were 0.8675 (0.6873, 0.9600), 0.8452 (0.6364, 0.9600), 0.8335 (0.7295, 0.8995), and 0.7463 (0.6294, 0.8853), respectively. In other cases, we can observe similar results.
The area under the curve (AUC) using the training data at baseline, 6 months, 12 months, and 24 months for prediction of AD in the test datasets at the same time points was 0.8571, 0.8291, 0.8583, and 0.7756, respectively. The low sensitivity of prediction of AD may be due to small and imbalanced sample size (51 AD and 100 controls). A much higher proportion of non-AD controls have decreased sensitivity but increased specificity. Deep VGG that has a large number of parameters to be estimated requires large sample sizes. Although we used data augmentation methods to increase sample sizes, augmentation methods still did not provide large and reliable sample sizes. Large sample sizes are an important issue for increasing the prediction of accuracy.

Region Selection and Interpretation
Relative importance of value d was sorted. Image areas whose relative importance value was in the top 10th percentile were considered as features that contributed substantially to the prediction of AD. We identified 23 important brain regions that contributed substantially to AD prediction. The results are shown in Figure 3 where each subfigure has 91 × 109 pixel sizes, where the darker the red color is, the more important the brain region is to the prediction accuracy. The brain regions with red color included the temporal lobe (the left temporal, medial, and right temporal lobes), ventricles and enlarged ventricle, occipital lobe, and prefrontal area. To further interpret the image analysis results and increase their transparency, we tested the causal relationships between DTI image ROIs and AD disease at baseline, 6 months, 12 months, and 24 months using CGAN-based statistics. After Bonferroni correction, P-value < 0.0022 was the threshold to declare significance. The number of identified brain regions that showed significant causation to AD at baseline, 6 months, 12 months, and 24 months was 1, 1, 2, and 4, respectively. Table 4 lists ROIs where P-values for testing causation between the ROI and AD were <0.05. Three remarkable features emerged from these results. First, as time passed, AD progressed from mild (early stage), via moderate (middle stage), to severe (late stage), which resulted in atrophy of more and more brain regions. Therefore, we observed the increased number of significant causal brain regions with AD as the study time of AD increased from the baseline to 24 months. Second, in general, as AD progressed, the significance of causation between the brain region and AD increased (P-values for testing causation decreased). Third, the brain region in ROI 18 (the ventricles and enlarged ventricle) (Figure 4) showed significant causation to AD at all four time points (baseline, 6 months, 12 months, and 24 months). The brain regions in ROI 14 (the left temporal lobe) (Figure 4) showed significant causation at 12 and 24 months after Bonferroni correction. The literature reports that these regions are related to AD. The left temporal lobe is involved in language and AD (Cretin et al., 2015;Flick et al., 2018;Trimmel et al., 2018), and the right temporal lobe atrophy is involved in severe impairment in emotion recognition (Everhart et al., 2015) and causes frontotemporal dementia (Gliebus, 2014), with the brain ventricles often affected AD (Ferrarini et al., 2006). Ventricle enlargement is a useful structural biomarker for the diagnosis of AD (Anandh et al., 2014).

Genetic Studies of Two Brain Regions
To uncover genetic architecture of brain regions, in addition to genetic-imaging association analysis, we conducted geneticimaging causal analysis using the CGAN where imaging signals within the brain region and SNPs within the gene were summarized by 2D functional principle scores and classical functional principle scores, respectively (Lopez-Paz and Oquab, 2016). The total number of candidate genes being tested was 61. After Bonferroni correction, the P-value for declaring significance of both causation and association was 0.00082. We presented the results of P-values < 0.05 in causal analysis and association analysis of genetic variation in 61 candidate genes with two brain regions, the left temporal lobe and frontal and temporal left lobe, and the right temporal lobe as seen in Supplementary Tables S1, S2, respectively, where 61 genes were obtained from genome-wide causation studies of AD in the manuscript (Lin et al., unpublished). In Supplementary Tables S1, S2, the P-values in bold green denote significant causation or association after Bonferroni corrections. The majority of genes that had causal or association relationships with brain FIGURE 4 | Three brain regions showed causation to AD. AD, Alzheimer's disease.
neuroimaging phenotypes were identified at all time points (baseline, 6 months, 12 months, and 24 months). We also observed that these identified genes had causal or association relationships with both the left temporal lobe and right temporal lobe regions. The identified genes CD33, COBL, and APP that had causal relationships with brain neuroimaging regions were confirmed multiple times in the literature (Bradshaw et al., 2013;Mez et al., 2017;Kovacs et al., 2018;Van Giau et al., 2018;Huang C.C. et al., 2019;Huang C.Y. et al., 2019). It was also reported that gene FGF4 was involved in neurodevelopmental disorders (Grillo et al., 2014), FRMD6 was implicated in AD (Hong et al., 2012), Dock9 played an important role in regulation of morphological changes in hippocampal neurons (Kuramoto et al., 2009), H3F3B was associated with a broad schizophrenia phenotype (Manley et al., 2018), SCYL1 was involved in cerebellar atrophy (Lenz et al., 2018), AKAP5 played a significant role in the regulation of sympathetic nerve activities (Han et al., 2016), and PIGC was involved in epilepsy and intellectual disability (Edvardson et al., 2017).

DISCUSSION
In this paper, we presented a general artificial intelligence (AI) platform for prediction of AD using DTI images. Nontransparency could be a major challenge of deep learning for medical image analysis. To meet this challenge, we introduced three approaches to medical image interpretation: feature selection and visualization, causal analysis of neuroimaging region, and genetic-imaging analysis. Feature selection and visualization methods selected and visualized brain regions as a potential pathology of AD. Further CGAN evaluation and two-sample tests discovered potential causal relationships between the brain neuroimaging regions and AD. We observed the increased number of significant causal brain regions with AD when AD progressed. In general, as AD progressed, the significance of causation between the brain region and AD increased (P-values decreased). We observed that the ventricles and enlarged ventricle and the left and right temporal lobes had strong causal relationships with AD. Temporal lobes including the hippocampus are crucial in AD development at the early stages, whereas the ventricles and enlarged ventricle are a useful structural biomarker for the diagnosis of AD. Joint causal analysis of genetic and images of the left and right temporal regions using CGAN evaluation and two-sample tests mapped CD33, COBL, FRMD6, APP, and other genes to the left and right temporal brain regions. Many findings in the paper can be confirmed in the literature. For example, both prediction analysis using deep learning and causal analysis using CGAN and a two-sample test identified the brain temporal lobe region that was involved in AD. The temporal lobe includes the hippocampus and its surrounding regions. It is well known that the temporal lobe consists of structures that are vital for long-term memory. There are numerous reports that the temporal lobe including the left, medial, and right temporal lobes are involved in AD pathology (Kakeda and Korogi, 2010;Li and Chen, 2015;Menéndez-González et al., 2015;Aggleton et al., 2016;Delgado-González et al., 2017;Pettigrew et al., 2017;Wolk et al., 2017;Jung et al., 2018;Kitchigina, 2018;Persson et al., 2018;Grajski and Bressler, 2019;Kenkhuis et al., 2019;Lam et al., 2019;Pasquini et al., 2019;Xie et al., 2019). DTI discovered the functional and structural connectivity between the medial temporal lobe (MTL) and posteromedial cortex (PMC) (Buckner et al., 2008;Pasquini et al., 2019). The MTL includes the hippocampal formation and other cortices. These regions underlie memory processing through interplay with neocortical areas from the PMC. AD-related pathological changes such as tau accumulation and amyloidβ deposition often affect the PMC and MTL regions. The functional and structural disconnections between the MTL and PMC cause the development and progression of AD.
The results in this paper are preliminary. Sample sizes need to be increased and additional datasets analyzed to replicate the results. The purpose of this paper is to stimulate further discussions regarding the great challenges we are facing in developing robust deep learning platforms that combine multiple modes of imaging tools and have high accuracy across multiple datasets and uncovering causal pathways from genetic variants to disease via brain imaging regions.

DATA AVAILABILITY STATEMENT
Publicly available datasets were analyzed in this study. This data can be found here: http://adni.loni.usc.edu/.

AUTHOR CONTRIBUTIONS
YL developed the software and conducted the data analysis. ZL contributed to the data preprocessing and partial writing. NL contributed to data preprocessing. QG conducted the partial data analysis. MX designed the study and wrote the manuscript.

ACKNOWLEDGMENTS
In this study, data used in the preparation of this article were obtained from the ADNI database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public-private partnership, led by principal investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial MRI, PET, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. For up-to-date information, see www.adni-info.org. The authors thank TACC for providing computational resources.


BREAKING! Presenting Fake News Corpus for Automated Fact Checking

Popular fake news articles spread faster than mainstream articles on the same topic which renders manual fact checking inefficient. At the same time, creating tools for automatic detection is as challenging due to lack of dataset containing articles which present fake or manipulated stories as compelling facts. In this paper, we introduce manually verified corpus of compelling fake and questionable news articles on the USA politics, containing around 700 articles from Aug-Nov, 2016. We present various analyses on this corpus and finally implement classification model based on linguistic features. This work is still in progress as we plan to extend the dataset in the future and use it for our approach towards automated fake news detection.

Introduction
Fake news is a widespread menace which has resulted into protests and violence around the globe. A study published by Vosoughi et al. of MIT states that falsehood diffuses significantly farther, faster, deeper and more broadly than the truth in all categories of information. They also stated that effects were more pronounced for false political news than for false news about terrorism, natural disaster, science, urban legends or financial information. According to Pew Research Center's survey 1 of 2017, 64% of US adults said to have great deal of confusion about the facts in the current events. Major events around the globe saw sudden jump in deceitful stories on internet during sensitive events because of which social media organizations and government institutions have scrambled together to tackle this problem as soon as possible. However, fake news detection has its own challenges.  Wardle and Derakhshan First and foremost is the consensus on the definition of fake news which is a topic of discussion in several countries. Considering the complexity of defining fake news, Wardle and Derakhshan instead created a conceptual framework for examining information disorder as shown in Figure -1. Based on this model, EU commission in 2018 categorized fake news as disinformation with the characteristics being verifiably false or misleading information that is created, presented and disseminated for economic gain or to intentionally deceive the public, and in any event to cause public harm. Since our work focuses on false information written intentionally to deceive people in order to cause social unrest, we will be following the definition defined by EU to categorize fake articles into various categories as defined in Section 3 of this paper.
The second challenge is the lack of structured and clean dataset which is a bottleneck for fake opinion mining, automated fact checking and creating computationally-intensive learning models

Related Works
There have been several datasets released previously for fake news, most notably Buzzfeed 2 and Stanford (Allcott and Gentzkow, 2017) datasets containing list of popular fake news articles from 2016. However, these datasets only contain webpage links of these articles and most of them don't exist anymore. Fake news challenge, 2017 3 released a fake news dataset for stance detection, i.e, identifying whether a particular news headline represents the content of news article, but almost 80% of the dataset does not meet the definition that we are following in our work. Many of the articles were satire or personal opinions which cannot be flagged as intentionally deceiving the public. (Wang, 2017) released a benchmark dataset containing manually labelled 12, 836 short statements, however it contains short statements by famous personalities and not malicious false stories about certain events that we are interested in. Finally, the dataset created by using BS detector 4 contains articles annotated by news veracity detector tool and hence, cannot be trusted, as the labels are not verified manually and there are many anomalies like movie or food reviews being flagged as fake.
In this paper, we overcome these issues by manually selecting popular fake and questionable stories about US politics during Aug-Nov, 2016. The corpus has been designed in such a way that the writing style matches mainstream articles. In the following sections, we define our motivation for creating such corpus, provide details on it and present a classification model trained on this corpus to segregate articles based on writing style. We also discuss properties of fake news articles that we observed while developing this corpus.
Other notable works like FEVER dataset (Thorne et al., 2018), TwoWingOS (Yin and Roth, 2018) etc. are focused on claim extraction and verification which is currently out of scope of this paper. In this work, we are solely focused on creating a clean corpus for fake news articles and performing classification of articles into "question-able" and "mainstream" based on writing style. Fact checking, fake news opinion mining and fake claim extraction and verification fall under future work of this paper. For classification task, previous works include Potthast et al. who used stylometric approach on Buzzfeed dataset and achieved an F1 score of 0.41; Pérez-Rosas et al. who used Amazon Mechanical Turk to manually generate fake news based on real news content and achieved an F1 score of 0.76 to detect fake news, and Ruchansky et al. who used Twitter and Weibo datasets and achieved F1 scores of 0.89 and 0.95 on respective datasets. In our work, we were able to achieve an F1 score of 0.97 on the corpus we have created, hence setting benchmark for the writing-style based classification task on this corpus.

Motivation
Fake news detection is a complicated topic fraught with many difficulties such as freedom of expression, confirmation bias and different types of dissemination techniques. In addition to these, there are three more ambiguities that one needs to overcome to create an unbiased automated fake news detector -1. harmless teenagers writing false stories for monetary gains 5 ; 2. AI tools generating believable articles 6 and; 3. mainstream channels publishing unverifiable stories 7 . Considering these elements, our motivation for this problem is to design a system with the focus on understanding the inference of the assertions, automatically fact check and explain the users why a particular article was tagged as questionable by the system. In order to do so, our first priority was to create an efficient corpus for this task. As we will see in the following sections, we have manually selected the articles that contain malicious assertions about a particular topic (2016 US elections), written with an intent of inciting hatred towards a particular entity of the society. The uniqueness of the articles in this corpus lies in the fact that a reader might believe them if not specifically informed about them being fake, hence confusing them whether the article was written by mainstream media or fake news channel. This uniqueness will ultimately help in creating a detector which is not biased towards mainstream news articles on the same topic (2016 elections in this case.)

Creation and Verification
The final corpus consists of 26 known fake articles from Stanford's dataset along with 679 questionable articles taken from BS detector's kaggle dataset. All the articles are in English and talk about 2016 US presidential elections. We, first, went through the Stanford's list of fake articles links and found the webpages that still exist. We then picked the first 50-70 assertive sentences from these articles. After that, we read the articles from BS detector's kaggle dataset and selected those articles which are written in a very compelling way and succeeded in making us believe its information. We then manually verified the claims made by these articles by doing simple Google searches and categorized them as shown in Table -1. Based on the type of assertions, we have come up with two types of labels: Primary Label: based on the assertions they make, articles are divided into 3 categories: 1. False but compelling (innovated lies having journalistic style of writing); 2. Half baked information i.e, partial truth (manipulating true events to suit agenda); and 3. Opinions/commentary presented as facts (written in a third person narrative with no disclaimer of the story being a personal opinion).
Secondary Label: is of 2 types: 1. articles extracted from Stanford's dataset have been tagged as fake as per their own description and; 2. articles taken from kaggle dataset are tagged as questionable. We believe tagging something fake, when they do contain some elements of truth, will be an extreme measure and we leave this task to the experts.
Finally, the corpus was cleaned by removing articles with first person narrative; removing images and video links and keeping only textual content; removing tweets, hyperlinks and other gibberish like Read additional information here, [MORE], [CLICK HERE] etc.

Analysis
Features: We used NLTK package to explore basic features of the content of news articles in terms Comparison with mainstream article: Mainstream articles from publishers like New York Times were extracted from all the news 8 kaggle dataset, which contains news articles from 15 US mainstream publishers. We selected articles from Oct -Nov, 2016 covering US politics during the time of election. There were total 6679 articles retrieved from this dataset which were then categorized into two labels as per our corpus's schema. Table -2 compares characteristics of top 3 sentences from mainstream news articles with our corpus. We can observe that there are not many dissimilarities except the information presented in both the articles.
Observed Characteristics: Although articles in our corpus have many similarities with mainstream articles, there are some underlying patterns that can be noticed by reading all of them together at once. Following are the observations that we made while verifying the claims presented in these articles.
1. All of the news articles were trying to create sympathy for a particular entity by manipulating real stories. They were either trying to sympathize with Donald Trump by mentioning media rhetoric against him, or with Hillary Clinton by mentioning Trump's past.
2. Similarly, they also made false claims against above mentioned entities, by referring leaked  documents from WikiLeaks or other similar sources. In our verification, we found that in most of these articles, only few claims matched the leaked story and rest were invented.
3. Articles first tell false stories against a certain entity and then asks the reader questions such as "Do you think mainstream media is conspiring against you by hiding this story?", "Do you think Google is making algorithms to create an illusion of reality?" After asking such questions, they either leave the reader hanging in contemplation or ask them to "share [the article] if you believe so." 4. The above point then leads to what (Nyhan and Reifler, 2010) describes as backfire effect in which correcting readers actually makes them believe false stories.
5. Fake news writers have learnt/are learning to use mainstream writing style such as, mentioning city name before starting the article, mentioning abbreviations like (SC) for Security Council, (AI) for Amnesty Investor, using elaborate vocabulary etc.  Table 3: To meet the real world scenario, train data and test data have been split with an approximate ratio of 1:10. After creating this corpus, we trained a classification model on the content to see if it can learn any writing patterns which are not visible to human eye and create benchmark results. Our model, as shown in Figure -3, is inspired by the works of (Anand et al., 2016) and uses bi-directional LSTM and character embedding to classify articles into questionable (1) and mainstream (0) by learning writing style. We have not performed any preprocessing like removing punctuation, stop words etc. to make sure the model learns every context of the content. 1-D CNN has been used to create character embedding which has been proven to be very efficient in learning orthographic and morphological features of text (Zhang et al., 2015). This layer takes one-hot vector of each character and uses filter size of [196,196,300]. We have used 3 layers of 1-D CNN with pool size 2 and kernel stride of 3. Finally, we have performed maxpooling across the sequence to identify features that produces strong sentiments. This layer is then connected to bi-directional LSTM which contains one forward and one backward layer with 128 units each. This model was trained with top 3 sentences, containing 20-100 words, and retrieved from total 7175 articles. As per the example shown in Table -2, it can be assumed that top 3 sentences are enough for a normal reader to understand what the article is talking about. Dataset splitting for training and testing was inspired by the findings in the study conducted by (Guess et al., 2019) of Princeton and NYU earlier this year which stated that Americans who shared false stories during 2016 Presidential elections were far more likely to be over 65. Current US age demographic suggests that 15% of the population are over 65 9 . Therefore, we have decided to have the ratio of questionable to mainstream news articles approximately 1:10, as shown in Table -3.

Training and Results
We first trained our model by randomly splitting our dataset into training, validation and test set of sizes 4592, 1148 and 1435 respectively. However, since the dataset is unbalanced, stratified k-fold cross-validation mechanism is also implemented with 5 folds. (Kohavi, 1995) states that stratification, which ensures that each fold represents the entire dataset, is generally a better scheme both in terms of bias and variance. The model was trained on an average number of 2, 296, 000 characters. We have evaluated our models on various metrics as shown in Figure -4. For this problem, we will be majorly focused on ROC and F1 scores. In both the cases, stratified k-fold outperforms random sampling significantly. Training with only random sampling resulted into underrepresentation of data points leading to many false positives. This can also be because writing style of questionable and mainstream articles are very similar. On the other hand, 5-fold cross validation performed significantly well in learning the patterns and avoiding the problems of false positives.

Conclusion
In this paper, we have introduced a novel corpus of articles containing false assertions which are written in a very compelling way. We explain how this corpus is different from previously published datasets and explore the characteristics of the corpus. Finally, we use a deep learning classification model to learn the invisible contextual patterns of the content and produce benchmark results on this dataset. Our best model was able to achieve state of the art ROC score of 97%. This is a work in progress and we are planning to extend this corpus by adding more topics and metadata. Future work also involves claim extraction and verification which can be further used to design an unbiased automated detector of intentionally written false stories.

Acknowledgement
I would like to extend my gratitude to my doctorate advisor, Dr. Rohini K. Srihari, for taking interest into my work and guiding me into the right direction. I would also like to thank my mentor assigned by ACL-SRW, Arkaitz Zubiaga, and anonymous reviewers for their invaluable suggestions and comments on this paper.