Modeling and analysis of leftover issues and release time planning in multi-release open source software using entropy based measure

In Open Source Software (OSS), users report different issues on issues tracking systems. Due to time constraint, it is not possible for developers to resolve all the issues in the current release. The leftover issues which are not addressed in the current release are added in the next release issue content. Fixing of issues result in code changes that can be quantified with a measure known as complexity of code changes or entropy. We have developed a 2-dimensional entropy based mathematical model to determine the leftover issues of different releases of five Apache open source products. A model for release time prediction using entropy is also proposed. This model maximizes the satisfaction level of user’s in terms of number of issues addressed.

INTRODUCTION
Release engineering focuses on qualitative and quantitative approach for developing a roadmap, which can shape the product that's ready for release. A systematic review for understanding the different approaches for release planning has been conducted by reviewing the research papers published in various academic journals and conferences [50]. In the OSS development model, users are categorized into various categories depending upon their skills and involvement in software development. In many cases, users also act as developers [48]. The contributors located at different geographical locations request for addressing NFs, IMPs and fixing the bugs through a centralized software platform [1 and 51]. In order to address the different issues reported by users and to fix the bugs the source code of the software need to be changed and these code changes are updated in source code vol 34 no 1 January 2019 33 repositories. These source code changes have been quantified using Shannon entropy [2, 3 and 12].
In every organization, the software development team always desires to produce a software release of high quality with a low fault content. To meet the enormous requirements, they also want to produce a software with frequent releases [14]. In literature, the release time problem for proprietary software has been discussed widely by considering one factor, the bugs which have been fixed in different releases. A release planning study has been conducted by interviewing the persons involved in release planning of the OSS and found that release planning based on the implementation of features requested by users faces various challenges and prefer for release planning based on a given interval of time [25].
In feature implementation based release planning, the practitioners observed that all the features are not taking uniform time in their implementation but some features take enormous time.
In OSS development and even in closed source development the developers/ release managers mainly prefer time based release as the feature based release planning has not been addressed adequately.
The OSS development has been carried by the volunteer contributors (active users) and has not taken into account the cost criteria [42,43].
To release the next version of the software one need to consider the number of addressed issues requested by the users. In OSS, active user's satisfaction can be measured in terms of number of the issues fixed. Different issues are characterized by different severity levels where severity of an issue shows the extent of its impact [44 and 52]. By considering the severity levels of different issues, we have calculated active user's satisfaction as number of issues fixed multiplied by severity levels. In order to increase the active user's satisfaction level, we have maximized this quantity.
In this paper, we propose models for a multi-release software product. We determine the release time by considering the fixing of NFs, IMPs, bugs and code changes. The models have been validated by using datasets of various products of Apache software project. We have taken into consideration the two existing software reliability growth models [37 and 38] for comparison. We observed that our models give highest cases of maximum performance.
Remaining part of the paper has seven sections. In section 2, we have summarized the related work. Data collection and model construction have been discussed in section 3. The experimental setup has been given in section 4. Section 5 documents the results and discussion. The optimal release planning has been discussed in section 6. Threats to validity have been given in section 7 and the conclusion of the paper has been given in section 8.

RELATED WORK
The monolithical software development offers a way for multiple releases as a result of functionality enhancements [21]. A better understanding of release process assists developers to design frequent and fault free software releases [14]. "A timely introduction of a clunker and a delayed entry of a masterpiece can destroy a product's chance of success" [27]. The uncertainty analysis has been carried out in different disciplines in order to measure the uncertainty arises due to internal or external factors. In software development too, the uncertainty analysis has been used by applying Shannon information theory in order to quantify frequent source code changes [2]. A study based on the quantified code changes has been conducted [3]. In this study, the authors also estimated the rate at which source code changes are taking place.
The quantified approach in understanding the different activities relating to software development and maintenance activities has been proven to be very useful. In literature, the search based software engineering approach has been used in discussing the next release problem [8 and 9]. The release time of a software can be affected by many factors like complexity, bugs, software architecture, software domains and tools [22, 23 and 24].
During the past three decades, various mathematical models have been proposed to quantify the quality of the software. Various release time planning approaches have been discussed in literature for commercial/proprietary software. These approaches have considered the fixing of bugs of current and just previous release [7]. In case of OSS, the release time strategy is different from the proprietary/closed source software in the sense that it considers not only the number of bugs fixed but the number of NFs and IMPs implemented. "When using a feature-based strategy, an open source project might make a release even if not all planned features have been implemented" [25].
For a single release problem in closed source software the authors proposed to determine the optimal release time by minimizing the development cost that include testing cost, debugging cost and fixing cost or maximizing the reliability level subject to predefined budget [4]. When the addons are included and the faults are removed a proportion of faults are generated, a release time problem by considering this phenomenon has been developed in [36]. An attempt has been made to develop multiple release time problem [7]. The approach determined the optimal time and the optimal amount of resources. But, again the next release has been decided on the basis of number of bugs removed in testing phase and operational phase. The Dempster-Shafer theory and differential evolution based multi software reliability allocation for a multimedia system has been proposed in [32]. An optimal software release time planning by considering delay incurred cost has been proposed in [33]. Multi-criteria based model has been proposed for the software reliability prediction [34]. In a study [35], the failure processes of testing have been investigated by considering the delay effect in fault fixing. Recently, a mathematical model for optimal time determination in multi-release software has been proposed in [53]. The authors considered different types of users, namely innovators and imitators in predicting the release time.
In this paper, next release planning has been proposed based on the predefined addressed features and bugs.
An economy outlook is presented by Cobb-Douglas function [6, 7 and 13].
In our work, we have used the Cobb-Douglas function to model the growth of fixing of different issues.

Data collection
We have taken data from five products, namely Avro, jUDDI, Hive, Pig and Whirr of Apache open source project [10 and 42] where bugs, NFs and IMPs have been presented with different signs as shown in Figure 1. The fault tracking data of Mozilla project for three successive versions Firefox 3.0, 3.5, and 3.6 (https: //bugzilla.mozilla.org/) [39] and for the product of gnomecontrol-center (https://bugzilla.gnome.org/) [40], for 4 successive versions from 2.0 to 2.3 have been selected [35]. The historical code change data has been extracted using GitHub tool [11]. Figure 2 shows the sample of different issue reports of Avro product. Figure 3 shows the screen shot to download code change history for Avro product from the GitHub repository.
The process of data collection for Apache project and Entropy calculation has been carried out using the methods and formulas as discussed and proposed in [2, 12 and 53].

Modeling for Multi-Release Software Product
In OSS, once the issues are reported, the triaging takes place and different issues are assigned to developers. The source code of different files gets changed during fixing of these issues and the product goes for next release. But, there are some issues which are still left in the current release, which get fixed in the next release. A mathematical model is necessary here, which will predict the leftover issues of a release to be fixed in the next releases. The class of time-domain software reliability models assumes that software failures display the behavior of a Non-Homogeneous Poisson Process (NHPP) [16][17][18][19][20]26].
Let Poisson probability mass function of a random variable, N(t), with parameter X (t) is defined as Various time domain models have appeared in the literature which describe the stochastic failure process by an NHPP [26]. 'a' : potential number of issues. Issues can have 1) bugs/NFs/IMPs, 2) NFs+IMPs and 3) bugs+NFs+IMPs. These issues in infiite time with finite failure NHPP can be written as with the following differential equation (2) as follows in [7 and 53].
Here, F(t) is a distribution function and f (t) = d dt F (t) is a density function. The quantity [a − X (t)] denotes the expected number of issues remaining in the software at time t.
Solving above equation at t = 0, X (0) = 0 we get the following Here, X (t) is the cumulative value of fixed bugs/new features/feature improvements / (new features + feature improvements) at any given time t.
During data collection, we observe that in the beginning, the cumulative number of issues fixed is slow and after that it increases and then stabilizes in subsequent releases. To model such behaviour of issue fixing, we proposed a model based on logistic function, i.e.
By using equation (4) in (3), we get Here, γ is a constant and depending upon its value, models can capture different types of issue fixing growth curves. The ϕ denotes an issue fixing rate per remaining issue.
Here, we consider that in equation  (5) is used to predict the potential number of bugs, new features, feature improvements, and (new features + feature improvements) at any given time for Release 1 data. Now, we consider that source code is changed to fix bugs, new features and feature improvements. The changes in the source code of the software can be quantified using entropy based measure [2, 3 and 12] To incorporate and consider the source code changes for fixing the issues, we extend the Cobb-Douglas type function [13]. We considered time and entropy both simultaneously instead of only time. In (6), s and u represent the time and entropy (the complexity of code changes) respectively. β is the code change elasticity to issues fixing time.
We can develop 2-dimensional model by using calendar time 's' and entropy 'u' which results in equation (7).  The above model can also be written in the following form and We used model given in equation (7) to predict the potential number of total issues at any given time for Release 1 data. We consider the same fixing rate of issues across different releases for the sake of simplicity.

Multi-Release Modeling based on Leftover Issues of just previous Release [53 and 54]
In software, different issues, namely bugs, new features and feature improvements are reported and get fixed in the current re-lease. The remaining unresolved issues which are leftover move to the next release. The mean value function of issues based on calendar time and entropy has been evaluated by using (5) and (7) respectively During our empirical investigation, we found that the issues, namely, new features and feature improvements are fixed in the current release and the unresolved are fixed in next release, means next release considers the leftover new features and feature improvements of the just released version of the software. But, in case of issues, namely bugs, leftover bugs of Release1 are fixed in Release 2, Release 3 and Release 4. It means, in an open source development environment leftover bugs of different releases are passed on to higher releases (up to next three-four releases). Here, we consider that leftover bugs of Release 1 can pass up to Release 4. Based on this empirical evidence, the different mean value functions for different releases have been modeled as follows: computer systems science & engineering We consider that in the first release different bugs are reported and get fixed are modeled by the following equation where a 1 is the potential bugs to be fixed in the first release at time t 1 . The leftover bugs of first release, i.e. a 1 (1 − F 1 (t 1 )) are added to the potential bugs of second release with fixing rate F 2 (t − t 1 ). Therefore, the mathematical equation representing the cumulative number of bugs fixed in the second release is given by 10) In above equation a 2 is the potential bugs to be fixed in the second release. In the line of modeling for the second release and along with taking into consideration the fact that the next release will contain the remaining bugs of all the previous releases, we can write the expressions for Release 3 and Release 4.

Multi-Release Modeling based on unresolved Bugs passed on to different Releases [36, 53 and 54]
If we consider that the next release consists of remaining issues of just previous release, then we can write the following expressions for different releases. The mathematical modeling for cumulative addressed issues for Release 1 is given as equation (13).
a 1 is the addressed potential issues of Release 1. The leftover issues, i.e. a 1 (1 − F 1 (t 1 )) with fixing rate F 2 (t − t 1 ) are added to the issue content of Release 2. Mathematical equations for cumulative addressed issues estimation in Release 2 and Release 3 are given in equation (14) and equation (15).
Similarly, we can write equation (16) for the n th release.

EXPERIMENTAL SETUP
In this section, we have discussed multi-release based on unresolved Bugs passed on to different releases and multi-release based on the Leftover Issues of just previous release.

Multi-Release based on unresolved Bugs passed on to different Releases
In an open source development environment, leftover bugs of different releases are passed on to the higher releases. We validated the proposed models given in equations (9), (10) and (11) for case 1 (in Table 1), on weekly bugs fixed data of three releases of Firefox and Gnome-control-center projects [35]. Results have been presented in Table 3 given in the next section.

Multi-Release based on the Leftover Issues of just previous Release
If we consider that the next release consists of the remaining issues of just previous release, then potential values of different issues (cases 1-5 of Table 1) in Release 1 can be estimated using equation (13) and the potential value of all the issues (case 6 of Table 1) can be estimated using model given in (7) as this model takes care of entropy. For Release 2 we have used model given in (14). For Release 3 we have used model given in (15). The generalized model for n th release given in equation (16) has been used to estimate parameters for rest of the further releases by considering just previous release leftover issues. For the first release, the leftover issues have been predicted by using the parameters estimation results of first release. Release 2 parameters are estimated by considering Release 1 leftover issues along with Release 2 dataset. For this we have used equation (14). With the resultant leftover issues, along with Release 3 dataset, we have estimated Release 3 parameters. For this we have used equation (15). For Release 4 and Release 5 parameters estimation we have followed the same process and used equation (16).
We have validated the proposed models discussed here for Apache open source products, namely Hive, Pig, Avro, jUDDI and Whirr [10]. We have used Nonlinear Regression (NLR) in Statistical Package for Social Sciences (SPSS) software to estimate the parameters (a, φ, β and α). Table 2 shows the values of different parameters we have used in NLR.

Parameter Estimation
Step Infinite method limit step size Value Sequential Quadratic Programming 2 1.00E+20 From the results, it has been observed that the proposed models representing all cases of Table 1 give a high goodness of fit. vol 34 no 1 January 2019

NUMERICAL EXAMPLES
Numerical application examples are given here for the illustration purpose for Firefox, Gnome-control-center and Apache software products. We have measured the performance of different estimation models for all the releases. Table 3 shows the parameter estimates of case 1 defined in Table  1, for Firefox and Gnome-control-center projects. Here, we consider that leftover bugs of Release 1 can pass upto Release 3. The number of fixed real bugs of i th release is shown by 'A i '. The potential number of bugs of i th release is shown by 'a i ' and the leftover bugs of i th release is shown by 'a i − A i '. The issues fixing efficiency for i th release is represented by the parameter φ i . γ i shows the variation in bug fixing pattern of i th release.

Multi-Release based on unresolved Bugs passed on to different Releases
In Table 3, we have documented the leftover bugs of each release, which are the addons to the fault content of the next releases. In case of Firefox, the estimation results show that 50 initial bugs are there in Release 1. But, fixed bugs are 48. Hence 50 − 48 = 2 bugs remained unresolved, which added to the next release fault content. We can draw similar inferences for other releases. For Release 2, 45 bugs have been fixed and the estimation also shows (a 2 + a 1 (1 − F 1 (t 1 ))) = 45 bugs, which means all the bugs have been detected and fixed in Release 2.
For Release 3, model shows under fit performance.
In case of Gnome-control-center, the estimation results show that 43 initial bugs are there in Release 1. But, fixed bugs are 42. Hence 43 − 42 = 1 bug remained unresolved, which added to the next release fault content. We can draw similar inferences for other releases. For Release 2, 35 bugs have been fixed and the estimation shows (a 2 +a 1 (1− F 1 (t 1 ))) = 95 bugs, which means 60 bugs remained unresolved and added to the third release fault content. Similarly, we observe that 220 leftover bugs added to next release from Release 3.
We observe that estimation models, exhibits good fit in terms of MSE, Bias, VAR, RMSPE and R 2 for all the releases.

Multi-Release based on the Leftover Issues of just previous Release
Tables 4 to 8 present parameter estimates of Case 1 to Case 4 ( Table 1) for different Apache datasets. Here, we considered that leftover issues of just previous release are added to the issue content of next release. The number of fixed real issues (bugs/IMPs/NFs/NFs+IMPs) of i th release is shown by 'A i '. The potential number of different issues of i th release is shown by 'a i ' and the leftover issues of i th release is shown by 'a i − A i '. The issues fixing efficiency for i th release is represented by the parameter φ i . γ i shows the variation in bug fixing pattern of i th release. γ i is the code change elasticity to issues fixing time for i th release.
In Table 4, we have documented the leftover bugs of each Avro release, which are the addons to the fault content of the next releases. The estimation results show that 149 initial bugs are there in Release 1. But, fixed bugs are 134. Hence 149 − 134 = 15 bugs remained unresolved, which added to the next Release 2 fault content. We can draw similar inferences for other releases. For Release 2, 81 bugs have been fixed and the estimation also shows (a 2 + a 1 (a − F 1 (t 1 ))) = 91 bugs, which means 10 bugs remained unresolved and added to the third release fault content. Similarly, we observe that 16   Similar, interpretations can be drawn for other products. By analyzing the above empirical results, we can evaluate the quality of maintainability and the adaptability of software in fixing of different issues. The number of issues left in different releases determines the release readiness and the stability of the software. In case of Apache datasets, we observed that for Avro product 61.5% of the potential issues get fixed before the next release. For Pig product except new features fixing in Release 4, all releases are following the same fixing level. For Hive product, all the issues fixing process achieved 61.5% performance. We observe that for jUDDI product issues fixing process for different issues, namely bugs, NFs and IMPs follow the same pattern except bug fixing for Release 4 and total issues fixing for Release 2.
The releases of Whirr product follow the same pattern except improvements and new features fixing in fourth release. Results show that 61.5% of potential issues of current release get fixed before the next release except the four cases in all five products. All the estimation models (Table 1)

Comparison of Performance for the proposed models
The proposed issue estimation models (Case 5 and Case 6 in Table 1) have been compared with Goel-Okumoto model [37] (equation (17)) and Yamada delayed S-shaped model [38] (equation (18)).
X i (t) denotes the i th release cumulative number of fixed errors at time t. 'a i ' is the potential number of errors. 'b i ' is the fixing rate of errors for i th release. Tables 9 to 13 present the results of these two models in comparison of proposed models. For Avro product (Table 9), the proposed models (Case 5 and Case 6 in Table 1) estimation results show that 332 issues (bugs+NFs+IMPs) have been fixed in Release 1. The estimated potential value of issues is 357. This means 25 issues which would have been fixed in Release 1 now will be fixed in Release 2. Similarly, 14 issues which would have been fixed in Release 4 will be fixed in Release 5. Similar, interpretations can be drawn for other products. Out of total 23 releases, in 20 releases proposed models give better R 2 than the two existing models.
We designed the experiment to test the statistical significance of the proposed model as discussed in case 6 in Table 1. The statistical significance has been validated using non-parametric Kolmogorov-Smirnov (K-S) test. The P-values of the experiment have been given in Table 14.
We have taken level of significance α = 0.025. We observed that the proposed model is statistically significant.

OPTIMAL RELEASE PLANNING
In closed source software, due to time and resource constraints it is not possible to address/fix all the issues in the current release vol 34 no 1 January 2019 and in OSS due to active user's demands. By putting a constraint on minimum number of fixed issues with different severity levels before the next release of the software, we can maximize the active user's satisfaction. The following objective function considers two different severity groups, S n1 (average severity for n th release issues) and S n2 (average severity for (n − 1) th and n th release issues which will be addressed in (n + 1) th release).
Maximize S n (t) = S n1 X n (t) + S n2 × a n + a n−1 (1 − F n−1 (t n−1 )) − (X n (t)) (19) In IEEE 982.2 defect indices definition [41] 10 weight has been used for high severity issues, 3 weight for medium severity issues and 1 for low severity issues. We have also used the same weights for different severity issues as given in [41]. By using (7) and (8), we can write the following function to consider active user's satisfaction.
We have solved the above nonlinear problem by using Genetic Algorithm (GA) [28].
To calculate optimal release time of n th release, estimated parameters of n th and (n−1) th releases have been used. We have solved equation (20) in MATLAB with an optimization tool by using solver "ga-Genetic Algorithm". Table 15 shows different parameter values to obtain optimal solution. We have taken Heuristic crossover function and Tournament selection function with Eliete count 2.
Similarly, we have observed that the estimated optimal release time for jUDDI third release is 19 months with 92% active user's satisfaction. The real release time we have observed is 14 months. The estimated optimal release time is close to the real release time. For Hive and Pig, we have estimated optimal release time of 14 months and 10 months for fourth releases at 98% and 98.5% user's satisfaction levels respectively. In case of Hive, the real release time is 13 months and in case of Pig it is 9 months. This shows estimated optimal release time is close to the real release time.
In case of Whirr product, we have estimated optimal release time of 10 months for third release at 97% user's satisfaction level which is 2 months more than the real release time (8 months).
We observed that the release time problem based on the complexity of code change metric is more practical and gives a close prediction.

THREATS TO VALIDITY
Factors affecting the validity of proposed work are as follows: Internal Validity: Some of the assumptions made by us may not always reflect the reality (e.g. leftover issues of n th release may not have a larger severity when they are added in the initial issue content of (n +1) th release). Moreover, we have not empirically validated the real number of leftover issues.

Construct Validity:
We used the information available in GitHub repository, such as code change history for calculating entropy. This information we calculated manually, and may contain some manual errors. The total code changes in files resulting from fixing of different issues has been considered in our work. Statement level code changes need to be considered instead of file level changes. We have taken some versions collectively (versions have less data points) as a release. The selection of the releases is based on the equal data points for every release. We do not claim any causal findings for other choices of releases.

CONCLUSION
We estimated the potential value of different issues based on the time and the complexity of code changes (entropy) in different releases of Apache open source products. The issues left unresolved in different releases have also been calculated. Results show that in all the five products out of total 23 releases in 19 releases, at least 61.5% of potential issues have been addressed before the next release of the software. The leftover issue content contributes in the upcoming releases. The proposed model (case 6 of Table 1) performance has been compared with two models (Goel-Okumoto model and Yamada delayed S-shaped model). Out of 23 releases for 20 releases, we observed that the proposed entropy based model results in maximum cases of maximum R 2 in comparison of these two models.
A model for release time prediction using entropy is also proposed. We optimized the objective function of release time problem using Genetic algorithm in MATLAB. The estimated optimal release time is close to the real release time of jUDDI, Whirr, Hive, Pig and Avro products at 92, 97, 98, 98.5 and 99.7% satisfaction levels.


Phenotypic Profiling of High Throughput Imaging Screens with Generic Deep Convolutional Features

While deep learning has seen many recent applications to drug discovery, most have focused on predicting activity or toxicity directly from chemical structure. Phenotypic changes exhibited in cellular images are also indications of the mechanism of action (MoA) of chemical compounds. In this paper, we show how pre-trained convolutional image features can be used to assist scientists in discovering interesting chemical clusters for further investigation. Our method reduces the dimensionality of raw fluorescent stained images from a high throughput imaging (HTI) screen, producing an embedding space that groups together images with similar cellular phenotypes. Running standard unsupervised clustering on this embedding space yields a set of distinct phenotypic clusters. This allows scientists to further select and focus on interesting clusters for downstream analyses. We validate the consistency of our embedding space qualitatively with t-sne visualizations, and quantitatively by measuring embedding variance among images that are known to be similar. Results suggested the usefulness of our proposed workflow using deep learning and clustering and it can lead to robust HTI screening and compound triage.

Introduction
Modern drug discovery is a rapidly evolving field with the use of modern AI technology. Nonetheless, it remains expensive and time consuming for the introduction of any new medicine (almost $3 billion, >10 years, [3]). Rather than hand-designing and testing novel drugs individually, the modern pharmaceutical approach is to use a library of compounds (e.g. 2 million compounds), and filter them with a sequence of complex imaging and biochemical tests. Typically for a high throughput screen, a single dose experiment is designed to remove vast majority of irrelevant compounds, e.g. by 100 folds.
Following the spectacular rise of deep learning techniques in computer vision, natural language processing and numerous scientific applications in recent years, deep learning has increasingly been applied to the field of chemoinformatics [2]. However, most recent work (e.g. [1]) has focused on predicting the biological effects of chemical compounds directly from chemical structure representations such as SMILES strings [9]. Morphological profiling is a complementary approach that can be used to predict a broad range of biological effects [4]. In this approach, candidate drugs are applied to cell cultures and imaged with high throughput fluorescence microscopy; depending on the bioactivity of the drugs, this can cause a variety of morphological changes to occur, yielding clues as to what effect a compound has on the cells. Indeed, [4] shows that a single network can be transferred to predict the outcomes of many other targeted assays.
Despite the rich information provided by morphological profiling, it is sometimes unknown which exact morphological phenotypes one should screen for. An example of this scenario is a high throughput screen conducted by AstraZeneca, in which novel compounds are screened for inhibition of IDOL -Inducible Degrader of the Low density lipoprotein receptor (LDLR) [5]. In this screen, HEK293S human embryonic kidney cells were engineered to express both Low Density Lipoprotein Receptor -Green Fluorescence Protein (LDLR-GFP) and IDOL. Since IDOL degrades LDLR, the presence of green fluorescence is an indicator of IDOL inhibition. However, due to the complex and poorly understood interactions between novel compounds and human cells, presence of the GFP signal is a necessary but not sufficient condition to infer IDOL inhibition -the actual phenotypic appearance of genuine hits is not known at the screening stage. In situations like this, the expert knowledge and intuition of a biologist is required to identify phenotypes that are indicative of genuine hits. Due to the high volume of images in a HTI screen, this cannot be done manually for each individual image, and due to the unknown nature of the target phenotype, supervised learning is not applicable.
In this paper, we propose a novel procedure for computing feature vectors for cellular images using a pretrained convolutional neural network (CNN). The resulting feature vector space can then be partitioned by unsupervised clustering, allowing us to decompose a HTI screen into a small set of visually distinct phenotypes. The expert judgement of biologists can then  Figure 1. Our feature extraction pipeline. We feed our images through a pre-trained VGG16 network, truncated before the fully connected layers, and concatenate the spatial means of three intermediate convolutional layer activations. This yields a vector of multi-scale convolutional features, which we later embed in 2D space via t-sne (see Figure 5). be applied to whole phenotype clusters rather than individual images, allowing a HTI screen to be filtered rapidly for interesting compounds. The feature vectors can also be embedded in 2D space for visualization using dimensionality reduction techniques such as t-sne [6], providing a visual summary of the phenotypic distribution.
In Section 2 we describe our feature extraction pipeline. Section 3 discusses clustering and in Section 4 we demonstrate and evaluate our approach on a high throughput imaging dataset (IDOL).

Feature Extraction
CNNs trained on large datasets such as ImageNet have been found to learn a hierarchy of features, with early layers learning general, task-agnostic features pertaining to texture and shape primitives, and later layers learning more task specific features [10]. Despite the obvious differences between ImageNet images (which are generally photographs) and fluorescence micrographs, the early convolutional layers of a CNN trained on ImageNet are general enough to respond to the differences in shape, colour and texture in fluorescent labeled cellular images.
Our feature extraction begins by computing feature maps for a cellular image, by feeding it through a pretrained CNN (Figure 1). The CNN architecture we use is VGG16 [8], pre-trained on ImageNet. Rather than taking final fully-connected layer activations as our feature vector, we extract our features by mean pooling the (pre-activation) feature maps of three early convolutional layers, and concatenating the means to produce a feature vector of length 1024 (one component for each feature map in the chosen layers). Extracting features from early convolutional layers rather than final fully connected layers has a number of advantages for large images in which the objects of interest are Firstly, high level representations learned by CNNs contain information that is immediately relevant for identifying the classes they are trained to recognize. In the case of ImageNet, these are everyday objects such vehicles and animals. High level features are unlikely to be very descriptive for cellular images, which differ substantially from the ImageNet images and training classes, but lower level features are still general enough to capture information about shape, texture and colour.
Secondly, because of the fixed weight matrix connecting the first fully connected layer to the last convolutional layer, fully connected layers require the input image to be of a fixed size. By using only the convolutional layers of the network, we avoid the need to downsample our images to 224 × 224 (for the VGG network), which would discard valuable high frequency information.
Thirdly, different layers capture information at different scales. Higher level layers have larger receptive fields, and describe patterns of greater size and complexity, but successively discard the higher frequency information captured by lower layers. By extracting features from the fourth, seventh and ninth convolutional layers, we obtain a multi-scale representation (see Figure 2).

Clustering
To discover distinct phenotypes in the dataset, we perform k-means clustering in feature space. The optimal choice for the number of clusters k is a trade-off between making the clusters as homogeneous as possible, and keeping their number low. Figure 3 shows the mean intra-cluster embedding variance as a function of k; since we observe diminishing returns past k = 70, we choose 70 as the optimal number of clusters.

Results
The most crucial aspect to validate is that our feature extraction does indeed embed similar images at similar points in the feature space. We can evaluate this quantitatively, by measuring the variance (concretely, mean squared distance to centroid) of different groups of image embeddings. The compounds tested in this dataset are chemically clustered by AstraZeneca in Extended-Connectivity Fingerprint embedding space [7], resulting in 711 chemical clusters. We would expect images corresponding to compounds from the same chemical cluster to have more tightly clustered feature vectors than the dataset as a whole, because similar compounds may lead to similar morphological changes in images. As expected, the mean intra-cluster feature variance is 66.5% that of the dataset as a whole. Furthermore, we would expect different images captured from the same well on an assay plate to be clustered more tightly still, because these images all correspond to the same compound. We observe the mean intrawell variance (4 images were captured per well) to be 4.0% that of the full dataset.
To validate the quality of our clusters, we display samples from six phenotypic clusters in Figure 4. We see that k-means has identified relatively homogeneous clusters, further validating the quality of our embeddings. Figure 5 shows a t-sne visualization of our entire dataset, with phenotypic clusters detected by k-means labelled by colour and annotated with representative samples from that cluster. To speed up the t-sne process, we used Principal Component Analysis (PCA) to reduce the dimensionality of our features from 1024 to 10. These 10 principal components explain > 90% of the variance in the embedding space. Clusters 4 and  Figure 5 show genuine GFP expression, while the others are judged as uninteresting by the biologists, and can be discarded, resulting in a 47-fold reduction. Using the full 70 clusters would result in more precise filtering and greater reduction still.

Conclusion
We have developed a novel workflow for high throughput screening. In this workflow, images were represented as deep learning feature vectors from a pretrained convolutional neural network. This was followed by the clustering of images with similar image phenotypes. This facilitates scientists to select interesting clusters for downstream screening in an attempt to find hit compounds. Because it uses generic convolutional features extracted from a pre-trained convolutional neural network, our method requires no training and can be applied to any cellular screen dataset without hyperparameter tuning -a significant saving in time. Our visualizations allow scientists to quickly assess the distribution of cellular morphologies in a high throughput imaging screen, or within a smaller subset of compounds, such as a chemical cluster. Meanwhile, our proposed workflow allows scientists to select interesting/promising phenotypes and quickly retrieve chemical clusters that show a high prevalence of said phenotypes.
Future work could gain further insight into the biological processes at work by investigating the relationship between our morphological embeddings and the ECFP embeddings of the chemicals that produced them, perhaps by predicting ECFP embeddings from morphological embeddings using supervised learning. Our techniques could also be applied to other imaging modalities, such as tissue pathology and mass spectrometry imaging, with minimal modification needed.


Comparison of Self-monitoring Feedback Data from Electronic Food and Nutrition Tracking Tools

Changing dietary habits and keeping food diary encourages fewer calorie consumption, and thus weight loss. Studies have shown that people who keep food diary are more successful in losing weight and keeping it off. However, no study has investigated the nutritional values produced by food journaling applications. This is crucial since keeping food diaries helps identify areas where changes needed to help user's loss weight, based on the application feedback. To achieve this, the provided data should be consistent among all applications. Otherwise, this will question the effectiveness and reliability of such tools in tracking diet and weight loss, and hence question user trust in these applications. This study characterizes the use of 4 food journaling applications to track user diet for 10 days (namely, MyFitnessPal, Lose It, FatSecret, CRONOMeter). We measured variations between the output of each application. The findings revealed an inconsistent and a variation in the output feedback given by all the 4 tools. Although some tools provided closer values, still their data were different and inconsistent. Moreover, some tools were missing essential nutritional fact data, such as sugar and fiber. We additionally compared a sample of food items common among all tools with the Swiss Food Composition Database and checked for their consistency with the same items in the database. The evaluation of the applications showed a gap in the data consistency among applications and the FCD, and questions how reliable they are for food logging and diet tracking. This study contributes to current research in health and wellbeing and can be referenced by researchers to provide deeper insights into the data consistency. Future work should examine ways to provide precise output that is common among all applications, so to guarantee the effect on weight loss.

Introduction
A strong correlation exists between dietary habits and intermediate health risk factors, such as diabetes, blood pressure and cholesterol [1]. Diet plays a crucial role in the development and prevention of cardiovascular disease; it's a key changing element that impacts all healthy risk factors. Therefore, to achieve positive impact, a balanced and mindful approach of overall dietary pattern is required. Keeping daily food diary to track diet, lose weight or build healthy habits with a paper or an app and other forms of record have shown to positively impact weight lost, diet management, controlling portion size and sticking to healthy habits [2]. The act of meal recording helps think about the meal, the time, and the portion size [3]. A study by Hollis et al., [4] concluded that people who kept a food diary were more likely to keep the weight they lost off, in another study by Burke et al., [5] found that people who kept a record of their meals and kept up with diet and exercise lost nearly twice as much as people who didn't keep a log (but did diet and exercise). A combined emphasis on dietary intake and physical activity is important to short-and long-term weight loss goals. There exists abundant list of mobile and web apps intended for daily diary creating and food journaling. These tools provide insight about caloric intake, the amount of nutritional values, and other dietary related data. The majority of these tools ask users to log their dietary intake, including item name, quantity and meal time. These applications are still tedious in terms of data entry and provide limited automation to detect and fill-in the data. In addition, the main issue with such tools is the nutritional facts output data. Such data have to be precise and consistent among all applications. For example, the caloric and amount of ingredient should be consistent along all the applications. In this research we describe data variation and output of eating quantity by each tool. We focus on the amount of caloric data provided by each tool (namely, MyFitnessPal 1 , Lose It! 2 , FatSecret 3 and CRON-O-Meter 4 ) together with other nutritional values (namely, protein, sugar, fiber, fat, carbs) and compare them to the Swiss food composition database 5 , then compare the final result in terms of portion size. We highlight data consistency given by each food journaling application. We begin by comparing their values and check them in terms of food weight and portion size. For that, we conducted an experimentation with 6 users and the 4 food journaling tools to track their food logging for 10 days. The user demographic consisted of 5 male and 1 female, all attending university level study. The findings revealed variation in output among all tools and with the Swiss database. We tracked daily meal, including breakfast, lunch, dinner and snacks. After data collection, we analysed the data by two reviewers. Missing or inconsistent values in one or more application were eliminated from the study. We analysed associated food list and nutrient values and compared them in terms of size and value similarity. Finally, a random list of 13 food items were selected and compared with the Swiss database in terms of portion size. There was a variation in the value provided by the tools and the database with respect to portion size. Although the importance of diet tracking for overall health and wellness, to our knowledge no study has examined and validated the output provided by food journaling application and compared their various nutritional values. We envisage this study to contribute to the existing body of research in the domain of diet management, weight loss and preventive healthcare.

Electronic Diet Tracking
Self-monitoring via mobile apps is a recent addition to self-regulation methods for weight loss. Self-monitoring exercise proven relevant to weight loss [6]. A study by Tsai et al., [6] describes Patient-Centered Assessment and Counselling Mobile Energy Balance (PmEB) application to self-monitor caloric balance in real time. The study concluded that PmEB is feasible for self-monitoring and weight management. A study by Noronha et al., [7] introduced PlateMate, an application that allows users take picture of their meal and receive estimates of nutritional facts. The study pointed the importance of data accuracy provided by such tools to help monitor dietary goals. However, current methods through self-reporting, expert observation or algorithmic analysis are time-consuming. Results showed that PlateMate is nearly as accurate as dietitian and easier to use by users. Yet, the data provided by mobile application often doesn't adhere to evidence-based content. Breton et al., [8] summarised the content of available weight control apps. Information on content, user ratings, and price were extracted. The findings showed many apps lack evidence-based content and suggested further research work. Tracking diet or physical activity is a social process and requires collaborative process rather than personal. Different tracking styles exist, including goal driven and documentary tracking. Rooksby et al., [9] characterised activity trackers as "lived informatics". This is contrasting with other discussions on personal informatics and quantified-self. The study interviewed users who track activity and found people interweave various activity trackers, sometimes with the same functionality. Physical activity and diet monitoring are key for weight loss programs. The purpose of a study by Turner et al., [10] was to assess the relationship between diet (mobile app, website, or paper journal) and physical activity (mobile app vs no mobile app) monitoring. The findings pointed to the potential benefits of mobile monitoring methods during the weight loss trials. The study suggested ways to predict which self-monitoring method works better [10]. Current data generated by apps are used as incentives to improve health and lower associated cost. To help improve patient's health and fitness, data can be collected from smartphone's built-in tools, such as Global Positioning System (GPS) tracking, accelerometer, and pedometer to measure health and fitness parameters. These apps then analyse the data and summarize it, provide frequent feedbacks as well as devise individualised plans based on user goals [11]. Health telematics is a growing up issue that is becoming a major improvement on patient lives, especially in elderly. M-Health solutions address emerging problems on health services, including, the increasing number of chronic diseases related to lifestyle, high costs of care. Silva et al., [12] presented a comprehensive review of the state on m-Health services and applications. Open and challenging issues on emerging m-Health solutions were proposed for further works. Mobile devices offer media rich and context aware features that are useful for eHealth applications. A paper by Liu et al., [13] examined apps from App Store and provided a status of mHealth apps and an analysis of related technology. The study serves as a reference point and guide for developers and practitioners interested in using iOS as a platform for m-health applications, particular from the technical point of view. mobile apps offer the potential to objectively monitor user's eating and activity behaviours and encourage healthier lifestyles. In this regard, BALANCE [14] is an app for long-term wellness management. The system automatically detects caloric expenditure via sensor data from a Mobile Sensing Platform unit. Diet tracking is linked to improved weight loss success. Mobile apps could allow for improved dietary adherence [15] and manage various aspects of diseases, such as diabetes through consistent self-monitoring blood glucose (SMBG) [18]. Moreover, food journaling is an effective way to regulate excessive food intake. However, manual food journaling is burdensome, and crowd-assisted food journaling has been explored to ease this process. Rabbi et al., [16] proposed an approach to label food images with only high performing labelers, since they provide good quality labels. The study provided a machine learning algorithm to automatically identify high performing crowd-labelers from a dataset of 3925 images collected over 5 months. Self-logging is a critical component to many health applications. However, sustaining the data logging is a tedious process. A work by Bentley et al., [17] demonstrated passive mobile notifications to increase food intake logging. Adding notifications increased the frequency of logging from 12% in a one-month, ten-user pilot study without reminders to 63% in the full 60-user study with reminders included. We considered 4 food journaling and nutrition tracking tools. These tools were selected after measuring factors related to their data collection, and the possibility to download and access these data. Moreover, we paid attention to the caloric values returned by these apps. We checked for the sentiment analysis features, namely perceived positiveness, addictiveness, crashes and negativeness of the selected apps as provided by AppTrace 6 . Below we list and define each point.
Positive: Reviews that love the app or a feature, or generally find it fun, great, perfect, epic, etc. People recommending others to buy/download it.
Addictiveness: Reviews talking about an app being enjoyable, useful, addictive, entertaining, fun, easy to use. People saying how they keep using an app or can't stop playing it.
Crash: Reviews talking about the app or a feature crashing, freezing, not loading or working properly or needing bug-fixing.
Negative: Reviews finding the app terrible, boring, unusable etc. People talking about deleting, uninstalling or even hating the app or a version; recommendations to others to not buy / download / upgrade.

MyFitnessPal
This mobile & web application tracks diet and physical activity to determine optimal caloric intake and nutrition for the user. This tool uses gamification elements to motivate 6 apptrace.com/ • • • • user engagement during food logging. MyFitnessPal provides motivation and community empowerment to promote healthy living. The app tracks user's breakfast, lunch dinner and snacks. It provides comprehensive nutritional values about each food item, in addition to providing total amount of calories per nutrient value consumed. The tool provides nutritional data about calories consumed, carbs, fat, protein, cholesterol, sodium, sugar and fiber contained in each item. In Table-1 we list the sentiment analysis features of MyFitnessPal together with their rankings as provided by AppTrace.

Lose It!
This app allows users to take picture of food and identify the food to get caloric count and nutritional information in a single snap. This app uses goal-setting techniques to help users achieve their weight loss, exercise, water intake and other health related activities. The application provides peer competition or joining a public challenge towards a common goal. The weight loss community is to provide inspiration, guidance, support and challenges. The tool tracks user daily meal, including also snacks and exercises. In addition, it provides a plan to achieve optimal goal. We obtained data about fat, cholesterol, sodium, carbohydrates, and protein from this app. In Table-2 we list the sentiment analysis features of Lose It! together with their rankings as provided by AppTrace.

FatSecret
This app provides the possibility to track user daily meal, including snacks. The app uses community empowerment and focuses on speeding up food logging process. It displays fat, carbs, protein and calories about each food item, it also provides nutritional facts related to serving size and amount. In Table-3 we list the sentiment analysis features of FatSecret together with their rankings as provided by AppTrace.

CRON-O-Meter
This app allows users to add food items, exercise and provides a summary of consumed calories and daily caloric limits based on user's BMI. The tool doesn't provide a way to insert periodic meals (e.g., breakfast, lunch, dinner). However, it provides great insights about 60+ micronutrients contained in each food item. This includes vitamins (e.g., Vitamin A, Vitamin B, Vitamin C), minerals (e.g., Calcium, Copper, Iron), lipids (e.g., Fat and Cholesterol), protein (e.g., Cystine, Histidine, Isoleucine) and carbs (e.g., Fiber and Sugar). The app synchs user nutritional data and activity with Apple's Health app. There was no sentiment analysis feature data about CRON-O-Meter. In Figure-1 we provide a pipeline overview for the process of user-application interaction to acquire the nutritional data. In Table-4 we highlight the food journaling tools considered in this study.

Research Analysis
This consists of the analysis steps undertaken to gather the data manually by users and save them in a file, then insert the data into each of the 4 food journaling applications.  After obtaining the result from the tools, search the data within the Swiss Food Composition Database and compare their output with the output obtained from the tools. Finally, we perform an analysis on the data significance and portion size estimation (see Figure-2 for the detailed data analysis followed to obtain the final result).

Manual Data Collection
This step consisted of manually recording the dietary intakes by users for 10 days. The recording included brunch and snacks data during the day. The data was recorded in an Excel file by each user and included also data collected during the weekend. User's data was anonymized, however we kept track of their demographic information. Next, the data was prepared to be inserted into each food journaling application.

Food Journaling Data Analysis
After obtaining the data, we began analysing them with the food journaling applications. The data collection included all daily meals, snacks and brunches consumed throughout the day. The analysis focused on gathering data about carbs, protein, sugar, fat and fiber. The same food item was inserted in all apps and checked for the above nutritional values. The data for the same food item were checked and recorded in a separate file. However, some food items didn't appear and were not provided by some tools. For example, FatSecret had no data about fiber and sugar contained in the food items. Therefore, such food items were omitted from the tables. Moreover, Lose It! had no caloric data given for all food item, hence it was omitted from the caloric table. We considered the above nutritional information because they're greatly associated with healthy lifestyle and chronic conditions. For example, abnormal blood lipid (fat) levels have a strong correlation with coronary artery disease, heart attack and coronary death [19]. While high blood pressure (hypertension) is a major risk factor for cardiovascular disease, and it is associated with high sodium consumption [19]. An attention was given to the consistency in details of the food items. If an item was available in less than two tools, it was eliminated from the study. That way, we obtained a data consistent among all tools and eliminated data bias and inconsistency. Having all food items in Italian, the only available food composition database supporting also Italian food items was the Swiss food composition database. Hence, all food items were searched in the Swiss database and recorded together with their nutritional information to be used later in the data analysis and comparison.

Data Comparison
This step consisted of studying patterns among the output of food journaling applications. This includes getting users data for 10 days, then inserting the data into the 4 food journaling applications. We compared the output data for all the 4 applications. For the sake of consistency, we present the data for 3 users and 3 days per user, which is enough to validate and prove our hypothesis. Below we discuss the data comparison for the users. We checked the output for carbs, protein, sugar, fat and fiber for each food item among all the tools

User #1
The data analysis revealed incompatibility in the output for all the applications. The amount of carbs output was different in all the applications (see Table-5). For example, the carbs outputted by MyFitnessPal and Lose It! are inconsistent and have different output value for the same item. The carbs in MyFitnessPal is 188 g, whereas it's 116.2 g in Lose It! for the same day. Moreover, the data output by MyFitnessPal and CRON-O-Meter are closer in terms of quantity. Finally, some food items found to be common among some applications than others.
By checking the amount of sugar in Table-6, we obtained the same inconsistency among the data produced. For example, the amount of sugar in MyFitnessPal is 34 g, whereas its 29.8 g in Lose It! for the same day. Moreover, the sugar value was missing from FatSecret. In Table-7, we see the protein amount variation among all tools and there is a clear inconsistency in the output that shows the variation in protein content for the same food item. For example, by looking at the day-1 data, we see 74 g, 54.6 g, 62.62 g, and 68.5 g calculated by MyFitnessPal, Lose It!, FatSecret and CRON-O-Meter, respectively. The same result was obtained about fat and fiber as seen in Table-8 and Table-9. Finally, the amount of fiber was not provided by FatSecret.

User #2
The output data is similar to user #1. That said, food items have different outputs by each application. The amount of carbs (see Table- The protein amount in Table-12 distributed over 3 days is inconsistent and varies among all tools. For example, the portion amount for day-1 was 42.7 g and 28.13 g by Lose It! and FatSecret, respectively. The same data inconsistency was found in the fat and fiber data, shown in Table-13 and Table-14. Finally, the amount of fiber was not provided by FatSecret.

User #3
Similar to the above discussed data, there is also inconsistency in all output data for user #3. For example, looking at the value outputted by all tools, we see in the carbs outputted by Lose It! and FatSecret is 160.3 g and 197.44 g, respectively. In Table-16, the sugar value for Lose It! is 47.6 g and is missing from FatSecret. In Table-17 and  Table-      for the protein, and 19.1 g and 30.54 g for the fat obtained from the same food item. Whereas in Table-19 the amount of fiber for Lose It! was 11.5 g and was missing in FatSecret.

Overall Caloric Output
Meal tracking is essential to promote healthy lifestyle. A study suggested that when combined with healthy diet and exercise plan, noting down what we eat can double weight loss [20]. However, getting inconsistent data can lead to further dietary related complications and mislead users to think they consumed more or less of an item, which could be biased. With all data variation given by the apps, this clearly leads to inconsistency in the overall caloric output provided for users as proof of their daily dietary consumption. The findings revealed different caloric outputs for all subjects. Some of the differences was big, for example, the the caloric output for user #1 by MyFitnessPal and CRON-O-Meter was 1524 cal and 1219 cal, respectively for the same day (See Table-

Data Variation
To measure the variation between the tools, we have converted the output from calories into grams to later measure the portion size for each user and the food items. The conversion revealed that among all users, some tools had closer values than others. This could be either due to similarity in their food database or food items and data calculation. However other apps provided completely different values (see below Tables).

Data Conversion
We followed the standard conversion methods to convert between calories, grams and kilojoules. For that, 1 kilogram = 7716.17917647 cal, or 1000 g, when converting between calories and grams (see Table-26).

Data Variation between Tools
After the conversion from calories to grams, we have calculated the data variation between the tools and listed them in a matrix format. The Tables below list the data variation between tools for all users per day.

Comparing Nutritional Data of Tools with The Swiss FCD
After obtaining the overall nutritional data from each tool and calculating their caloric output, in this step we compare their nutritional facts output (namely, protein, sugar, fiber, fat, carbs) with the data provided by the Swiss Food Composition Database. The goal is to measure their similarity/differences with the Swiss FCD, with respect to nutritional facts and potion size.

Data Significance & Portion Size Variation
To calculate the similarities/differences between these tools and compare them with the Swiss FCD, we selected a sample of 13 food items that were fully available and defined by each tool. We selected food items that are rich in one of the nutritional facts we targeted in this study, namely, protein, sugar, fiber, fat, carbs, and dairy. We listed the items with their nutrients and the caloric output provided by each tool. We then searched for these items in the Swiss FCD and inserted it into the table together with the nutrients and caloric outputs given by the Swiss FCD. The measuring unit was for 1 portion size and the matrix unit was either per 100g edible portion for solid food items, or per 100ml food volume for liquid food items (see Tables-Data Sample). Although the result showed patterns of similarity between food items among the tools or tools and the Swiss FCD, however, there was still some other variations among the tools and with the Swiss FCD. There is a need to understand the reason behind this variation, whether it's the result of a calculation, or the variation coexisted even in the FCDs used by these tools. We consider this as a future work and highlight room for more research to be carried out on food journaling tools and food composition databases.

Data Significance
After listing the 13 food items, we checked for the significances of their data in terms of grams. For that, we converted the caloric output for each tool and the Swiss DB into grams. We followed a standard conversion equation, that's 1 gram contains 7.71617917647 calories, and 1 kilogram is equal to 7716.17917647 calories, or 1000 grams. We list our result in the below Tables representing the caloric output in grams for the 13 food items selected. We represented the data in matrix form.           FatSecret

Findings
Food journaling is a way to evaluate eating habits. Combined with goal setting and group therapy sessions, food journaling emphasizes the importance of monitoring and modifying daily behaviour. In addition, diet and food tracking have an impact on weight loss and chronic disease prevention. Dietary related behaviours have an effect on user's lifestyle and overall health condition. However, the way existing food journaling tools track diet isn't sufficient, the data provided by these tools aren't validated and consistent. Getting different feedbacks among the apps can provide wrong information and lead to complications. The conducted experimentation with 6 users (5 male, 1 female) showed differences in the data output by each tool. Moreover, some food items were missing from some apps. Although only 4 apps used in the measure, which is a small subset of the available tools, the obtained result revealed some inconsistent and missing data among these tools. Moreover, the data obtained from the tools and compared with the Swiss FCD showed some inconsistency, although the similarity in terms of values among some tools. The overall caloric value outputted varies greatly among the tools and the food database. This questions the reliability and effectiveness of these tools, especially when intended for diet and weight loss tracking.

Discussion
The usability and applicability of food journaling application for diet and weight loss management have been largely investigated [22]. However, to our knowledge, this study is the first to compare output of food journaling tools and measure their consistency and compatibility. We performed in-depth exploration on the output data of such tools and the correlation between the data for the same food item. The data collection lasted for 10 days and we had a small number of users, this however was sufficient to validate our hypothesis. Participants used their own smartphone, which allowed them to be familiar with the application. Our results are applicable outside the research setting and to measure the efficacy of similar apps in different domains. Future direction should consider discussing reasons behind the variation in the output among these tools and create a consistent data where all tools share the same value for the same item. We should investigate why this variation occurs and if there is a correlation between the food database used and the unit calculation applied. In addition, there is a need to investigate where the error occurs, whether its on user side while food logging and estimating portion, on the app side while calculating or its generated by the food database.

Conclusion
With the advancement of technology, methods for monitoring food logging and other food related parameters have become simpler and more convenient. This paper characterized food journaling tools to track user diet for 10 days. The aim was to measure if tools provide consistent output. the study also focused on comparing this output with the Swiss FCD. We explored interesting relationships among dietary related data provided by these food journaling apps. The results indicate some similarity between the values among some tools, but the data was inconsistent and there was a variation in the output feedback. In addition, there was another inconsistency arose when we compared these data with the data in Swiss FCD. Some tools missed essential nutritional fact data, such as sugar and fiber. Based on the evaluation of the tools, there is a gap in the data they provided, and this questions their effectiveness to follow for food logging and diet tracking. Research is needed to explore ways to provide people with a self-monitoring method fitting their preferences. Overall, we pointed to some data related issues of electronic methods for self-monitoring of diet. Finally, ways to predict which self-monitoring method works best for an individual are needed. We envisage this study to contribute to the current research in mobile apps for mHealth and fitness and be used as a reference to understand the current positions for improvements in this domain.