Impact of Minutiae Errors in Latent Fingerprint Identification: Assessment and Prediction

We study the impact of minutiae errors in the performance of latent fingerprint identification systems. We perform several experiments in which we remove ground-truth minutiae from latent fingerprints and evaluate the effects on matching score and rank-n identification using two different matchers and the popular NIST SD27 dataset. We observe how missing even one minutia from a fingerprint can have a significant negative impact on the identification performance. Our experimental results show that a fingerprint which has a top rank can be demoted to a bottom rank when two or more minutiae are missed. From our experimental results, we have noticed that some minutiae are more critical than others to correctly identify a latent fingerprint. Based on this finding, we have created a dataset to train several machine learning models trying to predict the impact of each minutia in the matching score of a fingerprint identification system. Finally, our best-trained model can successfully predict if a minutia will increase or decrease the matching score of a

Introduction
In the last two decades, machine learning has attracted increasing interest in the research community, eager to automatize several processes in different application areas [1,2]. One of these areas is Biometrics [3][4][5][6][7][8][9], which aims to use several unique characteristics of the human body to automatically verify or identify people. From the several physiological traits of the human body, fingerprints hold the label of being the most reliable due to its uniqueness, permanence, acceptability, and collectability [2,3,5,10,11]. Fingerprints are considered genetically unique; up to now, there have been no reports of two individuals having the same fingerprint. Therefore, this biometric feature is widely used in several application fields, such as criminology, banking, and healthcare [2,[12][13][14][15].
Fingerprints can be classified into impressions and latent fingerprints [6,7,16,17]. Fingerprint impressions are acquired under controlled conditions and have high quality. By contrast, latent fingerprints are unintentionally left by someone when manipulating objects and have a low quality [18] (see Figure 1). Similarly, the methodologies for comparing fingerprints have been clustered into two groups. On the one hand, fingerprint verification, which aims to verify an individual's identity from two given impressions in a fully automatized process. On the other hand, fingerprint identification, which aims to search a background database for impressions that are the most similar to a given latent fingerprint query usually related to forensic applications [12,[19][20][21][22]. Example of a latent fingerprint and its corresponding impression, taken from the database NIST SD27 [23]. Notice how the latent fingerprint is a low-quality image, containing background noise that interrupts ridge flow. On the contrary, the impression is a clear and high-quality image taken under controlled conditions.
In 2006, in response to a misidentification of a latent fingerprint in a high profile case, the Federal Bureau of Investigation commissioned a review committee to evaluate the current state of forensic latent fingerprint identification technologies and processes and to better understand the bases of this discipline [24]. Their results showed that there are scientific areas where improvements in the practice can be made, particularly regarding validation, the analysis, comparison, evaluation, and verification (ACE-V) process, and data collection.
Fingerprint verification technologies based on digital impressions are dominated by fully automatic systems (e.g., smartphone authentication sensors). However, forensic latent fingerprint identification is usually performed through semi-automatic processes where human participation is still critical (e.g., minutiae labeling). Latent fingerprints are important evidence in thousands of real prosecutions all around the world. Together with DNA, the latent fingerprint is the most important biometric identification trait in these prosecutions. The main stages of latent fingerprint analysis are (a) reveal; (b) capture; (c) feature extraction, and (d) identification or matching. In nowadays forensic analysis, stages a to c are usually performed by human experts [25] while d is performed by automatic algorithms. These semi-automatic processes are error-prone; errors may originate either from a human or an algorithm. The analysis of these errors is critical to understand the performance of these systems. There are studies analyzing the likelihood of human errors in the stages a to c and showing these errors are habitual [26][27][28][29]. Nevertheless, there are not studies evaluating how these errors affect the identification stage, (d).
Minutiae extraction is a critical stage for any latent fingerprint identification system [13]. Usually, Automated Fingerprint Identification Systems (AFIS) work adequately for extracting minutiae from impressions but not for latent fingerprints. As a result, human fingerprint examiners are needed for extracting minutiae from latent fingerprints [21,22]. However, failing at marking minutiae or marking spurious minutiae can lead to misidentifying latent fingerprints, releasing criminals, or, even worse, yielding the apprehension of innocent people [21]. Recent studies [21,30] have shown that automatic fingerprint identification methods based on minutiae are often readily applicable and commonly outperform other types of methods. However, latent fingerprint identification rates are as low as 71.32% for Rank-1 when using medium-size databases (100,000 fingerprints) [30]. Consequently, latent fingerprint identification is an area of constant evolution, which aims to find new methods that help obtain better identification rates.
Based on the above discussion, three main research questions arise: • What is the impact of human errors in the performance of Automatic Fingerprint Identification Systems?
• Are there minutiae having a higher impact on the identification performance when missed? • If so, is there any feature of a minutia that can help determine its importance?
Aiming to develop methods that can help latent fingerprint examiners make better decisions with automatic methods as well as respond to the research questions mentioned above, in this paper, we propose to study how human error during minutiae extraction can affect the identification of latent fingerprints. By understanding the effect these mistakes have on matcher performance, we can identify research areas related to developing methods that seek to aid the latent fingerprint examiners.
The main contribution of this paper are summarized as follows: • To the best of our knowledge, this is the first study related to the matching algorithms' performance when minutiae are missed from latent fingerprints. • We quantify and discuss the impact of missing minutiae in the latent fingerprint on two latent fingerprint matching algorithms' performance by removing manually marked minutiae from latent fingerprints and calculating their matching score and rank, and comparing them to the original ground-truth latent fingerprints.
By removing minutiae from latent fingerprints, we simulate human mistakes. We show that even a single mistake could be fatal to the process. The variation in matching score when removing different minutiae leads us to believe that some minutiae are more important than others when it comes to latent fingerprint identification [13]. If we were able to determine the specific features of minutia that have a high impact on matching score, we could develop methods to aid latent fingerprint examiners by marking regions [31] where a mistake in feature selection would have a high negative impact in matching, in a kind of fingerprint quality map [18,32].
The remainder of this paper is organized as follows. In Section 2, we analyze previous works. After, in Section 3, we state the role of human errors in the forensic fingerprint analysis. Next, in Section 4, we define all our experimental setup. Then, in Section 5, we evaluate the impact of minutiae errors in latent fingerprint identification. Then, in Section 6, we experiment on predicting the utility of given minutiae for latent fingerprint identification. Finally, in Section 7, we present our conclusions and future work.

Previous Works
This section gives an overview of previous research that is closely related to ours. This includes the work done related to human performance in latent fingerprint minutiae extraction and matching, as well as the work related to the performance of fingerprint matching algorithms when the input is modified or distorted.

Research on the Performance of Experts in Latent Fingerprint Analysis
In this sub-section, we present works that study the performance of human latent fingerprint examiners when marking or matching latent fingerprints. These works are intended to show that even experts can make mistakes and serve as the motivation for our research on the effects of mistakes in this area.
In 2011, Ulery et al. studied the accuracy and reliability of forensic latent fingerprint decisions by using empirical approaches [26]. In particular, they studied how the quantity and quality of image features relate to the level of consensus among examiners and their decisions. The study of Ulery et al. was focused on determining the frequency of falsepositive and false-negative errors, the extent of consensus among examiners, and factors contributing to variability in results. In such a study, 169 latent print examiners participated, who were generally highly experienced (83% of the participants were certified as latent print examiners). The data included 356 latent fingerprints, from 165 distinct fingers from 21 people, and 484 impressions combined finally making up 744 distinct latent-impression image pairs. Ulery et al. balanced the number of fingerprint pairs used in their study and the number of examiners assigned to each pair. As a result, 100 image pairs from the 744 image pairs were assigned to each participant examiner.
The results of Ulery et al. showed that, on the one hand, the false positive rate error (FPR) was 0.1%, and never two examiners committed an error on the same comparison. These results indicate that blind verification (between two examiners) should be highly effective at detecting this type of error, which demonstrates the potential of information fusion approaches in this problem [33,34]. On the other hand, the false-negative error was much more frequent (7.5% of mated comparisons). An in-depth analysis detected that 85% of the examiners committed at least one false-negative error. Using a blind verification would detect most of the false-negative errors; however, this blind verification is not generally practiced in operational procedures. After studying the quality of the latent fingerprints (quantity of features, distortion, background, among others) [18,35], this study could not identify the prints' features associated with false positive or false negative errors.
Later on, in 2015, Ulery et al. analyzed the changes in latent fingerprint examiners' markup [27]. The authors collected 320 (231 mated and 89 nonmated) image pairs of fingerprints constructed from 301 latent fingerprints and 319 impressions. Each one of the 170 volunteer latent print examiners was randomly assigned to analyze 22 pairs of fingerprints. From the 170 participants of this study, 89% were qualified as latent print examiners.
The experiments executed by Ulery et al. showed that from 41,774 minutiae labeled, 87% were retained, 6% were moved, 7% deleted, and 17% were added, see Figure 2. From these reports, the authors concluded that the comparison rates for inclusion or exclusion of minutiae ranged from 14% to 54%.  [27]. On each latent fingerprint image, the retained minutiae are in yellow, moved in blue, deleted in red, and added in green.
A detailed report of Ulery et al. showed that most of the latent fingerprint examiners (86%) included or excluded minutiae in the majority of their comparisons. When examiners were individualized (without checking by using double examiners), they changed their markup of minutiae in most comparisons. In addition, examiners included minutiae less frequently when the fingerprint image pair was nonmated than mated, being a possible explanation that the comparison with a mated exemplar draws attention to additional corresponding features in the latent. The authors claimed that the high variation in rates could be due to participants' unfamiliarity with the tested tools as well as the instructions and bringing casework habits. Next, in 2016, Ulery et al. analyzed the interexaminer variation of minutia markup on latent fingerprints [28]. Similar to [27], Ulery et al. [28] used 170 latent print examiners from which 89% got the qualification of latent print examiners. The dataset collected 320 fingerprint image pairs, 231 mated (from the same finger and person), and 89 nonmated (from different fingers or individuals). The image pairs were made with 301 latent fingerprints and 319 impressions. Each examiner was randomly assigned 17 mated fingerprint image pairs and 5 nonmated fingerprint image pairs. Their results were based on the analysis of 3730 fingerprint image pair, in total, among all the 170 latent print examiners, with a median of 12 examiners assigned to each fingerprint image pair.
Ulery et al. [28] also proposed to analyze whether examiners agree on the inclusion or exclusion of minutiae by measuring the minor variations in minutia location. For doing that, they used the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [36] to group minutiae such that more than two examiners agreed they are the same on the latent fingerprint. In this way, as shown in Figure 3, clusters are labeled as a singleton (marked by only one examiner), minority (<50% of examiners), majority (50-90%), and supermajority (≥90%). . Two examples of latent fingerprints marked by several examiners, taken from [28]. Marked minutiae are shown as small black dots inside color-coded clusters. Singleton (marked by only one examiner), minority (<50% of examiners), majority (50-90%), and supermajority (≥90%).
The results of Ulery et al. showed that 66% of minutiae were reproduced by most examiners, 67% of the singleton clusters were in unclear areas of the fingerprints, and 98% of supermajorities were clear areas. After an analysis, the authors identified several factors that affect that several examiners can reproduce the same minutia in a fingerprint: clarity, region of interest, feature type, and location. Authors also claimed that a difference in minutiae markup could be due merely to a difference in how an interpretation is documented by an examiner.
Recently, in 2020, Kukucka et al. studied the impact of evidence lineups on fingerprint expert decisions. Their experiment involved the participation of 43 latent fingerprint examiners, at an average age of 44 years, most of them affiliated to a large laboratory. In the experiment, each examiner viewed four fingerprints, some of which belonged to a suspect. Fingerprints came from a crime scene, visited by the examiners, as the underlying hypothesis of Kukucka et al. is that an examiner could associate each latent fingerprint to a crime scene. Their results have shown that examiners made a positive identification of 37.8% for displayed images; by contrast, 25.6% images were classified as inconclusive. The main drawback of [29] is that examiners were aware of being part of a monitored study. This may explain why the study output an unusually high rate of inconclusive judgments attributed to suspicion of examiners aware they were under assessment. The study proposed by Kukucka et al. is more for the implementation of evidence lineups than for analyzing the fingerprint from a dactyloscopy point of view. As a consequence, fingerprint features such as minutiae, pore, core, among others, were not analyzed.
In summary, existing research has focused on studying human errors during the latent fingerprint markup. However, it has not considered the impact of missing minutiae on the score of matching algorithms or on the ranking results.

Impact of Fingerprint Variations in Automatic Fingerprint Recognition
In this section, we present an analysis on the performance of fingerprint identification systems in the presence of different distortions.
Studying the impact of fingerprint variations on fingerprint identification has attracted some attention in the last two decades, including position variability [35] and other distortions [10].
The most up-to-date and comprehensive work in this regard may be Grosz et al. [37], who performed a white-box evaluation of three fingerprint matchers so as to assess how different perturbations in fingerprints affect the matchers' scores. Of these algorithms, one, called SourceAFIS, was open-source and the other two were unnamed, minutiae-based commercial-off-the-shelf (COTS) matchers. Authors evaluated the robustness of these matchers against controlled perturbations of the fingerprints, specifically perturbations of minutiae sets. The fingerprint dataset used in this experimentation consists of fingerprint impressions, synthetically generated through SFinGe [38]. It is composed of 5000 different "master" fingerprints, each of which was used to generate two different impressions. For performance evaluation, Grosz et al. also generated a ground-truth minutiae set for each fingerprint.
Authors ran experiments where fingerprint impressions involved the following perturbations: (i) moving and rotating minutiae; (ii) adding spurious minutiae and removing ground-truth minutiae; (iii) considering a non-linear distortion of the minutiae sets; and (iv) combining the displacement and rotation of minutiae with the addition of spurious minutiae and removal of ground truth minutiae. The first two perturbations were generated by using a multivariate Gaussian Distribution model, which had parameters the researchers considered to model real possible perturbations. The non-linear perturbations were generated by using a non-linear distortion model learned from distorted fingerprints. Using these distortions, the authors performed two different experiments: (a) an analysis of uncertainty resulting from realistic amounts of perturbation and distortion; and (b) an evaluation of the recognition performance of each minutia-based matcher on increasing levels of perturbation and distortion.
To perform the uncertainty analysis, Grosz et al. generated 100 perturbations per perturbation type, per generated fingerprint. They obtained similarity scores and used them to calculate a global uncertainty score relating to each perturbation type. When performing the second experiment, the authors increased the perturbation parameters throughout eight iterations. The results of this experiment are reported as true acceptance rate at a fixed false acceptance rate of 0.01%. They also calculated impostor scores for each perturbation type, on each iteration. The authors found out that the non-linear distortion is the perturbation that reduced the mean of the similarity scores the greatest, followed by the combined perturbation, and after that, the removal of ground truth minutiae. The experiments suggested that the fingerprint identification algorithms are robust to elevate percentages of missing minutiae. The large number of minutiae in these experiments allows to maintain a competitive identification performance after missing minutiae.

Forensic Fingerprint Analysis: Role of Human Errors
This section introduces the main errors a human may incur in before and during the entire fingerprint identification process, possibly yielding misidentifications (see Figure 4). Latent fingerprint identification is composed of four main stages, namely: reveal, capture, feature extraction, and identification. The more common errors typically observed at each one of these stages are as follows [16]: Reveal: in this stage, an expert should use specific powders, based on the surface and shape of the object, to reveal the latent fingerprint. The errors one may incur at this stage are: • Using a revealing powder that does not correspond to the surface or shape of the object where the latent fingerprint is, which could wrongly reveal the ridges of the latent fingerprints.
• Using more revealing powder than necessary, creating a filling among the fingerprint ridges, and consequently, making it hard to be of use on the next stages.
Capture: in this stage, an expert must be careful at capturing the latent fingerprint under the best possible conditions. Errors that are typically observed at this stage are: • While lifting the latent fingerprint, deformations might be created in the ridges creating false bifurcations or false ends on the ridges. As a result, errors are induced that may severely affect the feature extraction stage. • Output an out of focus or low-resolution photo of the latent fingerprint. • Lack of use of a rule next to the latent fingerprint that may enable one to estimate the real scale of the fingerprint.
Feature extraction: during this stage, the examiners will extract all features necessary for the identification stage. The more common flaws one may experience at this stage are: • False features can be added from errors originated at the capture stage. • Actual features could be missed by examiners. • The position or angle of some features could be shifted due to human perception.
Identification: any flaw committed at previous stages will affect identification performance. However, the main errors that may show up at this stage are: • The ranking provides the corresponding impression, but the experts obviate this matching. • The ranking does not provide the corresponding impression due to human errors issued at the previous stages. • The expert issues a true positive identification when really it is a false positive.
All the human errors previously described above negatively affect the identification rate. One of the most important features for fingerprint identification is minutiae. A minutia is a minute detail on the ridges of a fingerprint, often ridge ending or bifurcation. Minutiae are one of the most used features for fingerprint identification [5,21,22]. Hence, we will analyze minutiae's impact on fingerprint identification and the matching algorithms upon the possible appearance of human errors.

Materials and Methods
This section describes the database used and its characteristics. After, the two tested matching algorithms are presented. Finally, our experimental setup is presented.
We used the database NIST SD27 [23], which is a public database widely used in latent fingerprint studies [13,39]. NIST SD27 contains images of 258 latent crime scene fingerprints with their manually marked minutiae, as well as their matched tenprint impressions with their extracted minutiae. The average number of minutiae in latent fingerprint images in NIST SD27 is 20, while the average number of minutiae in the corresponding impressions is 106.
Additionally, for some experiments, we used the NIST SD4 [40] as a background database. This database contains images of 2000 rolled fingerprints.
For determining the impact of our study on fingerprint identification, we used two matching algorithms that have demonstrated highly competitive performance in latent fingerprint identification: • Minutia Cylinder-Code (MCC) [41]: Matching algorithm based on a three-dimensional representation constructed using basic minutiae features such as angles and distances to other minutiae. • Deformable Minutiae Clustering using Cylinder-Codes (DMCCC) [19]: A matching algorithm independent of minutiae descriptors based on the use of clustering to improve robustness to non-linear transformations. In this case, we use Minutia Cylinder-Code as the minutiae descriptor.
We used CMC [42] curves as an evaluation measure adopting the norm ISO/IEC 19795-1, which indicates that CMC curves should be used for closed-set identification. As numerical measures of the CMC curve, we use from Rank-1 to Rank-100 identification rates. Notice that these ranks quantify the ratio of correct identifications in the first place and among the 100 first ranks, respectively, returned by an identification algorithm.
Our protocol consisted of three sets of experiments in which we removed minutiae from fingerprints and evaluated the impact of such minutiae removal on the two selected matching algorithms plus a final prediction experiment: 1.
The first set of experiments consists of randomly removing minutiae from latent fingerprints in the database and comparing the resulting CMC curves in a closed set comparison. We ran 35 experiments; the first 20 experiments consisted of removing a fixed number of minutiae from each latent fingerprint, from 1 to 20. The next 15 experiments consisted of removing a percent of minutiae from each latent fingerprint, from 0.1% to 0.75%. Each experiment was run 10 times with different randomlyselected removed minutiae, and the results of the 10 experiments are then averaged.

2.
The second set of experiments aims to measure the matching algorithms' negative impact when minutiae are removed. We remove every possible combination of one, two, and three minutiae from every fingerprint (more than 1,100,000 combinations for all fingerprints in the NIST SD27) for this set of experiments. Every new fingerprint (with the removed minutiae) was tested with the two selected matching algorithms to obtain their matching score and compared with the matching score obtained from the original fingerprints (without removed minutiae).

3.
The third set of experiments consists of determining the set of minutiae that less decreased the score and those that decreased the most the score. Using our second experiment results, we selected six combinations of minutiae for each fingerprint and each matching algorithm to see how the change in score is reflected in the CMC curve and the rank-100 identification. The three combinations which lowered the score the most are considered the "lower" class. The three combinations which lowered the score the least are considered the "higher" class.

4.
Finally, we perform one last experiment aimed to determine if it is possible to predict if removing a minutia will have a positive or negative effect on the matching score.
It is important to highlight that, for all experiments, the minutiae were not removed from fingerprints if a fingerprint was to be left with less than six minutiae, based on what INTERPOL recommends as a minimum for an AFIS [43].

Evaluating the Impact of Minutiae Errors
This section shows the experimental results obtained when removing fixed and variable numbers of random minutiae from each latent fingerprint.
For our first set of experiments, Figure 5 shows CMC curves for random fixed number removal of minutiae using both MCC and DMC as matching algorithms. From this figure, we can see how both matching algorithms degrade for the increasing number of minutiae removed. In addition, notice that MCC obtains worse identification rates than DMC regarding each CMC curve plotted. From these results, we can see that no matter the matching algorithm used, the identification rate is affected when minutiae are randomly removed from latent fingerprints. The drop of performance varies depending on the matching algorithm, but it is consistently high with rank-n identification degradation higher than 5% for each minutia removed.   Figure 6 shows the same results from Figure 5 in a different representation. From Figure 6, we notice that the identification rate decay proportionally to the number of minutiae removed. We can also see that rank one is more affected when minutiae are removed than the remaining ranks (from 2 to 6). In addition, we can notice that MCC's identification rates are more affected than those of DMC regarding the number of minutiae removed and the ranks analyzed. From our first set of experiments, we noted that the identification rate depends on both the number of removed minutiae and the total number of minutiae. Consequently, another essential experiment is to test removing a percentage of minutiae from each latent fingerprint.
For our second set of experiments, Figure 7 shows CMC curves for random proportion removal of minutiae using both MCC and DMC as matching algorithms. From this figure, notice that the identification rate decays very fast, with rank-n identification rate differences of 12% for a ratio of 0.2 missed minutiae. In addition, from Figure 7, we can see that the identification rate obtained by MCC (Figure 7a) is worse than the identification rate obtained by DMC (Figure 7b). These results suggest that the characteristics of the matching algorithm can serve to alleviate this drop in performance partially.  From our second set of experiments, we noted that some minutiae affected the matching scores differently than others. In addition, on the one hand, 2/3 of the minutiae lowered the matching score when removed. On the other hand, 1/3 of the minutiae have a null effect or even a small improvement of the matching score when removed when compared to the base score (no minutiae removed).
From the results obtained in our second set of experiments, we concluded that some minutiae are more critical for identification, but we still do not know why they are more important than others. The next experiment is performed to characterize minutiae that lower the matching score and compare them to the remaining minutiae to determine the reasons for this negative impact.
Using the results of our second set of experiments, we selected the 6 minutiae combinations of sizes one, two, and three that had the largest impact in matching score for each latent fingerprint and each matching algorithm to see how the change in scores reflected in the CMC curve and from the rank-1 to rank-100 identification.
The six combinations for each fingerprint and matching algorithm were determined by taking the three combinations of minutiae that lowered the score the most ("lower" class) and the three combinations of minutiae that raised the most ("higher" class). Figure 8 shows a latent fingerprint taken from NIST SD27 [23], which contains the minutiae labeled as "higher" class in green color, those labeled as "lower" class in red color, and the remaining minutiae in blue color. From this figure, we can see that those minutiae labeled as "lower" class are well separated from those labeled as "higher" class. In addition, we can notice that the sum of the score of all "lower" class minutiae (−0.1728) affects more negatively the identification score than the sum of the score of all "higher" class minutiae (0.0665).

Figure 8.
Example of a latent fingerprint taken from NIST SD27 [23]. Minutiae labeled as "higher" class are in green color, those labeled as "lower" class are in red color, and the remaining minutiae are in blue color. In both "higher" and "lower" classes, the minutiae contain the score difference that they introduce to the final score when that particular minutia is removed. Table 1 shows percentages in which there was a rank drop considering only those minutiae labeled as "higher" class. Table 2 shows percentages in which there was a rank drop taking into account only those minutiae labeled as "lower" class. Notice that for both tables, MCC obtained worse identification rates than DMC. In addition, in Table 2, the identification rates were most affected when those minutiae labeled as "lower" class were removed. Our third set of experiments shows how the identification rate is affected when those minutiae labeled as "lower" class are removed. Figure 9 shows a histogram of rank loss of MCC when removing the minutiae of the "lower" class. From this figure, we can notice how removing minutiae labeled as "lower" class can affect the MCC rank value with variations up to 2000 positions in a ranked list. It means that a fingerprint ranked in the top 50 positions can easily drop to top 1000. These results demonstrate the high impact of these errors in the performance of fingerprint identification systems. In this figure, we can see, as expected, that the more minutiae are removed, the higher the rank loss. Notice that when only two or three minutiae are missed, the drop is severe for a large number of fingerprints.  Figure 10 shows a histogram of rank loss regarding DMC when "lower" class minutiae are removed. Contrasting the results obtained by Figures 9 and 10, we can see that the matching algorithm DMC is less affected than MCC when minutiae labeled as "lower" class are removed. However, notice that when two or three minutiae are removed the histogram is still highly affected.

Predicting the Impact of Minutiae Errors
In the previous section, we showed that removing minutiae can decrease or increase the score of fingerprint matching algorithms. If we could automatically predict the impact of the minutiae in the score before performing latent fingerprint identification, we could: (1) help the forensics experts to improve their minutiae labeling process to maximize the probability that they find minutiae that lead to successful latent fingerprint identification [16], and (2) improve automated algorithms for latent fingerprint processing and recognition using quality-based processing [44] and quality-based information fusion [45][46][47]. With these goals in mind, we developed a framework to determine if it is possible to predict if a given minutia will have a positive or negative effect on the score of fingerprint matching. Thus, we could develop tools for forensics examiners to automatically classify regions by its impact on the performance of the identification algorithm, or quality-based algorithms for latent fingerprint processing [45,46]. This framework was created by using the results of removing every possible combination of one minutia from a fingerprint, as described in the previous section for the DMC matcher. We created features based on the location of each minutia and the distances of neighboring minutiae. We designed two different types of features: (1) the first type counts the number of minutiae in a radius neighboring the specified minutia, and (2) the second type is the distance from the specified minutia to the n closest minutia. Finally, the class values positive and negative were used for minutiae that, after being removed, increased or decreased the matching score, respectively. In the end, each datum point in the created dataset has the following information: • d [1][2][3][4][5][6]: Each of these features has the distance from the minutia to the closest minutia (d [1]), to the second closest minutia (d [2]), and so on. • r [15,30,45,60,75,90]: Each of these features counts how many minutiae there are in a radius (of 15 pixels, 30 pixels, and so on) around the specified minutia.
• class: Either a positive or negative impact on the matching score.
In total, this dataset has 12 features and a class feature. It contains 5215 instances, of which 1991 are positive examples (minutiae that increase the matching score when removed) and 3224 are negative examples (minutiae that lower the matching score when removed).
We tested eight different supervised classifiers. We selected popular classifiers of different types to get varied results. For each algorithm, we did five-fold distribution balanced stratified cross-validation [48], which is used for imbalanced class problems. We evaluated each algorithm with the area under the receiver operating characteristic curve (AUC) [49]. The AUC is calculated as: where TPR is true positive rate and FPR is false positive rate.
The eight supervised classifiers that we used were:  [54]. Figure 11 shows a bar graph comparing the average AUC of each supervised classifier when using five-fold distribution balanced stratified cross-validation. From Figure 11, we can see that PBC4cip [50] has the best performance with an AUC of .608, closely followed by FLDA and Random Forest [51]. The accuracy is still low, but these results suggest the existence of certain information that can be exploited to predict if a minutia will increase or decrease the matching score of a fingerprint with a better than random AUC. These results open a new research line related to creating automatic algorithms that help forensics experts to improve the minutiae labeling process in latent fingerprints.

Conclusions and Future Work
Aiming to develop automatic methods that can help latent fingerprint examiners make better decisions and can enable quality-based fingerprint processing, we carried out a study of how minutiae errors can affect the identification of latent fingerprints. Both human experts [28] and automatic algorithms [13] commit errors when labeling latent fingerprint minutiae. There is some knowledge already in the literature about the source and factors that originate those errors in both cases: human experts [26][27][28] and algorithms [13,21], but there is a lack of understanding of how those errors impact fingerprint identification systems. The present work is a step forward in understanding the impact of those minutiae errors.
From our experimental results, we can conclude that missing even one minutia from a fingerprint can have a very high negative impact on identification performance. In addition, some minutiae are more important than others to identify a latent fingerprint correctly. We also conclude that a result in the first places of the rank could be demoted to the last places of the rank when two or more minutiae (labeled as "lower" class) are removed. This fact of varying minutiae utility, or minutiae quality in terms of biometric utility [18], can enable specific processing for different minutiae [13] in quality-based conditional processing [44] or quality-based information fusion schemes [45][46][47].
In future work, we plan to deepen our study on what features make a minutia or a minutiae combination more important for identification performance. In addition, ongoing work is around determining critical areas of a latent fingerprint image in which missing a minutia could make the matching algorithms fail to identify a fingerprint correctly.

Short Biography of Authors
Octavio Loyola-González received his Ph.D. degree in Computer Science from the National Institute for Astrophysics, Optics, and Electronics, Mexico, in 2017. He has won several awards from different institutions due to his research work on applied projects; consequently, he is a Member of the National System of Researchers in Mexico (Rank1). He worked as a distinguished professor and researcher at Tecnologico de Monterrey, Campus Puebla, for undergraduate and graduate programs of Computer Sciences. Currently, he is responsible for running Machine Learning and Artificial Intelligence practice inside Altair Management Consultants Corp., where he is involved in the development and implementation using analytics and data mining in the Altair Compass department. He has outstanding experience in the fields of big data and pattern recognition, cloud computing, IoT, and analytical tools to apply them in sectors where he has worked for as Banking and Insurance, Retail, Oil and Gas, Agriculture, Cybersecurity, Biotechnology, and Dactyloscopy. From these applied projects, Dr. Loyola-González has published several books and papers in well-known journals, and he has several ongoing patents as a manager and researcher in Altair Compass.


extfieldsofstudy: Medicine Physics

text:
Adaptive phase-retrieval stochastic reconstruction with correlation functions: 3D images from 2D cuts

Precise characterization of three-dimensional heterogeneous media is indispensable in finding the relationships between structure and macroscopic physical properties (permeability, conductivity, and others). The most widely used experimental methods (electronic and optical microscopy) provide high-resolution bi-dimensional images of the samples of interest. However, 3D material inner microstructure registration is needed to apply numerous modeling tools. Numerous research areas search for cheap and robust methods to obtain"full"3D information about the structure of the studied sample from its 2D cuts. In this work, we develop a dynamic phase-retrieval stochastic reconstruction algorithm that can create 3D replicas from 2D original images - DDTF. The DDTF is free of artifacts characteristic of previously proposed phase-retrieval techniques. While based on a two-point $S_2$ correlation function, any correlation function or other morphological metrics can be accounted for during the reconstruction, thus, paving the way to the hybridization of different reconstruction techniques. In this work, we use two-point probability and surface-surface functions for optimization. To test DDTF, we performed reconstructions for three binary porous media samples of different genesis: sandstone, carbonate, and ceramic. Based on computed permeability and connectivity ($C_2$ and $L_2$ correlation functions), we have shown that the proposed technique in terms of accuracy is comparable to the classic simulated annealing-based reconstruction method but is computationally very effective. Our findings open the possibility of utilizing DDTF to produce fast or crude replicas further polished by other reconstruction techniques such as simulated annealing or process-based methods.

I. INTRODUCTION
Precise characterization of three-dimensional heterogeneous media is indispensable in finding the relationships between structure and macroscopic physical properties (permeability, conductivity, and other properties). [1][2][3].The most widely used experimental methods (electronic and optical microscopy) provide high-resolution bidimensional images of the samples of interest. However, 3D material inner microstructure registration is needed to apply modeling tools. Obtaining quality 3D information is technically more complicated and requires expensive equipment and qualified operators. Thus, porous media reconstruction from thin sections [4,5] is actual and has many petroleum and food engineering [6,7], material and soil sciences, [8,9], electrochemistry [10], and * cherkasov.am@phystech.edu medical applications [11]. Effective reconstruction procedures allow the creation of high-quality digital models for further analysis and calculation of physical properties. This is true for various multiphase materials, including porous media with a binary structure consisting of pores and solids. A versatile set of methods is available in the literature for stochastic reconstructions, including creating 3D replicas from 2D slices. Truncated Gaussian random fields [12,13] are a relatively computationally inexpensive approach but do not always allow the preservation of morphological properties of porous media. Simulated annealing (SA) using correlation functions [14,15] allows obtaining high-quality reconstructions, but it is a long-duration process even when accelerated (e.g., hierarchical simulated annealing) [16,17]). Note that hierarchical annealing schemes with correlation function rescaling were proposed recently to overcome this computational disadvantage [18]. Multiple-point statistics (MPS) [6,19,20] reconstruction was shown to reproduce mor-phology quite well compared to other methods. However, it is computationally expensive and may produce repetitive textures based on the number of points used for statistics. Quite surprisingly, some MPS methods are even slower and less accurate than SA based on twopoint statistics [21]. Deep learning approaches [22][23][24] are getting popular and show great promise, but even more computationally expensive than MPS and SA, with lower accuracies not balanced by massive training times. Process-based methods are very efficient and accurate, but their applicability is limited to granular porous media [25,26]. Phase-retrieval [27] is a fast algorithm that allows reconstructing volume from its two-point statistics [3] (or two-point probability correlation function). A method for approximating three-dimensional two-point statistics by a set of two-dimensional ones has been developed by Hasanabadi et al [28], and a three-dimensional image has been reconstructed from a bi-dimensional image. This approach has high performance and retains the basic microstructural properties of the media. However, the resulting image is rather noisy: artifacts like stripes and individual misplaced voxels appeared in the reconstructed sample. While this had only a minor impact on simulated electrical conductivity, other physical properties, especially permeability, are very sensitive to such noise [29]. The originally described phase-retrieval method [27] used two-point statistics between each possible combination of pairs of pixels/voxels. This way, it was possible to reconstruct the images exactly [30]. However, in this case, the amount of two-point statistics is larger than the 2D or 3D image itself (even if we discard the parts redundant due to symmetry). Such reconstructions do not seem to have much practical application, especially considering the fact that two-point statistics that can be measured experimentally, for example, with the help of small-angle scattering [31] or X-ray tomography [32], is limited to ensemble average over the whole volume or some direction. Ensemble statistics is a kind of compressed two-point statistics and can be effectively utilized to reduce the structural information about the object at hand [33][34][35]. On the other hand, if the aim is to reconstruct 3D information from the 2D slice, using the complete information available from such a slice using full two-point statistics may be beneficial. In this work, we develop a phase-retrieval algorithm of three-dimensional sample reconstruction from its bidimensional cut-section free of the abovementioned disadvantages. The paper is organized as follows. We review the mathematics behind the phase-retrieval algorithm as well as the algorithm itself. Next, we describe rotation as a method for increasing the dimensionality of self-convolution using the fullest possible two-point statistics read from a 2D image. A new phase retrieval algorithm based on additional constraints in Fourier space and static and dynamic adjustments of three-dimensional self-convolution is presented. To demonstrate achieved improvements, we simulate single-phase (or saturated) flow on both original and reconstructed samples of three porous media sam-ples of different genesis. We also evaluate the two-point cluster correlation function known to provide non-trivial connectivity information.
Microstructure m n s is a binary multidimensional array where values 0 and 1 correspond to void and solid voxels, respectively. (s 1 , ..., s N ) : s 1 ∈ (1, 2, ..., S 1 ), s N ∈ (1, 2, ...S N ) is a vector of space coordinates while n enumerates different components present in certain images, and N is the dimension of space to which microstructure appertains. Term correlation is commonly used in signal processing for convolution of the image with inversed self. However, this term in statistics has another meaning. Thus in this article, correlation in the sense of image processing is called convolution, while correlation is obtained as a result of either averaging convolutions by different directions in one image or by image ensemble averaging. Convolution of m n s is defined the following way: where S = (S 1 , ..., S N ), t = (t 1 , ..., t N ) ∈ R n . It contains sufficient data for retrieval of the initial binary image to within a translation and inversion. Fulwood et al. establish the link between Fourier transform of microstructure and self-convolution so that the Gershberg-Saxton algorithm of image retrieval from magnitude of its Fourier transform becomes applicable.
f nn s e 2πisk/(S1·...·Sn) The resulting algorithm 1 consists of main stages. Fourier transform of microstructure on current iteration.

5:
Replacement its magnitude with the magnitude obtained from self-convolution.

6:
Inverse Fourier transform of replacement result.
Satisfying constraints in real space: Using black padding around the input image before calculating its self-convolution and taking into account the position of padding during recovery results in a better convergence and reduction of translation uncertainty [36]. Nevertheless, in the case of approximated convolution reconstructed image with padding tends to have lower porosity in the slices adjacent to padding. Thus usage of padding is appropriate in cases when the role of boundary effects does not seriously influence macroscopic properties like porosity and permeability.
B. The transition from 2d convolution to 3d convolution via rotation One way to obtain a one-dimensional correlation for isotropic microstructure is by averaging its selfconvolution in all directions. Self-convolution has axial symmetry. We propose rotation as a way to translate convolution from bi-dimensional to three-dimensional space. Appropriate transition from 2d to 3d convolution is rather important, as it results in quality of reconstruction. In representative volume of porous media there is linear relationship between expected values of 3d-convolution and 2d-convolution for equal shift values. For binary im- is where p is intersection porosity of original and shifted image. For 3d-convolution for shift (t, 0, 0) expectation is In case of adding black padding and doubling the linear size of an image S = 2S The approximation with properties mentioned above can be used as input for the phase-recovery algorithm. Sample retrieved from this 3d-convolution has a preferential direction which is the convolution rotation axis. Rotation of 2d-convolution around ox, oy, oz axis and averaging the results lead to isotropy of reconstructed image (2). Three-dimensional convolution does not always correspond to existent microstructure, resulting in defects in a reconstructed image such as deformation and noise (figure 1). Adding constraints in Fourier space such as frequency filtering is an efficient way of noise reduction. Further in the article, samples are presented without padding for better presentation and visibility. Though filter adjustment should be performed in each specific case, the width of the corresponding filter in real space should be no less than one voxel. Figure 3 shows the reconstructed samples. The first image is obtained with no filtering. The following images correspond to space filters' widths of 0.5, 1, 1,5, and 2 voxels. Choice of appropriate mask width is the solution to the optimization problem for the cost function. For example, it is possible to minimize the difference between the correlation of the input image and the median correlation calculated of all slices. Stochastic optimization is warranted here because of a significant number of local minima. Thus modified algorithm 2 at step 6 also contains multiplication. Fourier transform of microstructure on current iteration.
Replacement its magnitude with the magnitude obtained from self-convolution.

6:
Frequency filtration with Gaussian filter 7: Satisfying constraints in real space:  Figure 4 shows that transition functions from 2dcorrelations to 3d-correlation for such objects as square, circular disc, and 2d-plurigaussian are non-linear and sufficiently different. We refer to such transition functions for correlations as "Dimensional Transition Functions" or DTF.

Static dimensional transition functions
Thus approximated 3d-convolution should be adjusted later on account of microstructure morphology. Plurigaussian modeling of microstructures is one of the widely used methods [12] for microstructure modeling. Random topography field T is obtained by convolving kernel k(x) with u(x) ∈ U -random field with uniform probability density distribution.
The convolution of the Gaussian kernel with the random field is shown in figure 5.
We implemented an algorithm 3 to generate an approximated microstructure in which 2d-slice corresponds with the input image. Its realization is described thereunder. Calculate the self-convolution of a 2d-image. 2: Average self-convolution by all directions to obtain 1D-correlation. 3: Translate 1D-correlation to 3d-space by rotation. 4: Generate a 3d-random field with uniform distribution. 5: Convolve kernel obtained by rotation of self-convolution and random field with uniform distribution. 6: Truncate the resulting plurigaussian so that the porosity of the middle slice of the resulting microstructure is the closest to the porosity of the input image. 7: Circularly shift truncated plurigaussian for alignment with input slices using a maximum of cross-convolution. 8: Apply morphological erosion or dilation to minimize differences between correlation functions of the input slice and the particular slice of truncated plurigaussian. 9: Calculate correlation function from radial shift for certain 2D-slice of plurigaussian approximation. 10: Calculate correlation function from radial shift for the whole plurigaussian approximation. 11: Plot set of points. Each point corresponds to a particular radial shift where the x coordinate is the value for 2d-correlation, and the y-value is 3d-correlation. 12: Approximate the dependency by power-law to apply it to 3d-autoconvolution obtained by rotation from bidimensional one Such plurigaussian microstructure is used only for building dimensional transition function in each specific case. A synthetic case like circular disk (figure 4) is an example when appropriate convolution transform is essential. Using traditional scaling, we obtain significantly worse results while plurigaussian approximation for circular disk leads to the reconstruction of an object similar to a solid sphere.
In the case of small-scale microstructure, DTF-correction does not play a significant role because the transition function is almost linear. Properties of the resulting sample differ in relation to noise realization because the kernel is convolved with random noise. Thus it is necessary to choose the best realization according to quality metrics such as correlation function. However, in the case of images whose characteristic scale is comparable with its size, plurigaussian approximation does not always give a reasonable guess for dimensional transitional coefficients.

The dynamic dimensional transition function
Quality for building both convolution and DTF influences reconstruction results. The solution is on-the-fly updating of the convolution in such a way that allows adjusting parameters of convolution during phase-retrieval algorithm execution. For example, parameters can be chosen to minimize difference according to a particular metric between a specific slice or slices of reconstructed and input image. The choice of metric is a single issue and depends on the integral characteristic of the 3dimage ( [3]). Approximated 3d-autoconvolution has radial and angular parts. The radial part is relatively smooth in all points except (0, 0, 0), while the angular one somewhat oscillates around zero field with magnitude decreasing with the growth of distance from autocorrelation center (0, 0, 0). Coefficients in the linear combination of these two parts in the resulting approximation for autoconvolution are also optimization parameters. Thus modified algorithm 4 is the following.
where a and b are dimensional transition coefficients.
where rad and ang are radial and angular parts respectively, and c is optimization coefficient. 3: Making random binary noise initial guess for recovered microstructure (m 1 s ) 0 . 4: repeat

5:
Fourier transform of microstructure on current iteration.
Replacement of its magnitude with the magnitude obtained from self-convolution.

7:
Frequency filtration with Gaussian filter which width is optimization parameter d. Calculating cost objective δ j as the difference between the correlation function of the input image and the correlation function of a particular slice of reconstruction.

10:
if δ j is min(δ(1), ..δ(j)) then Satisfying constraints in real space: Updating parameters a, b, c, d according to the optimization algorithm. For example, the correlation function shows good performance for the solid sphere mentioned above, even in the case of the wrong guess of SDTF. Even in case of inappropriate guess for static DTF convolution can be successfully modified for satisfying constraints for 2d-slices ( figure 6). There is no need to optimize parameters a and b due to the linear transition function for porous media. However, angular coefficient c and filter width d still need to be determined. In this case, the algorithm is the following. Replacement its magnitude with the magnitude obtained from self-convolution.

7:
Frequency filtration with Gaussian filter which width is optimization parameter d. Thresholding (m 1 s ) j to conserve porosity of input image.
where th is thresholod and (m 1 s ) 2d is input image.

10:
Calculating cost objective δ j as negative likelihood function (appendix A1) for surface correlation function of the input image and all the slices of reconstruction.

11:
Updating parameters c, d according to the optimization algorithm. While Algorithms 5 and 4 look very similar, they possess some principal differences, the initial guess being the most pronounced one. In Algorithm 5, it is always the same random noise. On the other hand, in Algorithm 4, the random noise is used as the first guess, and each next iteration uses the best previous results as input. Such an implementation allows combining optimization steps consisting of phase recovery with different parameters to equivalent phase-recovery with a more significant number of iterations. The influence of this approach is significant in cases when the number of phase recovery iterations per one optimization step is not enough to reconstruct microstructure from random noise. This seems unnecessary for general porous media (unlike the ball reconstruction problem) due to the less localized distribution of voxels resulting in less iteration number necessary for the convergence. Note that a simple increase in iterations number per optimization step would resolve this problem but is expected to increase overall iterations. Reconstruction of Turing pattern (which is a complex structure often used for algorithm attestation) by algorithm 5 leads to the following results ( figure 7). It can be seen that the slice of the reconstruction, as well as the original, contains both isolated solid and void areas of similar size, but the thickness of patterns is rarely conserved.

The classical simulated annealing reconstruction technique
In order to benchmark the new methodology developed in this work, we compare the reconstruction results against a classical reconstruction based on correlation functions. To make both techniques equal, we applied modified Yeong-Torquato reconstruction [5] and utilized only the two-point probability function S 2 (r) [3] describing the probability that two points separated by a vector displacement r(x 1 , x 2 ) between x 1 and x 2 lie in the pore phase. We calculate S 2 (r) functions in two orthogonal and two diagonal directions, thus, giving four independent CFs for each input 2D image, which are then used separately during reconstruction [29], which are averaged to estimate the other 4 CFs for the unknown third dimension. For any set of correlation functions considered in the Yeong-Torquato technique, matching correlation functions of a given realization with a target CFs set is based on pixel permutations. If a set of two-point correlation functions used in reconstruction is provided in the form of f α 2 (r), where α is a type of CF (different directions in our case) and r is a segment of varying length, the difference between two realizations of the structure can be expressed as the sum of squared differences [5,29]: where f α 2 (r) andf α 2 (r) are the values of the correlation function sets for two realizations (where the former represents a reference set while the latter represents current reconstruction state), w α are weighting parameters chosen based on the input of each CF into energy E for disordered structure [29]. The energy E in eq. 5 is minimized by the simulated annealing optimization. The Metropolis algorithm is used to determine the probability of accepting any permutation p: where T is the temperature of the system, and At initialization, the temperature is chosen so that the probability p for ∆E 0 equals 0.5 [5]. The following cooling schedule based on geometrical progression is used: where k is the time step and λ is a parameter smaller than but close to unity (λ=0.999999 for all reconstructions presented here). An optimized Capek pixel permutation approach based on interface choices [37,38] was used. Periodic boundary conditions were applied for CFs evaluation. The reconstruction procedure was terminated after 10 6 consecutive unsuccessful permutations.

E. Flow simulation
Fluid flow in pore space is described by the Navier-Stokes equation (9) [39].
where v= (v x , v y , v z ), µ, ρ and p are velocity field, viscosity, density and pressure field respectively. In the case of small Reynolds numbers Re = ρvl µ << 1, which is typical for flow in porous media, equation 9 can be transformed into the form (10) where domain Ω is pore space and ∂Ω is its boundary. Permeability K can be determined form Darcy's law (11) where µ is viscosity, L is the distance for which a pressure difference ∆p is applied, and Q is flow rate through the cross-sectional area S. The set of equations (10) has been solved FDMSS solver with 4th order spatial accuracy scheme [40]. To ensure good convergence in all modelling cases either 3.510 3 iterations were performed or we reached error criterion < 0.05 which is based on imbalance for both continuity and motion parts of equation (10).

F. Samples for reconstruction and comparison between techniques
To verify the newly proposed modified phase-retrieval algorithm and compare it against classical technique, we chose three 3D porous media images of different genesis: artificial ceramic [41], sandstone and carbonate rocks [42]. The choice was motivated by a wide range of porosities within these samples and their relative homogeneity and isotropy. Such properties are essential, as we assume the structures to statistically homogeneous so that we can consider r as a scalar distance between pixels while computing correlation functions and isotropy is needed to evaluate the unknown 3rd dimension to perform 2D into 3D reconstruction. We chose a single 2D slice with the porosity value closest to the porosity of the whole original 3D image as input data to both methods. Both simulated annealing and phase retrieval reconstructions are performed with periodic boundary conditions. To compare the reconstruction against the original 3D image, we compute error based on flow simulations using the following definition: where K reconstruction is the permeability of the stochastic reconstruction at hand, and K original is the simulated permeability based on the original 3D XCT image.

III. RESULTS AND DISCUSSIONS
Visual comparison between the original binarized XCT 3D images and their reconstructed replicas revealed no particular information regarding differences between phase-retrieval and simulated annealing methods ( figure  8). To the naked eye, it seems that SA performs somewhat better for ceramic sample, while PR resembles carbonate and sandstone originals in a superior fashion. In contrast to unmodified phase-retrieval from 2D to 3D images [28], our approach produces no apparent artifacts or distortions. Permeabilities of the stochastic replicas provide a much more robust benchmark 9. We immediately observe that both reconstruction methods underestimate permeability -this is to be expected as they are mainly based on the S 2 correlation function, which provides only a limiting information content [43,44] for complex porous media images used for the testing. For high porosity ceramic sample, PR replicas had higher permeabilities than SA, with one of the reconstructions being very similar to the original 3D image. For low porosity sandstone and carbonate samples, SA proved to provide consistently lower errors. To understand the permeability results better and to make the comparison between classical SA and developed PR method fully quantitative morphologically, further analysis in terms of L 2 , SS 2 , and C 2 correlation functions is presented in figures 10, 12. For ceramic sample, PR has higher connectivity (based on C 2 ) and larger pores (based on L 2 ). This results in high permeability values as explained by large and connected pores (visible on 8). For the carbonate sample, PR outperforms SA based on all additional CFs. Correlation functions for sandstone replicas reveal no obvious winner, as phaseretrieval shows better L 2 statistics while diverging more compared to annealing in terms of C 2 . This overall situation is rather interesting, as PR replicas being better in terms of CFs did not guarantee their permeability error K error lower that for SA reconstructions. Note that the S 2 correlation function's match was always perfect by design (both PR and SA minimized the difference).
Putting all the results from 8-12 together, it is impossible to choose either PR or SA as a more favorable approach, and this was not the target of the comparison. All in all, we conclude that both approaches are compa- rable in terms of their accuracy and perform differently for various sample structures. We iteratively improve the structure by voxel permutations with annealing, while phase-retrieval is more of a sequential algorithm. As SA swaps voxels preferentially to the interfaces between binary phases for improved convergence [37,45,46], we introduced surface-surface optimization [47] within PR to counterbalance such permutations. Without this augmentation, i.e., if based on the S 2 correlation function alone, PR performed slightly worse than SA. Thus, the current version of the comparison is deemed fair. Based on the large variety of images used in this work, we are safe to conclude that annealing with S 2 computed in orthogonal and diagonal directions produces consistently better results as compared to full-map S 2 self-convolution and, thus, annealing can be considered to be a polishing step [48]. While classic SA and modified PR method as described here do provide quite similar reconstructions in terms of balanced morphological and permeability assessment, phase-retrieval is computationally much less taxing. Based on our Matlab implementation, phase-retrieval with 40 optimization steps with 20 iterations each took approximately 40 mins for 300 3 voxels replicas. In contrast, simulated annealing took around 2 h for the same problem. While comparison of SA-based reconstruction times is always hampered by stopping criteria, these two 2 h was the average time needed to overcome PR in accuracy based on S 2 correlation statistics used as optimization targets in both algorithms. We believe that our PR implementation has much potential for speeding up and cannot be compared directly against legacy C++ implementation (the SA method). Another interesting aspect of the DDTF approach is its ability to utilize transition functions (4) to create stochastic yet highly deterministic 3D structures from 2D images. For example, this ability could be used to re-create 3D grain morphology from 2D cuts. This way, phase-retrieval can be coupled with process-based reconstructions [25,49] to create three-dimensional shapes during the packing process. Most importantly, in this work, we have explicitly shown that phase-retrieval can utilize more sophisticated structural statistics beyond the S 2 two-point correlation function -we tried SS 2 and C 2 so far, or other morphological metrics such as fractal dimensions [50,51]. This means that fast 3D reconstructions based on modified PR with the incorporation of other additional correlation functions can be used as input data for the SA algorithm [48] or building block for MPS reconstruction [6,20,52], as well as the abovementioned hybridization with process-based methods. Such potential hybridization of different methods and our modification that makes phase-retrieval similar to other main-stream reconstruction techniques paves the way to the overall unification of all approaches, as they solve the very similar optimization problem.

IV. CONCLUSIONS
In this work, we developed a dynamic phase-retrieval stochastic reconstruction algorithm for creating 3D repli-cas from 2D original mages -DDTF. The DDTF is free of artifacts characteristic of previously proposed phaseretrieval techniques. While based on two-point S 2 correlation functions, any correlation function (surface-surface correlation was utilized here) or other morphological metrics can be accounted for during the reconstruction, thus, paving the way to the hybridization of different reconstruction techniques. To test DDTF, we performed reconstructions for three binary porous media samples of different genesis. Based on computed permeability and connectivity (C 2 and L 2 correlation functions), we have shown that the proposed technique in terms of accuracy is comparable to the classic simulated annealing-based reconstruction method but is computationally very effective. This opens the possibility of utilizing DDTF to produce fast or crude replicas further improved by other reconstruction techniques such as SA or process-based methods. Improving the quality of reconstructions based on phase-retrieval by adding additional metrics into the reconstruction procedure is a research activity with a high potential to enhance the methodology significantly.

ACKNOWLEDGMENTS
This work was supported by Russian Science Foundation grant №19-72-10082 (M.V.K. and K.M.G.). The generous cooperation of the authors is within the Flow and Transport in Media with Pores research group (FaT iMP, www.porenetwork.com) and uses some of its software.

Appendix A: Maximal likelihood
The way to estimate probability that certain discredized curve y certatain (x i ) belongs to set of curves y j (x i ), ..., y N (x i ) is the maximal likelihood method. The cost objective δ needed to be minimized in this case is the following.
where mean y mean and standard deviation σ i are defined in A3 and A2 respectively.


extfieldsofstudy:
Physics
Computer Science

text:
Cognitive ergonomics of DNA-algorithms

To improve the ergonomic qualities of graphical interfaces and optimize the visual perception of large data presented in text form, a method of information processing based on DNA algorithms is proposed. These algorithms were developed by the author in the study of the molecular genetic and structural properties of long nucleotide sequences. The result of processing information presented in text form is an object of discrete geometry, two-dimensional projections of which can serve for visual identification of the internal structure of information, as well as for interpretation as characteristic symbols. At the same time, the considered transformations have the property of visualizing the characteristic cluster structure of information. It is shown that genetic nucleotide sequences, as well as texts in natural language, lend themselves to visual classification, which contributes to an increase in the productivity of mental work when analyzing big data. It is shown that DNA algorithms can reduce the load on the visual analyzer during the initial examination of data presented in text form. This makes it possible to apply this method when operators interact with various information streams and text data of arbitrary nature, as well as to improve the ergonomic qualities of specialized graphical interfaces.

Introduction
The quality of information perception depends on the way it is presented. Any information stream can be represented as a sequence of characters of a finite alphabet. A person's ability to perceive information through a visual analyser is closely related to the ability to visualize data. Despite progress in the field of machine learning, the problem of analyzing and visualizing big data has not yet been completely solved. In this regard, in this work, the task is to develop such a computer system that allows to visualize large amounts of information that are not available for quick and primary analysis in their original form due to their significant volume. This article presents the author's understanding of Gaussian numbers [1] from the standpoint of genetic coding. Gaussian numbers (complex numbers whose real and imaginary parts are integers) have long attracted the attention of researchers. The author initially set the task to represent the DNA molecule with exactly two-dimensional (complex) numbers. This turned out to be possible due to the system of orthogonal Walsh functions encoding the physico-chemical properties of nucleotides when displaying the molecular genetic sequence.
Earlier, to optimize mental work, we developed an algorithm for automatically generated cognitive graphics based on the neurosemantic approach [2-3] for transforming a linear information flow into a hierarchical graph using the mechanism of auto-structuring [4]. This contributes to the measurement, IOP Publishing doi:10.1088/1757-899X/1129/1/012048 2 assessment and regulation of information, the achievement of the adequacy of mental stress and prevention of neuro-emotional overstrain of operators interacting with large amounts of information.
In 1990, John Archibald Wheeler proposed the "it from bit" doctrine, according to which information is a fundamental concept in physics [5]. The Hawking paradox about the disappearance of information in black holes is known [6]. At the same time, information is one of the occupational health risk factors. In [7] was mentioned that from 64% to 90% of computer users experience computer syndrome as a reaction of the body to work with a computer (in particular, it is pain in the back, neck and arms). At the same time, new methods of professional ergonomics are being developed for human interaction with various information systems [8]. In [9], data on clinical, pathogenetic, prognostic and therapeutic aspects of computer syndrome were studied. In [10][11][12], cognitive factors that must be taken into account when developing decision support systems based on large amounts of data are identified, in particular, cognitive limitations of a person, manifested in the context of data analysis, are considered. Thus, the principles of cognitive ergonomics can help to reduce health risks and optimize a mental work when operators interact with information flows with large amounts of data.
From the point of view of cognitive ergonomics, the currently developing methods of automatically generated computer graphics for the analysis of big data are of interest. In [13], a system for automatic text visualization was developed, which receives text in a natural language as input and creates an interactive three-dimensional scene on its basis. In [14], a method for obtaining information from a text using text network analysis is presented. The automatically generated structure, in combination with other indicators, can be used to determine the level of cognitive diversity of the text.
Technologies for the analysis of genetic information are also being developed. OrganellarGenomeDRAW [15] is a tool for drawing graphic maps of plastid and mitochondrial genomes with access to a local copy of the NCBI RefSeq organelle genome database, which allows visualizing large sets of organelle genomes.

Calculation model: genometric approach to big data analysis
Genometry is a group of DNA algorithms for multiscale information display developed by the author through the use of a system of orthogonal Walsh functions with the ability to visualize in various parametric spaces. The basic algorithm has three main steps. 1) Scaling. A sequence of characters from the set {A, G, C, T} or {A, G, C, U}, encoding nitrogenous bases, is divided into fragments of equal length N, where N is a free parameter of the algorithm -the length of "semantic units". The resulting fragments of equal length will be called N-measures or N-plets. 2) Parametrization. Taking into account the system of genetic sub-alphabets: G = C "3 hydrogen bonds" / A = T "2 hydrogen bonds"; C = T "pyrimidines" / A = G "purines"; A = C "amino" / G = T "keto" sequence of nitrogenous bases can be represented as three binary sequences, consisting of zeros and ones. The choice of encoding method (what is considered to be zero or one) affects the symmetry transformations of the final rendering. 3) Visualization. The resulting binary record of fragments is their representation in the form of three sequences of decimal or other uniquely identifying values (code frequencies, the number of certain symbols in each N-measure, and other characteristic parameters). So, converting binary N-plets to decimal numbers allows to display them in the selected coordinate system. Other numerical interpretations of N-plets for each of the sub-alphabets are possible, considered in particular at works [16-17] based on the matrix genetics of prof. Sergey Petoukhov [18][19]. Therefore, we are talking about a family of DNA-algorithms. The obtained values set the coordinates of points in visualization parametric spaces.
The study of long genetic nucleotide sequences using a genometric approach can be attributed to the problem of visual perception of big data. A conceptual diagram with options for presenting genetic information to the operator's visual analyser is shown in figure 1.
It should be noted that any information flow can be represented as a sequence of characters of some finite alphabet. An arbitrary signal can be recoded and represented in a quaternary code using the genetic alphabet of nucleotides by analogy with how information is stored in a computer's memory in  The flow of information to the operator's visual analyzer is shown schematically. The left hemisphere is responsible for the logic and analysis of textual information, the right hemisphere is responsible for the genometric image. The DNA algorithm allows you to additionally use the right hemisphere to increase mental productivity.  The result of genometric processing of arbitrary information is a visualization, which can be interpreted as a sign or symbol. Various visualization options make it possible to evaluate the symbolic composition of the information signal and track its changes in different sections with the possibility of scaling at different values of N. Also, using scaling, you can evaluate the structure of a particular section of the signal.
Thus, DNA algorithms make it possible to automatically generate cognitive computer graphics for multiscale visualization of the internal structure of signals of various nature. This can be useful for visualizing and identifying areas with different information content within a particular information signal, as well as for comparative analysis of various signals.

Results and discussion. Perception of information flows and big data in a text form
At figures 1-4 the results of the transformations that reveal the characteristic structure of the original information is shown. As studies have shown [16], a randomly generated symbolic sequence gives a genometric pattern, all the points of which are scattered chaotically. In this regard, the DNA algorithm can be used to visually evaluate frequency distributions in pseudo-random signals. This can be used, in particular, to evaluate the performance of noise generation algorithms. An example of visualization of the frequency distribution of the signal is shown in figure 3. It can be seen that, according to two subalphabets, the frequencies are periodic, which indicates the periodic nature of the original signal. The ability to find the internal ordering of the information signal in the form of an individual fractal structure of genometric visualization is an important feature of DNA-algorithms. In this regard, the use of DNA-algorithms for the primary study of big data presented in text form can be used as a tool for the cognitive computer graphics. This makes it possible to apply this method to expand the capabilities of operators' interaction with large data of arbitrary nature, as well as to improve the ergonomic qualities of specialized graphical user interfaces (GUIs). One of the promising options for using the results of research in the field of ergonomics of perception based on DNA-algorithms is the possibility of help of non-specialists to compare the genetic and phenotypic data of living organisms, as well as to search for previously unknown patterns in other subject areas. The methodology for researching visual information with the involvement of a large number of users has been shown to be effective and is known as "crowd surfing". It should also be noted the new possibility of labeling food and seeds (by analogy with a barcode) in addition to simple genometric annotation of genomes.
Due to the fact that the features in the nucleotide composition of DNA/RNA of various organisms are clearly visible, it can be concluded that an ergonomic visualization of molecular genetic data based on the proposed approach is possible. For the analysis of textual information, DNA-algorithms are also applicable, but it seems that, unlike genetic texts, texts in natural languages do not differ from each other so much as different DNAs. At the same time, large volumes of textual data of various subjects in natural or programming languages, as well as binary data stored in the memory of computers, can have genometric representations available for visual analysis in various parametric spaces. Visualizations of sound files, biomechanical, biomedical signals, etc. are of practical interest.

Genometric Information System
Genometric information system is an information system used to collect, analyse, store and process information using DNA-algorithms. One of the important properties of genometric information systems is the possibility of using technologies of collective intelligence and knowledge engineering. Mind is viewed as a more general category than intelligence. Mind is inherent only in living organisms. Three directions can be distinguished based on a single technical and organizational solution:  scientific analysis of big data incl. biological, literary and musical;  application of DNA visualization technologies in the educational process;  the use of the results and methods in the field of scientific art to change the way of perception of natural and creative objects. Since the method can be positioned within the framework of the direction of scientific art, it becomes possible to attract humanitarian organizations to conduct virtual exhibitions. Thus, DNA-algorithms allow the creation of new ergonomic instruments for scientific research. Opportunities are also emerging to popularize scientific fields such as molecular genetics and Big Data through the ability to generate specific digital media content.

Conclusion
The considered methodology makes it possible to operate with genometric mappings at the level of semantics and semiotics of Big Data. This is possible because the result of processing any information signals using-DNA algorithms are objects of discrete geometry, the projections of which can carry the function of characteristic identification symbols. This is useful in particular for enhancing the ergonomics of graphical user interfaces. At the same time, the possibility of internal interpretation of genometric representations is remains, since DNA-algorithms allow displaying the internal structure of genetic coding.
It is shown that genetic nucleotide sequences (as sequences of symbols of the genetic alphabet) and texts in natural language (as sequences of symbols of the natural language alphabet) lend themselves to visual perception, which contributes to an increase in mental productivity of operators that are interacting with Big Data.
The question of choosing a method for coding information is actual, since this can affect the result of processing by DNA-algorithms. For example, natural language texts can be analysed at the phonetic and morphological levels. In this regard, for the further systematization of genometric mappings, a coding method is important, taking into account the peculiarities of the internal structure of the information signals themselves. It is of particular interest to take into account the cyclic fluctuations of IOP Publishing doi:10.1088/1757-899X/1129/1/012048 6 various biological processes, such as heart rate cycles, circadian rhythms, etc., analysis of the timing structure of musical works, speech intensity, etc. This is possible using the presented approach.