Cross Modal Focal Loss for RGBD Face Anti-Spoofing

Automatic methods for detecting presentation attacks are essential to ensure the reliable use of facial recognition technology. Most of the methods available in the literature for presentation attack detection (PAD) fails in generalizing to unseen attacks. In recent years, multi-channel methods have been proposed to improve the robustness of PAD systems. Often, only a limited amount of data is available for additional channels, which limits the effectiveness of these methods. In this work, we present a new framework for PAD that uses RGB and depth channels together with a novel loss function. The new architecture uses complementary information from the two modalities while reducing the impact of overfitting. Essentially, a cross-modal focal loss function is proposed to modulate the loss contribution of each channel as a function of the confidence of individual channels. Extensive evaluations in two publicly available datasets demonstrate the effectiveness of the proposed approach.

Introduction
While face recognition technology has become a ubiquitous method for biometric authentication, the vulnerability to presentation attacks (also known as "spoofing attacks") is a major concern when used in secure scenarios [9], [12]. These attacks can be either impersonation or obfuscation attacks. Impersonation attacks attempt to gain access by masquerading as someone else and obfuscation attacks attempt to evade face recognition systems. While many methods have been suggested in the literature to address this problem, most of these methods fail in generalizing to unseen attacks [10]. In a practical scenario, it is not possible to anticipate all the types of attacks at the time of training a PAD model. Moreover, a PAD system is expected to detect new types of sophisticated attacks. It is therefore important to have unseen attack robustness in PAD models.
The majority of the literature deals with the detection of these attacks with RGB cameras. Over the years, many feature-based methods have been proposed using color, texture, motion, liveliness cues, histogram features [7], local binary pattern [25], [8] and motion patterns [3] for performing PAD. Recently several CNN based methods have also been proposed including 3D-CNN [13], part-based models [22] and so on. Some works have shown that using auxiliary information in the form of binary or depth supervision improves performance [4,14]. However, most of these methods have been designed specifically for 2D attacks and the performance of these methods against challenging 3D and partial attacks is poor [24]. Moreover, these methods suffer from poor unseen attack robustness.
The performance of RGB only models deteriorates with sophisticated attacks such as 3D masks and partial attacks. Due to the limitations of visible spectrum alone, several multi-channel methods have been proposed in literature such as [30], [12], [31], [11], [2], [6], [5], [15][16][17][18] for face PAD. Essentially, it becomes more difficult to fool a multichannel PAD system as it captures complementary informa-tion from different channels. Deceiving different channels at the same time requires considerable effort. Multi-channel methods have proven to be effective, but this comes at the expense of customized and expensive hardware. This could make these systems difficult to deploy widely, even if they are robust. A variety of channels are available for PAD, e.g., RGB, depth, thermal, NIR spectra, SWIR spectra, ultraviolet, light field imagery, etc. Out of these various modalities, we find that RGB-D devices are commercially available and are quite affordable, making it possible to deploy them in real-world scenarios. The Intel RealSense family of devices, Microsoft Kinect, and the OpenCV AI Kit (OAK-D) [1] are examples of standard devices that do not require any additional effort to obtain multi-channel images. Due to the wide availability of these channels in an integrated package, we choose RGB and Depth as the two channels to be used in this work. However, the proposed framework can be trivially extended to any combinations of channels.
Even when using multiple channels, the models tend to overfit to attacks seen in the training set. While the models could perform perfectly in attacks seen in the training set, degradation in performance is often observed when confronted with unseen attacks in real-world scenarios. This is a common phenomenon with most of the machine learning algorithms, and this problem is aggravated in case of a limited amount of training data. The models, in the lack of strong priors, could overfit to the statistical biases of specific datasets it was trained on and could fail in generalizing to unseen samples. Multi-channel methods also suffer from an increased possibility of overfitting as they increase the number of parameters due to the extra channels.
In this work, we address this issue in two different directions. First, we use a multi-head architecture, which follows a late fusion strategy to combine different channels of information. Instead of joining the representations into a joint final node, we keep three different heads separately for the individual branches and the joint branch, this can be viewed as a form of architectural regularization. The proposed architecture can be found in Fig. 1. This enables us a way to supervise individual channels together with the joint representation, ensuring robust representation would be learned in individual as well as joint branches. Secondly, we propose a cross-modal focal loss function to supervise the individual channels that modulate the loss function factoring in the confidence of the channels present.
The main contributions of this work are listed below: • A frame-level RGB-D face PAD method is proposed which operates on synchronously captured RGB-D samples.
• A new loss function called cross-modal focal loss (CMFL ) is proposed, which can be used to supervise individual channels in a multi-stream architecture.
• Though the model is trained for a multi-channel scenario, it can also be deployed with individual channels by just using the score from the head corresponding to the available channel.
• We show the efficacy of the proposed framework in two publicly available datasets consisting of a wide variety of challenging unseen attacks.
Additionally, the source code and protocols to reproduce the results are available publicly * .

Proposed approach
Different stages of the proposed PAD framework are described in this section.

Preprocessing
The PAD pipeline acts on the cropped facial images. For the RGB image, the preprocessing stage consists of face detection and landmark localization using the MTCNN [32] framework, followed by alignment. The detected face is aligned by making the eye centers horizontal followed by resizing them to a resolution of 224 × 224. For the depth image, a normalization method using the median absolute deviation (MAD) [27] is used to normalize the face image to an 8-bit range. The raw images from RGB and depth channels are already spatially registered so that the same transformation can be used to align the face in the depth image.

Network architecture and loss function
This section details the proposed network architecture and loss function.

Architecture
From the prevailing literature, it has been observed that multi-channel methods are robust against a wide range of attacks [15][16][17][18]. Broadly, there are four different strategies to fuse the information from multiple channels, they are 1) early fusion, meaning the channels are stacked at the input level (for example, MC-PixBiS [18]). The second strategy is late fusion, meaning the representations from different networks are combined at a later stage similar to feature fusion (for example MCCNN [17]), a third strategy is score level fusion where individual networks are trained separately for different channels and score level fusion is performed on the scalar scores from each channel. A fourth strategy is a hybrid approach where information from multiple levels is combined as in [28].
Though multiple channels can perform well with a wide variety of attacks, they tend to overfit to known attacks when all channels are used together and trained as a binary classifier. To avoid this, we propose a multi-head architecture that follows a late fusion strategy. The architecture of the proposed network is shown in Fig. 1. Essentially, the architecture consists of a two-stream network with separate branches for the component (RGB and Depth) channels. The embeddings from the two channels are combined to form the third branch. Fully connected layers are added to each of these branches to form the final classification head. These three heads are jointly supervised by a loss function which forces the network to learn discriminative information from individual channels as well as the joint representation, reducing the possibility of overfitting. The multi-head structure also makes it possible to perform scoring even when a channel is missing at test time, meaning that we can do scoring with RGB branch alone (just using the score from the RGB head) even if the network was trained on RGB-D.
The branches are comprised of the first eight blocks from DenseNet architecture (densenet161) proposed by Huang et al. [19]. In the DenseNet architecture, each layer is connected to every other layer, reducing the vanishing gradient problem while reducing the number of parameters. We used pre-trained weights from the Image Net dataset to initialize the individual branches. The number of input channels for the RGB and depth channels has been modified to 3 and 1 for the RGB and depth channels, respectively. For the depth branch, the mean values of three-channel weights are used to initialize the weights of the modified convolutional kernels in the first layer. In each branch, a global average pooling (GAP) layer is added after the dense layers to obtain a 384-dimensional embedding. The RGB and depth embeddings are concatenated to form the joint embedding layer. A fully connected layer, followed by a sigmoid activation is added on top of each of these embeddings to form the different heads in the framework. At training time, each of these heads is supervised by a separate loss function. At test time, the score from the RGB-D branch is used as the PAD score.

Cross Modal Focal Loss function (CMFL )
Having individual heads make it possible to train a multichannel model with the capability to handle missing channels at test time. Now, one naive way to supervise this network would be to supervise the individual branches with binary cross-entropy (BCE).
However, the usage of BCE for individual channels may not be ideal. The issue is illustrated as follows; we can consider different channels as different views of the same sample, and for some attacks, it may not be possible to distinguish it just from one view. It is possible that the images of some attacks would look perfectly like bonafide sam- ples when viewed in just one channel. For example, facial makeup when viewed in depth channel would look exactly like the depth map of a bonafide sample. The naive way of supervising the depth branch with BCE in such cases might lead to overfitting. However, in the same scenario, discriminating makeup would be more obvious in RGB and joint representations. From this example, it can be seen that supervising the individual branches separately might not result in robust decision boundaries. One way to approach this issue is to use the prediction probabilities from the current branch and the other branch to change the loss contributions of samples in each branch. We propose a cross-modal focal loss function to supervise the individual channels, which modulates the loss function based on the confidence of the current and alternate channel present. For each branch, the samples which could be classified correctly should be well separated in the score space. At the same time, we encourage the individual branches to produce unsure scores when there is not enough discriminatory information, rather than overfitting to some statistical bias in the training data. However, this applies only when the other branch can confidently classify the sample correctly.
More formally, consider a binary classification problem where the samples are multi-modal i.e., each sample is a pair of images or features that capture different views with complementary information. Now assume that the classification problem cannot be done with a single channel alone (or is a very hard problem). Combining the features from both the channels and using a learning strategy using the joint features could provide a solution for this. However, this could lead to overfitting and cannot handle missing channels at test time.
If we use BCE on individual branches, the loss will penalize heavily the samples which cannot be classified with the information available with the specific channel. In such a scenario, the model might start to overfit to the biases in the dataset to minimize the loss function, resulting in an over-fitted model.
To avoid this, we propose cross-modal focal loss (CMFL) to supervise the individual channels. The core idea is that, when one of the channels can correctly classify a sample correctly with high confidence, then the loss contribution of the sample in the other branch can be reduced. If a channel can correctly classify a sample confidently, then we don't want the other branch to penalize the model more. CMFL forces each branch to learn robust representations for individual channels, which can then be utilized with the joint branch, effectively acting as an auxiliary loss function.
The idea of relaxing the loss contribution of samples correctly classified is similar to the Focal Loss [23] used in object detection problems. In Focal Loss, a modulating factor is used to reduce the loss contributed by samples that are correctly classified with high confidence. We use this idea by modulating the loss factoring in the confidence of the sample in the current and the alternate branch.
Consider the expression for cross-entropy (CE) in a binary classification problem: Where y ∈ {0, 1} denotes the class label (y:0 attack, y:1 bonafide) and p ∈ [0, 1] is the probability of the class. We follow a similar notation p t as in [23], which is the probability of the target class: and write CE(p, y) = CE(p t ) = − log(p t ).
In the α-balanced form CE loss can be written as: The standard α-balanced focal loss (FL) [23] adds a modulating factor (1 − p t ) γ to the cross entropy loss, with a tunable focusing parameter γ ≥ 0, making the loss formulation as.
Consider the two-stream multi-branch multi-head modal in Fig. 3. X p and X q denotes the image inputs from different modalities, and E p , E q , and E r denotes the corresponding embeddings for the individual and joint representations. In each branch, after the embedding layer, a fully connected layer (followed by a sigmoid layer) is present which provides classification probability. The variables p, q and r denote these probabilities. The proposed Cross Modal Loss Function (CMFL) is given as follows: The function w(p t , q t ), depends on the probabilities given by the channels from two individual branches. This modulating factor should increase as the probability of the other branch increases, and at the same time should be able to prevent very confident mistakes. Hence for this study, we use the harmonic mean of both the branches weighted by the probability of the other branch. This reduces the loss contribution when the other branch is giving confident predictions. And the expression for this function is given as: Note that the function w is assymetric, i.e., the expression for w(q t , p t ) is: Meaning the weight function depends on the probability of the other branch. Now we use the proposed loss function as auxiliary supervision,and the overall loss function to minimize is given as: We have set the value of λ non-optimally as 0.5 for this study. The loss curve for the cross-entropy and the proposed loss is shown in Fig. 2. When the probability of other branch is zero, then the loss is equivalent to standard crossentropy. The loss contribution is reduced when the other branch is able to correctly classify the sample. ie, when an attack example is misclassified by network CN N p , the network CN N p is penalized unless model CN N q can classify the attack sample with high confidence. As the w(p, q) → 1 the modulating factor goes to zero, meaning if one channel is able to classify it perfectly, then the other branch is less penalized. Also, the focussing parameter γ can be adapted to change the behaviour of the loss curve. We used an empirically obtained value of γ = 3 in all our experiments.
Without the loss of generality, the framework can be extended to other multi-channel classification problems as well.

Implementation details
We performed data augmentation during the training phase with random horizontal flips with a probability of 0.5. The combined loss function is minimized with Adam Optimizer [21]. A learning rate of 1 × 10 −4 was used with a weight decay parameter of 1 × 10 −5 . We used a mini-batch size of 64, and the network was trained for 25 epochs on a GPU grid. During the evaluation of the model, the scores from the RGB-D head was used to calculate the final PAD score. The proposed architecture has about 6.39M parameters and about 9.16 GFLOPS. The architecture and the training framework were implemented using the PyTorch [29] library.

Datasets used
We have used two publicly available datasets for the experiments, namely WMCA and HQ-WMCA , which contains a wide variety of 2D, 3D, and partial attacks.

WMCA dataset
The Wide Multi-Channel presentation Attack (WMCA ) [17] database contains a wide variety of 2D and 3D presentation attacks, with a total of 1679 video samples from 72 subjects. Multiple channels collected synchronously, namely color, depth, infrared, and thermal channels, collected using two consumer-grade devices, Intel ® RealSense™SR300 (for color, depth and infrared), and Seek Thermal Com-pactPRO (for the thermal channel) is available with this database. Though four different channels are available in this database, in this work, we focus on the RGB and depth data obtained from the Intel ® RealSense™SR300 device.

HQ-WMCA dataset
The High-Quality Wide Multi-Channel Attack (HQ-WMCA ) [18,26] dataset consists of 2904 short multi-modal video recordings of both bonafide and presentation attacks. This database again consists of a wide variety of attacks including both obfuscation and impersonation attacks. Specifically, the attacks considered are print, replay, rigid mask, paper mask, flexible mask, mannequin, glasses, makeup, tattoo, and wig (Fig. 4). The database consists of recordings from 51 different subjects, with several channels including color, depth, thermal, infrared (spectra), and short-wave infrared (spectra). In this work, we consider the RGB channel captured with Basler acA1921-150uc camera and depth image captured with Intel RealSense D415.

Protocols
Since both of the datasets contains a wide variety of attacks, we have created leave-one-out (LOO) attack protocols individually for both of the datasets. Specifically, one attack is left out in the train and development set and the evaluation set consists of bonafide and the attack which was left out in the train and development set. This constitutes the unseen attack protocols or zero-shot attack protocols. The performance of the PAD methods in these protocols gives a more realistic estimate of their robustness against unseen attacks in real-world scenarios. Further, for cross-dataset experiments, we have created grandtest protocols in both of the datasets which consist of attacks distributed in the train, development, and test sets (with disjoint identities across folds).

Metrics
For the evaluation of the algorithms, we have used the ISO/IEC 30107-3 metrics [20], Attack Presentation Classification Error Rate (APCER), and Bonafide Presentation Classification Error Rate (BPCER) along with the Average Classification Error Rate (ACER) in the eval set. We compute the threshold in the dev set for a BPCER value of 1%.
For cross-database testing, Half Total Error Rate (HTER) is adopted following the convention in [16], which computes the average of False Rejection Rate (FRR) and the False Acceptance Rate (FAR). HTER is computed in the eval set using the threshold computed in the dev set using the equal error rate criterion (EER).

Baseline methods
For a fair comparison with state of the art, we have implemented 3 different multi-channel PAD approaches from literature for the RGB-D channels. Besides, we also introduce the proposed multi-head architecture supervised with BCE alone, as another baseline for comparison. The baselines implemented are listed below. MC-PixBiS: This is a CNN based system [14], extended to multi-channel scenario as described in [18] trained using both binary and pixel-wise binary loss function. This model uses RGB and depth channels stacked together at the input level. MCCNN-OCCL-GMM: This model is the multi-channel CNN system proposed to learn one class model using the one class contrastive loss (OCCL) and Gaussian mixture model as reported in [16]. The model was adapted to accept RGB-D channels as the input. MC-ResNetDLAS: This is the reimplementation of the architecture from [28], which won the first prize in the 'CASIA-SURF' challenge, extending it to RGB-D channels, based on the open-source implementation * . We used the initialization from the best-pretrained model as suggested in [28] followed by retraining in the current protocols using RGB-D channels.

RGBD-MH-BCE:
This uses the newly proposed multi-head architecture shown in Fig.1, where all the branches are supervised by binary cross-entropy (BCE). In essence, this is equivalent to setting the value of γ = 0, in the expression for the cross-modal loss function. This is shown as a baseline to showcase the improvement by the new multi-head architecture alone and to contrast with the performance change with the new loss function.
Proposed: This is our final proposed framework, it uses the multi-head architecture we proposed as shown in Fig.1, together with the newly proposed loss function. More specifically, the individual channel branches are supervised by the newly proposed cross-modal focal loss function (CMFL ). The details of the parameters of the baseline methods can be found in our open-source implementation † .

Experiments
We have conducted experiments in both WMCA and HQ-WMCA datasets, specifically the leave-one-out protocols to evaluate the robustness against unseen attacks. The results are described in the following sections.

Results in WMCA dataset
The performance of the proposed system and baselines in the LOO protocols of WMCA are shown in Table 1. It can be seen that the baselines MCCNN-OCCL-GMM and MC-ResNetDLAS performs poorly in the challenging unseen attack scenarios. Among baselines, the MC-PixBiS model * Available from: https://github.com/ AlexanderParkin/ChaLearn_liveness_challenge † Will be available upon acceptance.
achieves the best performance with an average ACER of 10.5±16.7%. The model with the new architecture, RGBD-MH-BCE achieves reasonable performance as compared to the baselines. However, the proposed method, with the new CMFL loss achieves the best mean accuracy. Also, comparing with RGBD-MH-BCE, the proposed loss function shows a clear improvement in performance achieving an average ACER of 7.6 ± 11.2%.

Results in HQ-WMCA dataset
The HQ-WMCA dataset is consists of more challenging attacks as compared to WMCA . Specifically, there are different types of facial tattoos and partial attacks which occupy only a part of the face. These attacks are much harder to detect when they are not seen in the training set, as they appear very similar to bonafide samples. The experimental results in HQ-WMCA are tabulated in Table 2. Similar, to WMCA database, the baselines MCCNN-OCCL-GMM and MC-ResNetDLAS does not perform well in the LOO protocols of HQ-WMCA database. In addition, the MC-PixBiS method, which achieved reasonable performance in WMCA performs poorly HQ-WMCA dataset. This could be due to the challenging nature of the attacks in the database. It can be seen that the new multi-head architecture proposed, RGBD-MH-BCE, already improves the results as compared to all the baselines with an average ACER of 13.3 ± 16.5. Further, with the addition of the CMFL loss, the ACER further improves to 11.6 ± 14.8%. The results indicate that the proposed architecture already improves the performance in challenging attacks, and the proposed loss further improves the results achieving state of the art results in the HQ-WMCA dataset.
To summarize, the newly proposed multi-head architecture itself improves the performance over other baselines. The addition of the CMFL loss further improves the performance in both WMCA and HQ-WMCA datasets.

Ablation studies
To further analyze the performance, we conduct various ablation studies using the proposed approach. We carry out these experiments in the HQ-WMCA dataset as it contains more challenging attacks.  Table. 3. It can be seen that the optimal value of γ = 3. The model with γ = 0 is equivalent to BCE loss on each of the branches.  Table 4, it can be clearly seen that the performance improves, as compared to using BCE even when using a single channel at the time of deployment. This shows that the performance of the system improves when the loss contributions of samples which are not possible to classify by that modality are reduced. Forcing the individual networks to learn a decision boundary leads to overfitting resulting in poor generalization. In the proposed approach, the network can learn a robust classifier for individual channels as well. This observation opens up the possibility to use multichannel datasets at training time and to deploy the model for the RGB channel only. This could be of practical importance in scenarios where using additional hardware at deployment time is prohibitive, requiring PAD systems to be deployed using legacy RGB cameras.  Here we analyze the score distributions of the proposed model in the grandtest − c protocol of the HQ-WMCA dataset. As opposed to LOO protocols, the grandtest − c consists of different types of attacks roughly equally distributed in the train, development, and evaluationf olds.
The multi-head model provides three different scores from each of the branches, i.e., RGB, depth, and RGB-D branches. Here we show the distribution of scores for different types of attacks in the evaluation set of the HQ-WMCA dataset.
From Fig. 5, it can be seen that the attacks are more compact in RGB head compared to that of depth channel. In depth channel, where classifying attacks are harder, the bonafide distribution is shifted to the left. This can also be seen in the score distribution of attacks in depth channel, i.e., while attack scores are not as low as we desire, still, it is far from bonafide, the loss does not push the depth branch to classify all the attacks correctly, and it performs well on attacks where depth channel contains discriminatory information. Looking at the final 'RGB-D' part, it can be seen the proposed loss function makes the classifications using the joint representations from individual channels robust. To summarize, the proposed framework encourages the individual branches to produce non-confident scores for attacks that are confidently classified by the other branch. The discriminative representation from the joint branch, learned in this fashion, results in a robust PAD system.

Cross database evaluation
To evaluate the robustness in cross-database scenarios we have performed cross-database experiments between models trained in WMCA and HQ-WMCA . This evaluation amounts to both cross sensor and unseen attack at the same time which is much more challenging than typical crossdatabase evaluations. WMCA database uses the Intel Realsense SR300 camera which returns both RGB and depth streams. IntelRealsense SR300 uses dot pattern projection for depth computation. Whereas in HQ-WMCA the RGB channel is acquired using a high-quality Basler acA1921-150uc camera and the depth image was captured with Intel RealSense D415 (which uses stereo for computing depth). The mismatch between the sensors and quality between the sensors used in these datasets makes the performance degrade. The results from the cross-dataset test are tabulated in Table. 5. The models were trained on the grandtest protocol of each dataset and was evaluated on the dev and eval set of the other dataset. We report both the intraand cross-database performance for the sake of completeness. It can be seen that in the intra dataset evaluations, the proposed method achieves good performance in both the datasets. However, in the cross-database evaluations, the performance degrades. The mismatch between the sensors could be one reason for the degradation in performance. Performance is slightly better when the model trained in HQ-WMCA is evaluated on WMCA which could be due to the wider variety of attacks present in the training set. Moreover, it can be seen that methods with better perfor-mance in the target dataset, perform worse in the source dataset.

Discussions
From the experiments in WMCA and HQ-WMCA , it can be seen that the proposed approach outperforms the state of the art methods. We have also shown that the proposed approach is useful in cases where multiple channels are available at training time and when there is a need to deploy with individual channels. Even though the proposed approach works well in intra dataset scenario, cross-database performance needs further improvement. As future work, more preprocessing and data augmentation strategies could be developed to alleviate the discrepancies between datasets to improve the cross-database performance. While we have selected RGB and depth channel for this study, mainly due to the availability of off the shelf devices consisting of these channels, it is trivial to extend this study to other combinations of channels as well, for instance, RGB-Infrared, and RGB-Thermal.

Conclusions
In this work, we proposed a new architecture for RGB-D presentation attack detection, applicable to other multichannel classification problems as well. Also, we proposed a new cross-modal focal loss function which could be useful for two-stream networks. The proposed cross-modal focal loss function modulates the loss contribution of samples based on the confidence of individual channels. The proposed framework can be trivially extended to multiple channels and different classification problems where information from one channel alone is inadequate for classification. This loss forces the network to learn complementary, discriminative, and robust representations for the component channels. The structure of the framework makes it possible to train models using all the available channels and to deploy with a subset of channels. Extensive evaluations in two publicly available datasets demonstrate the effectiveness of the proposed approach.


The Innovative Model of Runway Sustainable Management on Smaller Regional Airports

: Airport runways are exposed to large trafﬁc loads and other external factors which cause their transformation. Monitoring the state of the unevenness and determining the runway defectiveness is a complex procedure which includes fast and qualitative measurements of the execution of ﬂatness, their evaluation and the determination of the level of runway defectiveness. For this purpose, an improved process of determining airport runway defectiveness is proposed, based on geodetic survey and the management information system prototype, which enables evaluating the results of the executed survey and suggesting the runway maintenance measures. A similar analysis of the considered data on the example of a smaller, regional airport in Maribor, Slovenia, was performed with the help of our own algorithm, which, on various platforms, enables the tabular and graphical display of surface irregularities. The algorithm prototype allows the automated monitoring and evaluating of the unevenness or runway states, determining their defectiveness and advising adequate action. With the help of the suggested method and developed information system prototypes, it is possible to conﬁrm the demand for professionally adequate and on-time planning of necessary maintenance works, with which it is surely possible to prevent an eventual unexpected closing of the runway and related costs. The suggested method is also usable in evaluating the state of other trafﬁc infrastructure.

Introduction
Logistics systems, including those regarding airport infrastructure, in most cases, function effectively and successfully only if all of the corresponding elements function effectively and successfully. In their research [1,2], note that improper and insufficient logistics infrastructure is the second most important factor affecting logistics. In [3], the authors suggest a guidance model in the case of accidents or disorders which limit the function of logistics (including those regarding airport) infrastructure. The extraction of such data is generally linked to automated procedures, and the demand for their promptness, exactness and availability at any time encourages the development of systems which allow automated extraction and processing in real time [4][5][6].
Aircraft traffic has a significant role in the global economy [7]. The authors of [8] point out that improving airport infrastructure can reduce air traffic costs by up to 15%.
Traffic safety is key in aircraft traffic. Most international organizations work to provide traffic safety in aircraft traffic. Everybody is aware that a high level of safety provision in aviation means a safe and uninterrupted occurrence of aircraft traffic. Due to this, it is necessary to unify, as much as possible, all the procedures which are related to providing traffic safety and establish the generally well-established procedures (methodology). Considering the need to design a global safety plan, the International Civil Aviation Or-• events on the area intended for aeroplane landing and take-off, and in their surroundings; • events related to loss of control over the aircraft while flying; • events while flying in the take-off or landing phase.
The present article considers in detail the first group of events, which can occur on the runway and in its surroundings. Among others, in this group belong the following events with a high level of risk to develop disorders and which are directly connected with the stage (status) of runways flatness: • unusual contact with the runway when landing-rough landing (hereinafter: ARC); • the trip from the runway (hereinafter: RE); • loss of control over the aircraft on the runway or taxiway (hereinafter: LOC-G) and • collision with the obstacles when aircraft take-off or landing (hereinafter: CTOL).
RE events present the biggest part of plane crashes and often surpass 25% of all plane crashes in commercial flights [7,[9][10][11]. Deformations on the runway pavement surface obstruct pilots' aircraft management and cause onboard vibrations, which makes it difficult to correctly read the measure instruments, cause mechanical defects on the aircraft and diminish the contact of the tire with the floor, which affects the correct action of the braking system. Most difficulties present dents and the production of aquaplaning [12].
Therefore, the safe occurrence of aircraft traffic is, to a great extent, connected with providing safety on the runway and its surroundings [13][14][15][16][17]. In order for these disorders to be limited and diminished to the greatest extent possible, the runways must also be in the best possible state (adequately maintained). Runways must provide adequate load capacity, good surface friction and a smooth aircraft run. All of these demands depend on adequate runway maintenance. Well maintained runways are very important for safe aircraft take-off and landing [18].

Monitoring the Runways' Conditions
Detection of runways' unevenness and determining their shape, dimensions and the causes of why these occur is a complex process which demands an interdisciplinary approach [19]. The interdisciplinary approach to problem solving is used in many modern scientific types of research, since it is necessary to research the exposed issue from a number of aspects. Recently, automated extraction and spatial data processing on roadway and runway unevenness are the subjects of many types of research [20][21][22][23][24][25][26]. The authors, in research, mainly focus on interpretation, pavement surface and runway analysis and the development of algorithms for automated recognition or extraction of unevenness (cracks, holes etc.) from the recordings, determining the dimensions of the unevenness (width and depth) and their classification [27][28][29]. The use of geodetic methods in determining the vertical deviations and detecting deformations on runway surfaces and predicting the occurrence of new deformations is not yet sufficiently researched and not many research pieces on this topic can be found in the open literature [30].

Management Information Systems
Airports are large, complex, dynamic systems which are constantly under the influences of internal and external participants. This demands the effective management of all processes which are necessary for uninterrupted airport function. The authors of [31][32][33][34][35] researched the use and prevalence of some information systems which are used at airports in the United States of America on the aerial and terrestrial parts of the airport. They note, in the research, that there are many types of information systems which are management support. The airports' equipment with them is very different and, to the greatest extent possible, it depends on the airport purpose and size [36]. In the 1990s, the Airport Associ-ation Council International showed the need to establish information systems intended for management maintenance; the authors of [37] also note that the establishment of such information systems is key for effective maintenance.
In the present work, we discuss the maintenance area, we focused in detail on logistics infrastructure maintenance the runway maintenance and the development of the process decision-making model, which will support basic maintenance process and direct participants in the maintenance process, and will also be used as management support for decision-making regarding the airport infrastructure maintenance area, mostly in runway maintenance. Infrastructure conditions data are a critical part of such models, which are extracted with measurements in the field [38,39]; in addition, the selection of the process decision-making models must be based on qualitative data the on the state of the infrastructure. Due to this fact, we focus in detail on the process decision-making model development in order to determine the deformations of the runways; we integrated it into the process decision-making maintenance model. This was necessary to establish Airport Pavement Management System (APMS). In the development of the APMS, we considered the demand that it mostly needs to be useful on smaller airports:by its extent, as well as the loading, and that it allows collecting, storing and maintenance of more types of measurement data and analysis performances.
APMS and PMS are built from six basic building blocks [40]: inventory of infrastructure characteristics (construction data, traffic data and infrastructure state data), the assessment of the infrastructure state (for prediction the future states, alternatives, decision making), the database (overview the results, executed and planned activities), the data analysis (choosing the optimal alternative), the results preparation (reports) and the feedback.
For logistics transport systems management, the decision-makers have computersupported decision-making systems called Traffic Management Systems (TMS). Due to the requirement of airport specificity, the Air Transportation System (ATS) have developed parallelly. For adequate and timely road maintenance, the decision-makers use various technical and economic analyses when creating the maintenance strategies which are implemented in the Pavement Management System (PMS). Due to the airport specificity, the APMS is also used for the need to create adequate maintenance strategies. These include the procedures which are of help when evaluating and searching for adequate maintenance strategies. APMS is used by many airports around the world [41,42].
PMS and APMS development are subjects of many types of research [43][44][45][46]. The downside of the development of APMS is that it is intended mostly for larger and frequently used airports, but not for smaller or less frequently used airports, e.g., the Maribor Edvard Rusjan Airport, which is mostly intended for emergency flights, the performance of exhibition, training and school flights. In all described systems, there is not a system for automated unevenness detection on the runways and their evaluation. On the runways, instances of unevenness can appear in the shape of a lift of the runway, cracks or other dents, and all of these can affect safe landing or take-off of lighter aeroplanes. The biggest problem that can occur is aquaplaning risks. The authors conduicted the automated detection of the dents with the help of a point laser displacement sensor and accelerometer. They concluded that the biggest issues are in the accuracy of detecting the damage location in the dependence of certain measure time intervals. The runways unevenness is the biggest extent impact that increases the possibilities of suspension damage, the malfunction of sensitive devices, uneven contact with the floor, cargo damage or even aeroplane damage.

Contribution
The research works in the field of identifying different methods and methodologies for measuring unevenness at the airport runways have shown that the suggested method has not yet been used or carried out. Furthermore, in runway maintenance, special attention is given to larger, more burdened (also richer) airports which can implement larger financial inputs into management information systems, which are of course available in the market.
Automated systems for evaluation of runway state at smaller regional airports are not detected. This is why the contribution of this article is provided with: • a changed (adjusted) method for execution of the measurements of unevenness on regional airports and • an adjusted, sustainably inclined information system for airport infrastructure management on smaller regional airports.

Methods and Materials
The methodology to establish an automated system for monitoring and evaluating runway unevenness was composed of the following steps: • a critical analysis of valid demands for evaluating the runways' unevenness;an adapted method for measuring the runway's state; • the analysis of (geodetic) methods; • the production of the adapted method suggestion; • the test of suggested method: larger financial inputs into management information systems, which are of course available in the market. Automated systems for evaluation of runway state at smaller regional airports are not detected. This is why the contribution of this article is provided with: • a changed (adjusted) method for execution of the measurements of unevenness on regional airports and • an adjusted, sustainably inclined information system for airport infrastructure management on smaller regional airports.

Methods and Materials
The methodology to establish an automated system for monitoring and evaluating runway unevenness was composed of the following steps: • a critical analysis of valid demands for evaluating the runways´ unevenness;an adapted method for measuring the runway's state; • the analysis of (geodetic) methods; • the production of the adapted method suggestion; • the test of suggested method:  the execution of the measurements;■▪  the recalculation and results evaluation.
• an adapted information system for airport logistics infrastructure management: • the analysis of the possibility to upgrade existing information systems; • the suggestion and upgrade execution; • the test in a real environment.
Substantively, the research was executed in terms of a PhD thesis at the Faculty of Logistics, the University of Maribor, and the fieldwork was executed at the Maribor Edvard Rusjan Airport, Slovenia. The airport is a public international airport with the reference code 4D and is, in terms of volume of turnover, among the smaller public airports. The runway is 2500 m long, 45 m wide and has a load capacity PCN 86/F/A/X/T. The measurements of the unevenness on the Maribor Edvard Rusjan Airport runway were executed in agreement with the airport management.
The runway´s state was found out with the help of a geodetic survey. For this purpose, the vehicle with built-in geodetic sensors with which the state of runways was found out, was made. The key geodetic equipment was a robotized total station RTS Leica Nova TS50. With it, the moving towing vehicle on which the 360° prism was installed, was observed. RTS Leica Nova TS50 has the accuracy which is complied with the ISO 17123-3 for measuring angles and is 0,5" and with ISO 17123-4 for measuring the lengths and is 0,6 mm ± 1 ppm. All measurements were executed in a unified coordinate system.
The dual-frequency base GNSS receiver Topcon Hiper Pro was installed on the towing vehicle, with which the coordinates of the driving trajectories were encompassed and measured larger vertical unevenness were noted down. The data with RTS and GNSS were encompassed with the speed of 6-10 readings per second which means that, at an individual measurement at the constant speed of 6 km/h, on average, 7800 trajectory points were measured. In terms of the research, we made three independent measurements of the unevennessstate, which were later mutually analysed and compared.
The established geodetic net presents the geometric base of measuring of the runwaystate. This is essential to consider the rules of planning geodetic nets which determine that it is necessary to control the capture and data processing, consider the corrections of measured quantities and use an appropriate balancing method [47,48].
Stabilized points of the geodetic zero net provide stability for a longer time, and with optimal execution and multiple repetitions of the measurement (Figure 1).
To provide as much accuracy as possible, three classic terrestrial dimension methods were used: triangulation (the GNSS method which allows bonding on the national spatial the execution of the measurements; larger financial inputs into management information systems, which are of course available in the market. Automated systems for evaluation of runway state at smaller regional airports are not detected. This is why the contribution of this article is provided with: • a changed (adjusted) method for execution of the measurements of unevenness on regional airports and • an adjusted, sustainably inclined information system for airport infrastructure management on smaller regional airports.

Methods and Materials
The methodology to establish an automated system for monitoring and evaluating runway unevenness was composed of the following steps: • a critical analysis of valid demands for evaluating the runways´ unevenness;an adapted method for measuring the runway's state; • the analysis of (geodetic) methods; • the production of the adapted method suggestion; • the test of suggested method:  the execution of the measurements;■▪  the recalculation and results evaluation.
• an adapted information system for airport logistics infrastructure management: • the analysis of the possibility to upgrade existing information systems; • the suggestion and upgrade execution; • the test in a real environment.
Substantively, the research was executed in terms of a PhD thesis at the Faculty of Logistics, the University of Maribor, and the fieldwork was executed at the Maribor Edvard Rusjan Airport, Slovenia. The airport is a public international airport with the reference code 4D and is, in terms of volume of turnover, among the smaller public airports. The runway is 2500 m long, 45 m wide and has a load capacity PCN 86/F/A/X/T. The measurements of the unevenness on the Maribor Edvard Rusjan Airport runway were executed in agreement with the airport management.
The runway´s state was found out with the help of a geodetic survey. For this purpose, the vehicle with built-in geodetic sensors with which the state of runways was found out, was made. The key geodetic equipment was a robotized total station RTS Leica Nova TS50. With it, the moving towing vehicle on which the 360° prism was installed, was observed. RTS Leica Nova TS50 has the accuracy which is complied with the ISO 17123-3 for measuring angles and is 0,5" and with ISO 17123-4 for measuring the lengths and is 0,6 mm ± 1 ppm. All measurements were executed in a unified coordinate system.
The dual-frequency base GNSS receiver Topcon Hiper Pro was installed on the towing vehicle, with which the coordinates of the driving trajectories were encompassed and measured larger vertical unevenness were noted down. The data with RTS and GNSS were encompassed with the speed of 6-10 readings per second which means that, at an individual measurement at the constant speed of 6 km/h, on average, 7800 trajectory points were measured. In terms of the research, we made three independent measurements of the unevennessstate, which were later mutually analysed and compared.
The established geodetic net presents the geometric base of measuring of the runwaystate. This is essential to consider the rules of planning geodetic nets which determine that it is necessary to control the capture and data processing, consider the corrections of measured quantities and use an appropriate balancing method [47,48].
Stabilized points of the geodetic zero net provide stability for a longer time, and with optimal execution and multiple repetitions of the measurement (Figure 1).
To provide as much accuracy as possible, three classic terrestrial dimension methods were used: triangulation (the GNSS method which allows bonding on the national spatial the recalculation and results evaluation.
• an adapted information system for airport logistics infrastructure management: • the analysis of the possibility to upgrade existing information systems; • the suggestion and upgrade execution; • the test in a real environment.
Substantively, the research was executed in terms of a PhD thesis at the Faculty of Logistics, the University of Maribor, and the fieldwork was executed at the Maribor Edvard Rusjan Airport, Slovenia. The airport is a public international airport with the reference code 4D and is, in terms of volume of turnover, among the smaller public airports. The runway is 2500 m long, 45 m wide and has a load capacity PCN 86/F/A/X/T. The measurements of the unevenness on the Maribor Edvard Rusjan Airport runway were executed in agreement with the airport management.
The runway s state was found out with the help of a geodetic survey. For this purpose, the vehicle with built-in geodetic sensors with which the state of runways was found out, was made. The key geodetic equipment was a robotized total station RTS Leica Nova TS50. With it, the moving towing vehicle on which the 360 • prism was installed, was observed. RTS Leica Nova TS50 has the accuracy which is complied with the ISO 17123-3 for measuring angles and is 0,5" and with ISO 17123-4 for measuring the lengths and is 0,6 mm ± 1 ppm. All measurements were executed in a unified coordinate system.
The dual-frequency base GNSS receiver Topcon Hiper Pro was installed on the towing vehicle, with which the coordinates of the driving trajectories were encompassed and measured larger vertical unevenness were noted down. The data with RTS and GNSS were encompassed with the speed of 6-10 readings per second which means that, at an individual measurement at the constant speed of 6 km/h, on average, 7800 trajectory points were measured. In terms of the research, we made three independent measurements of the unevennessstate, which were later mutually analysed and compared.
The established geodetic net presents the geometric base of measuring of the runwaystate. This is essential to consider the rules of planning geodetic nets which determine that it is necessary to control the capture and data processing, consider the corrections of measured quantities and use an appropriate balancing method [47,48].
Stabilized points of the geodetic zero net provide stability for a longer time, and with optimal execution and multiple repetitions of the measurement (Figure 1).
In measuring the running surface state, the object of observation was a trailer with an installed 3600 reflective prism. The movement of the trailer was concerned as a kinematic process, and the position of the trailer was executed with the GNNS method and with the terrestrial cinematic dimension method. The robotized electronic tachymeter, Leica Nova TS50, followed the movement of the trailer and noted its position with the frequency of 10 measured positions in a second. Supernumerary measurements, which provide classic balancing of geodetic observations, were executed [51].  To provide as much accuracy as possible, three classic terrestrial dimension methods were used: triangulation (the GNSS method which allows bonding on the national spatial coordinate system D96/TM), trilateration (a precise terrestrial method), and precise levelling.
The positional net balancing was executed based on the smallest squares method, with the software package Columbus 4.6. The points position accuracy of the geodetic zero net was estimated with standard ellipses of the errors [49,50] and with global accuracy measurement [50]. When balancing the altitude net, the Earth curvature effect and systematic environment effect were considered.
GRID (see Figure 1) presents a rectangular net of points of a mutual distance of 6 m. GRID was used as the base in the benchmarking of individual dimensions with the trailer. All relative deviations of individual dimensions are determined relatively based on the GRID. The GRID dimension points were executed with a precise terrestrial method.
The running surface state measurement was executed with an assumption that the object of observation (the runway) is still. Three dimensions for three meteorological periods (low temperatures in January 2018, medium temperatures in April 2018 and high temperatures in June 2018) were executed with the described equipment. Special attention was given to the touchdown runway zone (Figure 2).
In measuring the running surface state, the object of observation was a trailer with an installed 3600 reflective prism. The movement of the trailer was concerned as a kinematic process, and the position of the trailer was executed with the GNNS method and with the terrestrial cinematic dimension method. The robotized electronic tachymeter, Leica Nova TS50, followed the movement of the trailer and noted its position with the frequency of 10 measured positions in a second. Supernumerary measurements, which provide classic balancing of geodetic observations, were executed [51].

Determination of the Vertical Deviations
The direct comparison of measured points in more various time dimensions and with the condition that the points, according to the position, do not coincide, is not possible [52]. With appropriate processing of the point cloud (set of points), changes of the shape and position of the observed object can be found.
The data processing of individual dimensions was executed in the following steps and is described in more detail in [53]: The calculated values are shown in Table 2. The algorithm for calculation of vertical deviations and graphical presentation of the calculation results was made in the software package MATLAB R2019b. The software LetMB.m is intended for the calculation of the regression plains and vertical deviations and graphical display of results. The algorithm is executed in two steps [52,53]: • In the first step, with the use of pseudoinverse matrix or Moor-Penrose matrix inverse for each time dimension, the parameters of regression plains-and in, hereinafter, vertical deviations of these regression plains from the GRID-are calculated, • In the second step, individual time dimensions are mutually compared and vertical shifts of regression plains between individual time dimensions are calculated.
At the execution of the measurements in the real environment, it is sensible to use at least two independent measuring methods [54]. As a supplementary method, the established geomechanics method was used: Falling Weight Deflectometer (FWD) (Figure 3). With the help of this method, comparison and evaluation of the relevance of the suggested method for monitoring the state of runways were allowed [53,55].

Determination of the Vertical Deviations
The direct comparison of measured points in more various time dimensions and with the condition that the points, according to the position, do not coincide, is not possible [52]. With appropriate processing of the point cloud (set of points), changes of the shape and position of the observed object can be found.
The data processing of individual dimensions was executed in the following steps and is described in more detail in [53]: The calculated values are shown in Table 2.
The algorithm for calculation of vertical deviations and graphical presentation of the calculation results was made in the software package MATLAB R2019b. The software LetMB.m is intended for the calculation of the regression plains and vertical deviations and graphical display of results. The algorithm is executed in two steps [52,53]:

•
In the first step, with the use of pseudoinverse matrix or Moor-Penrose matrix inverse for each time dimension, the parameters of regression plains-and in, hereinafter, vertical deviations of these regression plains from the GRID-are calculated, • In the second step, individual time dimensions are mutually compared and vertical shifts of regression plains between individual time dimensions are calculated.
At the execution of the measurements in the real environment, it is sensible to use at least two independent measuring methods [54]. As a supplementary method, the established geomechanics method was used: Falling Weight Deflectometer (FWD) (Figure 3). With the help of this method, comparison and evaluation of the relevance of the suggested method for monitoring the state of runways were allowed [53,55].

Management of Irregularities on the Airport Runway
International organizations in the area of aviation-including the European Union Aviation Safety Agency (EASA) in the documents Acceptable Means of Compliance (AMC) and Guidance Material (GM) to Authority, Organisation and Operations Requirements for Aerodromes in Certification Specifications (CS) and Guidance Material (GM) for Aerodromes Design and ICAO in the document Annex 14, among others-also prescribe the permissible presence of irregularities (dents and bulges) on runways and other surfaces intended for aircraft moving. On the surface, there must not be unevenness in the shape of bulges higher than 1.25 cm, measured with a 3 m long crossbar in any direction and dents deeper than 3 mm, measured with the 3 m long crossbar in any direction, which can be filled with water.
This demanding method is based on manual work; due to that, the method is unreliable, time-consuming and financially extensive. The key disadvantage of this method is that unevenness is not located in a unified coordinate system.
Due to the obsoleteness of the demanding but valid method, it was determined in the terms of the research that the measuring procedure, and consequently, the evaluation of the runway state (unevenness) is attempted to be automated. It was necessary to repeatedly study the strategy of evaluating the measurements results with modern measuring equipment and provide the methodology for making the adapted information system for the airport logistics infrastructure management.
The basis with which to establish the adapted information system presents the fact that the maintenance of the runway must be performed periodically, at short-term, middle-term and long-term plans, which are known in advance, with known equipment and methods, with the intention to as well as possible, with as little procedures as possible, with as little congestion as possible, and in an economically favourable way maintain the runways in the best condition. An overdue restoration of the deformations causes them to appear in the bigger extent, which consequently means that maintenance or renovation costs are bigger, and time necessary for renovation is longer [56][57][58][59]. Due to safety and economic reasons, it is necessary to constantly maintain the runways in an optimal state. The runway life expectance is approximately 20 years, but with proper and timely maintenance, this can be extenced to 40 years [59].
Extensive and prolonged renovations of the runways cause the closure or limited function of the runways, which can mean a failure in revenue from the principal activity for the airport manager. Furthermore, airline costs during the renovation forces the aircraft to ground or for an alternative location to be found, not to mention that the costs due to the delays occurred in the air traffic increase as well [60].
In developing the adapted information system for the airport logistics infrastructure management, the following assumptions were made:

Management of Irregularities on the Airport Runway
International organizations in the area of aviation-including the European Union Aviation Safety Agency (EASA) in the documents Acceptable Means of Compliance (AMC) and Guidance Material (GM) to Authority, Organisation and Operations Requirements for Aerodromes in Certification Specifications (CS) and Guidance Material (GM) for Aerodromes Design and ICAO in the document Annex 14, among others-also prescribe the permissible presence of irregularities (dents and bulges) on runways and other surfaces intended for aircraft moving. On the surface, there must not be unevenness in the shape of bulges higher than 1.25 cm, measured with a 3 m long crossbar in any direction and dents deeper than 3 mm, measured with the 3 m long crossbar in any direction, which can be filled with water.
This demanding method is based on manual work; due to that, the method is unreliable, time-consuming and financially extensive. The key disadvantage of this method is that unevenness is not located in a unified coordinate system.
Due to the obsoleteness of the demanding but valid method, it was determined in the terms of the research that the measuring procedure, and consequently, the evaluation of the runway state (unevenness) is attempted to be automated. It was necessary to repeatedly study the strategy of evaluating the measurements results with modern measuring equipment and provide the methodology for making the adapted information system for the airport logistics infrastructure management.
The basis with which to establish the adapted information system presents the fact that the maintenance of the runway must be performed periodically, at short-term, middle-term and long-term plans, which are known in advance, with known equipment and methods, with the intention to as well as possible, with as little procedures as possible, with as little congestion as possible, and in an economically favourable way maintain the runways in the best condition. An overdue restoration of the deformations causes them to appear in the bigger extent, which consequently means that maintenance or renovation costs are bigger, and time necessary for renovation is longer [56][57][58][59]. Due to safety and economic reasons, it is necessary to constantly maintain the runways in an optimal state. The runway life expectance is approximately 20 years, but with proper and timely maintenance, this can be extenced to 40 years [59].
Extensive and prolonged renovations of the runways cause the closure or limited function of the runways, which can mean a failure in revenue from the principal activity for the airport manager. Furthermore, airline costs during the renovation forces the aircraft to ground or for an alternative location to be found, not to mention that the costs due to the delays occurred in the air traffic increase as well [60].
In developing the adapted information system for the airport logistics infrastructure management, the following assumptions were made: • the measurements of runway unevenness need to be executed with modern geodetic equipment which provides the measurment in the unified coordinate system, high accuracy and repeatability of the measurements; • a variable state of the airport logistics infrastructure is established, which is easily and uniquely determinable based on the executed measurements with the modern geodetic equipment; • the strategy of airport logistics infrastructure maintenance is determined or suggested based on the executed evaluation of the state of the airport logistics infrastructure.
The final product is in the shape of the adapted information system prototype for the airport logistics infrastructure management. This was created in the software environment Bizagi Modeler, 3.6.0.044 version and was planned with a BPMN standard modelling technique, which is process-oriented and intended for the software development [61,62]. The conceptual model was made in the shape of LetMB high-level diagram.

Monitoring the Runways State
The mathematical model to calculate vertical deviations and graphical presentation of the calculation results is made in the MATLAB R2019b software package. For this purpose, the LetMB.m programme was made, which is intended for the calculation of regression plains, vertical deviations and graphical display of the results. The calculation was made in two steps.
In the first step, for each time dimension, the calculation of the regression plain parameters and vertical deviations of regression plains from GRID was made. The results are shown in Table 1 for all three-time dimensions. In the second step, the vertical shifts of regression plains between individual time dimensions were calculated. The comparison results are average vertical shifts of regression plains between time dimension. In Figure 4, only the results of third-time dimension are graphically displayed. On the left side, the results of the whole area of the processing are displayed and on the right is the detail. With the colour scale, the values of the difference of the average vertical shift between the compared plains are shown. The red coloured areas show that the deviations are positive (regression plain is above the GRID-bumps) and the blue coloured areas show that the deviations are negative (regression plain is below the GRID-holes). In the case of third time dimension measurement, the biggest deviation detected was in the value of 0.51 cm. The analytical results of the all time dimensions are shown in Table 2.
Sustainability 2020, 12, x FOR PEER REVIEW 9 of 19 ger than the ICAO demands, and the graphical analysis of the peaks (red encircled area in Figure 5) shows that even on shorter distances-assessed under 5 m-the deviations of the value of one equidistance (0.005 m) can be detected which very likely exceeds the other ICAO demand.
(a) (b)  In Figure 5, the distances between the areas of the biggest vertical deviations in the case of the third time dimension measure in the touch-down area are shown. The biggest detected deviations on the concerned area were 3 cm (dark blue colour), the distance to other peaks of areas with the highest deviation was, on average, 29 m.  Later, with the purpose to determine the runway road surface value assessment state, we tried to implement the requirements of evaluating the runway state the according to ICAO (see Figure 5).
It was concluded that the ICAO demands in the computer environment cannot be used directly.
It was seen from the Figure 5 that the distance between the peaks of the deviations of individual time dimension measures from the demanded GRID was even up to 10× bigger than the ICAO demands, and the graphical analysis of the peaks (red encircled area in Figure 5) shows that even on shorter distances-assessed under 5 m-the deviations of the value of one equidistance (0.005 m) can be detected which very likely exceeds the other ICAO demand.
In Figure 5, the distances between the areas of the biggest vertical deviations in the case of the third time dimension measure in the touch-down area are shown. The biggest detected deviations on the concerned area were 3 cm (dark blue colour), the distance to other peaks of areas with the highest deviation was, on average, 29 m. To update the obsolete evaluation system, it is suggested that, as the base for evaluating the runway state, the value of evaluated vertical deviations are obtained based on executed measurements and analysis, as it is suggested with the adapted method to monitor the runways state.
The airport logistics infrastructure maintenance strategy of smaller regional airports is determined in compliance with graph on Figure 6 as: • execution of basic measurements (zero net establishment of, the first time measurement), establishment (upgrade) of information system for airport logistics infrastructure maintenance on smaller regional airports (IMR)-basic condition for maintenance system to function; • evaluation of assessed vertical deviations and determining the value of variable condition of airport logistics infrastructure (SLL) on the touch-down area: • SLL = »great«: no indentation bigger than 1 mm and no lifts of the runway surface bigger than 1 cm is not detected, • SLL = »good«: no indentation bigger than 3 mm and no lifts of the runway surface bigger than 2 cm are not detected, the SLL demand = »great is exceeded«; • SLL = »satisfactory«: no indentations bigger than 5 mm and no lifts of the runway surface bigger than 3 cm is not detected, the SLL demand = »good is exceeded«; • SLL = »bad«: no indentations bigger than 10 mm and no lifts of the runway surface bigger than 5 cm is not detected, the SLL demand = »satisfactory is exceeded«; • SLL = »very bad«: no indentations bigger than 15 mm and no lifts of the runway surface bigger than 7 cm, the SLL demand = »bad is exceeded«; • SLL = »inappropriate«: detected indentations bigger than 15 mm and lifts of the runway surface bigger than 7 cm, • intervention based on the evaluation of the airport logistics infrastructure state on smaller regional airports- Figure 6; • the monitoring is performed in regular periodical overviews every 5 years or since the last complete renovation or last larger maintenance works (surfacing). To update the obsolete evaluation system, it is suggested that, as the base for evaluating the runway state, the value of evaluated vertical deviations are obtained based on executed measurements and analysis, as it is suggested with the adapted method to monitor the runways state.
The airport logistics infrastructure maintenance strategy of smaller regional airports is determined in compliance with graph on Figure 6 as: • execution of basic measurements (zero net establishment of, the first time measurement), establishment (upgrade) of information system for airport logistics infrastructure maintenance on smaller regional airports (IMR)-basic condition for maintenance system to function; • evaluation of assessed vertical deviations and determining the value of variable condition of airport logistics infrastructure (SLL) on the touch-down area: • SLL = »great«: no indentation bigger than 1 mm and no lifts of the runway surface bigger than 1 cm is not detected, • SLL = »good«: no indentation bigger than 3 mm and no lifts of the runway surface bigger than 2 cm are not detected, the SLL demand = »great is exceeded«; • SLL = »satisfactory«: no indentations bigger than 5 mm and no lifts of the runway surface bigger than 3 cm is not detected, the SLL demand = »good is exceeded«; • SLL = »bad«: no indentations bigger than 10 mm and no lifts of the runway surface bigger than 5 cm is not detected, the SLL demand = »satisfactory is exceeded«; • SLL = »very bad«: no indentations bigger than 15 mm and no lifts of the runway surface bigger than 7 cm, the SLL demand = »bad is exceeded«; • SLL = »inappropriate«: detected indentations bigger than 15 mm and lifts of the runway surface bigger than 7 cm, • intervention based on the evaluation of the airport logistics infrastructure state on smaller regional airports- Figure 6; • the monitoring is performed in regular periodical overviews every 5 years or since the last complete renovation or last larger maintenance works (surfacing).
The proposed model for assessing airport infrastructure conditions has been tested. Figure 7 shows the result of the assessment of the condition of the test runway based on real time data of the third time dimension measure. All analyses were perform in QGIS v 14.0 with tools of vektorisating and labelling polygons. As expected, the condition of airport logistics infrastructure (SLL) is inappropriate in the area of touch down where, in the event of rain, it could be expected larger puddles will appear right at the place where the aircraft needs the greatest possible grip when landing. Figure 6. Intervention based on the evaluation of the airport logistics infrastructure state on smaller regional airports [53,56,57].
(a) (b) Figure 6. Intervention based on the evaluation of the airport logistics infrastructure state on smaller regional airports [53,56,57].
The proposed model for assessing airport infrastructure conditions has been tested. Figure 7 shows the result of the assessment of the condition of the test runway based on real time data of the third time dimension measure. All analyses were perform in QGIS v 14.0 with tools of vektorisating and labelling polygons. As expected, the condition of airport logistics infrastructure (SLL) is inappropriate in the area of touch down where, in the event of rain, it could be expected larger puddles will appear right at the place where the aircraft needs the greatest possible grip when landing.
Monitoring (IMR) is based as a sub-process in the maintenance process (APMS-MB). In the monitoring process, the external geodetic measurements' executant, according to the foreseen plan, executes measurements and writes the measurements results into the central database (»CB-LetMB«).
The results shown were also confirmed with supplementary measurements. The Visualization of Results of the Monitoring with the Use of Augmented Reality Since the preliminary analysis, we set an indirect goal which claims that the results of the model will be shown in a way that it will be presented to the user from a different perspective and provide them with some kind of ambient experience. AR-technology allows the data overview in a real environment or allows one to view the data in the interaction with the real environment.
The application was tested in February 2020. The outlines and surfaces of vertical deviations (2D) on the processing area were displayed in the interaction with the real environment. In practice, it means that the mobile device on which the real environment was displayed was directed towards the processing area on the runway and at that moment, on the mobile device's screen, the contours and areas of vertical deviations were displayed. It is shown, in Figure 8, how the user sees the real picture of the runway, augmented with the outlines and areas of vertical deviations.  Monitoring (IMR) is based as a sub-process in the maintenance process (APMS-MB). In the monitoring process, the external geodetic measurements' executant, according to the foreseen plan, executes measurements and writes the measurements results into the central database (»CB-LetMB«).
The results shown were also confirmed with supplementary measurements.

The Visualization of Results of the Monitoring with the Use of Augmented Reality
Since the preliminary analysis, we set an indirect goal which claims that the results of the model will be shown in a way that it will be presented to the user from a different perspective and provide them with some kind of ambient experience. AR-technology allows the data overview in a real environment or allows one to view the data in the interaction with the real environment.
The application was tested in February 2020. The outlines and surfaces of vertical deviations (2D) on the processing area were displayed in the interaction with the real environment. In practice, it means that the mobile device on which the real environment was displayed was directed towards the processing area on the runway and at that moment, on the mobile device's screen, the contours and areas of vertical deviations were displayed. It is shown, in Figure 8, how the user sees the real picture of the runway, augmented with the outlines and areas of vertical deviations.
The testing was uninterrupted. The communication with the server and data transfer was executed as expected. At data transfer, no difficulties were expected since the quantity of transferred data was small. The application was first tested with the used of external GNSS and then with the use of internal GNSS, which was built-in in the mobile device. The accuracy of data displaying with the use of external GNSS was estimated on ten centimetres, and the accuracy of the data display with the use of internal GNSS was much worse and was estimated to approximately one meter. AR use aimed to display areas of vertical deviations and non-accurately show the position of an individual deformation; therefore, it is estimated that the meter accuracy of the data display is appropriate. The testing was uninterrupted. The communication with the server and data transfer was executed as expected. At data transfer, no difficulties were expected since the quantity of transferred data was small. The application was first tested with the used of external GNSS and then with the use of internal GNSS, which was built-in in the mobile device. The accuracy of data displaying with the use of external GNSS was estimated on ten centimetres, and the accuracy of the data display with the use of internal GNSS was much worse and was estimated to approximately one meter. AR use aimed to display areas of vertical deviations and non-accurately show the position of an individual deformation; therefore, it is estimated that the meter accuracy of the data display is appropriate.
The graphical data on areas of vertical deviations and deformations, and especially their visual presentation in interaction with the real world, is in our opinion key in decisionmaking about the way and extent of the execution of maintenance or renovation works. The visualisation of graphical data with the use of advanced technologies, among them the AR, allows the decision-makers better data presentation and analysis results and allows the overview and interpretation of these data in a real environment without other professionals being presented (designers, builders, geodesists, etc.) As it is shown in Figure 8, the decision-makers, or all who overview and analyse the data, overview them in a real environment or interaction with it.

The Expanded Innovative Model for the Continuous Monitoring of the Runway's Deformations
The monitoring (IMR) was, hereinafter, expanded with new actors, since the incorporation of additional supplementary measurements with which runway deformations can be detected or confirmed is intended. Thus, the expanded BDP-monitoring (IMR all), in which the geodetic measurement executant and supplementary measurement provider are included as outer actors, was created and shown in Appendix A Figures A1-A3.
The monitoring results (IMR all) are determined areas of vertical deviations, which are a combination of findings based on the visual runway check-up, the results of used geodetic method and the results of supplementary method, which is shown in [53]. The areas of the deformities can be shown as 2D contours of the plane of deviations from the regression plane, as a 3D plane of deviations from the regression plane or on a plate in a real space.

Discussion
Airport runways are exposed to the function of external and internal forces; therefore, they must be regularly checked and properly restored. For this purpose, on the airport in Maribor, which is not exposed to heavy air traffic, we executed the research of the runway unevenness with a geodetic survey, which is fast, accurate and economic. The measurements were executed in four time periods. The first measurement presents a comparative surface-GRID. The obtained results were processed and displayed in two different ways which offer similar results. The biggest vertical deviations from GRID are obtained in the third time dimension which was also used as the comparative one. With both methods, we got the biggest deformations on the same places of the runway, which proves the applicability hypothesis of various methods for measuring the unevenness state on the runways.
Later, the adapted management information system was developed, and with the help of the suggested automated system of evaluating the detected unevenness state on the runway, the execution of proper measures can be suggested. Parallelly, maintainance strategies and instructions which would need to be implemented in the case of it certain airports decide to use this method were developed.
Since the subject of the research was tested and applied only at the certain regional airport (Maribor Airport, Slovenia), where we concluded that the execution of suggested measures is possible and feasible, it is sensible to later test the suggested measures also on the other comparable airports.
In terms of the research, it was concluded that a generally valid and used evaluation system of unevenness is not appropriate for use in a computer environment [7]; therefore, the new adapted suggestion of evaluation was developed, which would be sensible to check in the wider experts public and according to the executed developed procedures, based on the normative provisions, place into official documents.

Conclusions
Computer-supported tools are also becoming more and more effective in the areas where, for whatever reason still (at least in the normative sense), the evaluation principles which were exposed in the middle or at the end of the 20th century are still valid. With the executed research, we have shown and proven that runway unevenness can be measured with modern tools, and the obtained results can be directly used in sustainably directed information systems for runway management on smaller regional airports.
In order to enable the measurement of the runway condition in geodetic accuracy, a vehicle with sensors was manufactured as part of the research. With the help of the vehicle, the vulnerability of the runway was measured in such a way that the results of the measurements are given with geodetic accuracy and transferable (digital record). The measurements themselves were performed during the closure of the test airport, and the method itself is made in such a way that it is not time consuming. A similar approach is not observed in the present literature.
The measurement results were computer-processed using our own algorithm (MAT-LAB) and compared using existing free geographic information system (QGIS) algorithms. The condition of the airport infrastructure was assessed on the basis of the difference found between the actual condition of the runway (time dimension measure) and the original constructed condition (GRID). From the point of view of monitoring the condition of runways, depressions or holes (the measured plane is below the GRID) may represent those conditions that could be problematic and would need to be monitored to ensure traffic safety.
In order to enable work with the available measurement results and the process of assessing the condition of the flatness of the runway in question in the field, a computer tool for smartphones was established. It allows the decision-makers better data presentation and analysis results and allows the overview and interpretation of these data in a real environment without other professionals being presented.
The used computer aided approach provides as much accuracy as possible, repeatability of measurements and thus professional credibility in the evaluation process of the runway unevenness state. Finally, considering the sustainability principle and runway life cycle, the innovative model of sustainable runway management was made; its use provides proper control of the runway state and proper professional action in the case of detected deviations.
With the suggested type of management, the always-useful runways state, the optimum recovery of airport infrastructure at minimal maintenance finance inputs and the most safe and efficient occurrence of air traffic at the airports is provided.  Informed Consent Statement: Informed consent statement was obtained from all subjects involved in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author. The data are not publicly available. The data were obtained on the basis of own research within the doctoral dissertation of dr. Damjan Doler at the Faculty of Logistics, University of Maribor [53].
Acknowledgments: Authors of the article thank the airport staff, Faculty of Logistics, University of Maribor and the Troia d.o.o., Slovenj Gradec, Slovenia company, which provided the hardware for the realisation of the idea of the use of augmented reality in the procedures of evaluation of the runways state.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
Sustainability 2020, 12, x FOR PEER REVIEW 15 of 19 provided the hardware for the realisation of the idea of the use of augmented reality in the procedures of evaluation of the runways state.

Conflicts of Interest:
The authors declare no conflict of interest.
Appendix A Figure A1. The first part of BPD˙s process decision-making maintenance model IMR all.