extfieldsofstudy:
Computer Science
Mathematics

text:
The Pentagonal Fuzzy Number:Its Different Representations, Properties, Ranking, Defuzziﬁcation and Application in Game Problems

: In this paper, different measures of interval-valued pentagonal fuzzy numbers (IVPFN) associated with assorted membership functions (MF) were explored, considering signiﬁcant exposure of multifarious interval-valued fuzzy numbers in neoteric studies. Also, the idea of MF is generalized somewhat to nonlinear membership functions for viewing the symmetries and asymmetries of the pentagonal fuzzy structures. Accordingly, the construction of level sets, for each case of linear and nonlinear MF was also carried out. Besides, defuzziﬁcation was undertaken using three methods and a ranking method, which were also the main features of this framework. The developed intellects were implemented in a game problem by taking the parameters as PFNs, ultimately resulting in a new direction for modeling real world problems and to comprehend the uncertainty of the parameters more precisely in the evaluation process.

Uncertainty and Uncertainty Measure
Uncertainty theory plays important role in modeling for engineering and science problems. Recently, huge developments have taken place in this area. Analysis based on uncertainty theory is regularly required in the following fields, although it is not limited to these areas; mathematical modeling, physics, chemistry, economics, artificial intelligence, legal fact-finding, medical science, business administration, psychology, decision sciences, etc. Various concepts have been formulated to measure uncertainty. Fuzzy logic involvesthe constructionof an approximate acumen mechanism to tackle the uncertainty related with human behavior. Many activities and approaches have been designed to counteract or curtail the uncertainty that is generatedby decision making, however, there are few comprehensive theories available in the literature. Researchers are currently striving to construct a common approach.
There are some important differences between different types of imprecise or uncertain parameters. If we consider an Interval number, the following observations can be made: (i) The fact belongs to some certain interval (ii) The concept of the membership function is missing.
If we consider a Fuzzy number [1][2][3], the following observation can be made: (i) The conception of belongingness of the component has to be considered (ii) The concept of membership function is derived.
If we consider an Intuitionistic fuzzy number [4], the following observations can be made: (i) The conception of belongingness, or non-belongingness of the component has to be considered (ii) The concepts of the membership, or non-membership function are derived If we take a Neutrosophic fuzzy number [5], the following observations can be made: (i) The conception of truthfulness, falsity and indeterminacy of the component have to be considered (ii) The concept of membership function for truthfulness, falsity and indeterminacy are derived We can easily see the idea by using a Figure 1 that shows how different type of PFN can be formed as below:

Fuzzy Sets and Number
Zadeh [1] invented a new idea, which is known as fuzzy sets theory (FST). The basic theory of uncertainty has been used with immense success in different fields. Chang and Zadeh [2] discovered the main idea of fuzzy sets and numbers. Mathematicians have further studied the different result on that theory [3,6] and the impressive and considerable development of ideas and different application of FST have resulted in the topic gaining a great deal of attention.
Generalized trapezoidal linear Arithmetic operation on LPP Linear programming problem Dahooie et al. [18] Trapezoidal and triangular linear Defuzzification and fuzzy additive ratio assessment method Oil and gas well drilling projects

Pentagonal Fuzzy Numbers
Many researchers have investigated PFN with various types of membership functions. In this part we study the available papers that are related to PFN (see below).
From our survey, we can say that membership functions are taken as linear with symmetry at both ends for the majority of cases. However, what is the scenario if the non-linear membership function or interval-valued membership functions concept is taken and there is asymmetry on both ends and the concept of generalized fuzzy numbers is involved? Obviously, the formations are very different. Also, the results are different from previous cases. In this paper, we try to take all the possibilities for the formation of PFNs into account. For details we have to see Table 2.

Verbal Phrasesin Uncertainty Theory
An important consideration is how can we relate the concept of uncertainty theory to practical examples and what is the verbal phrase for a particular type of uncertainty. Example 2. Suppose some traffic sergeants wants to compute the traffic intensity in a congested crossing in a certain time domain. They have different points of view of the congestion in their mind so they use different kinds of uncertain parameters.As before, we can take any one of the uncertain parameters.
The verbal phrase for different types of uncertain numbers for the above problems are shown in Table 3.

Ranking and Defuzzification
The concept of ranking and the defuzzification methods for any fuzzy numbers are not novel among decision makers. But what is the basic concept of the ranking and the defuzzification methods and what are their inter-relations?
Ranking a fuzzy number involves measuring up to two fuzzy numbers, and defuzzification is a technique whereby the fuzzy number is renewed to an approximated crisp number. Just as the decision maker takes two concepts that are the same, similarly, for this problem we have to convert the fuzzy number to a corresponding crisp number and compare the number on the basis of crisp values.

Motivation
Fuzzy sets theory plays several significant roles in the theory of uncertainty for modeling. An important issue is that if anybody wants to take a PFN, then what should its pictorial representations (uncertainty quantification area) look like? How should we define the membership functions? From this viewpoint, we formulated different types of PFN that may be a good choice for a decision maker in a practical scenario.

Novelties
There are several published works where pentagonal fuzzy sets have been formulated and decision makers have applied these to various fields. However, there are still many important scopes to be worked on for PFN. A summary of the work we have done on PFN is as follows: (i) The development of different types of interval-valued PFN, i.e., symmetric linear PFN, asymmetric linear PFN, symmetric nonlinear PFN, and asymmetric nonlinear PFN were defined. (ii) The representation of the said PFNs in parametric form was defined. (iii) The ranking and defuzzification of PFN were done. (iv) The number was applied to a fuzzy game theory problem.
We can easily see by the following Figure 2 for the difference and relation between defuzzification and ranking. (iv) The number was applied to a fuzzy game theory problem.
We can easily see by the following Figure 2 for the difference and relation between defuzzification and ranking

Structure of the Paper
The article is structured as follows as shown in the Figure 3:

Structure of the Paper
The article is structured as follows as shown in the Figure 3:

Structure of the Paper
The article is structured as follows as shown in the Figure 3:

Preliminaries
Definition 1. Fuzzy set: Let us take a set A, which is defined by If in the pair x, µ A (x) , the first one, x belonsg to the classical set A and the second one µ A (x) belongs to the interval [0, 1], then set A is called a fuzzy set. Here µ A (x) is called a Membership function.
The upper and lower membership function of IVFN is defined by

PFN and Its Different Representations
In this section we extend special types of PFNs in different viewpoints.

Definition 4.
Pentagonal fuzzy number (PFN): [28] A PFN A = (a 1 , a 2 , a 3 , a 4 , a 5 ) should satisfy the following condition:  [28] A linear PFN is written as A LS = (a 1 , a 2 , a 3 , a 4 , a 5 ; k) whose corresponding membership function is written as for a 4 ≤ x ≤ a 5 0 for x > a 5 Definition 6. α-cut or the parametric form of linear PFN with symmetry: [28] α-cut or parametric form of LPFNS is written by the formulae is the nondecreasing function with respect to α and A 2R (α), A 1R (α) is the decreasing function with respect to α Remark 1. The main idea of the symmetric PFN is that the left picked point is equal to the right picked point (see in Figure 4 the same picked value is k).

Definition 7.
Non-Linear PFN with symmetry: A linear PFN is written as A LNS = (a 1 , a 2 , a 3 , a 4 , a 5 ; k) (n 1 ,n 2 ;m 1 ,m 2 ) where the membership function is written as Definition 8. α-cut or parametric form of non-linear PFN with symmetry: α-cut or parametric form of LPFNS is written by the formulae where A 1L (α), A 2L (α) are increasing functions with respect to α and A 2R (α), are decreasing functions with respect to α.

Remark 2.
The basic idea of the mentioned number in Figure 5 is that the left picked value and right picked value are same but the boundary of the fuzzy area be supposed to not linear. The membership function can be formed as a non-linear function. So, we have to present the non-linearity on the membership function as an addition.

Linear Pentagonal Interval-Valued Fuzzy Number with Symmetry
A linear pentagonal interval-valued fuzzy number is written as A LPIS = {(a 1 , a 2 , c, a 4 , a 5 ; k, p), (b 1 , b 2 , c, b 4 , b 5 ; w, q)} whose upper membership function and lower membership function are defined as follows

The α-Cut of a Linear Pentagonal Interval-Valued Fuzzy Number with Symmetry
Corresponding Alpha-cut (α − cut) are defined for the upper and lower membership functions as follows.
The α-cut or parametric representation of LPIS is represented by the formulae are decreasing functions with respect to α 2 , α 1 , respectively. Figure 6  The -cut or parametric form of LPIS is represented by the formulae

Linear Pentagonal Interval-Valued Fuzzy Number with Asymmetry
A linear pentagonal interval-valued fuzzy number is written as A LPI AS = {(a 1 , a 2 , c, a 4 , a 5 ; k, p, r), (b 1 , b 2 , c, b 4 , b 5 ; w, q, s)} whose upper membership function and lower membership function are defined as follows

The α-Cut ofa Linear Pentagonal Interval-Valued Fuzzy Number with Asymmetry
Corresponding Alpha-cut (α − cut) are defined for the upper and lower membership functions as follows.
The α-cut or parametric form of LPIS is represented by the formulae are decreasing functions with respect to α 2 , α 1 , respectively. Figure 7 p = r, q = s then, the above number becomes symmetric PFN.

α-Cut ofNon-Linear Pentagonal Interval-Valued Fuzzy Number with Symmetry
Corresponding Alpha-cut (α − cut) are defined for the upper and lower membership functions as follows.

α-Cut ofNon-Linear Pentagonal Interval-Valued Fuzzy Number with Asymmetry
Corresponding Alpha-cut (α − cut) are defined for the upper and lower membership functions as follows.

Verbal Phrase for the PFN and Interval-Valued PFN
In the case of real-life problems such as the salary of laborers in a factory, everything depends on lots of parameters. For example, there are some laborers whose salaries are very low, some are moderately low, and some have low and moderate salaries while others have high and very high salaries owing to years of experience and their designations. To identify such distinct cases, we need the pentagonal fuzzy number in different forms. Thus, we use verbal phrases to identify its nature.  . Non-linear interval-valued PFN with asymmetry for 1 , 2 < 1, 1 , 2 > 1.

Verbal Phrase for the PFN and Interval-Valued PFN
In the case of real-life problems such as the salary of laborers in a factory, everything depends on lots of parameters. For example, there are some laborers whose salaries are very low, some are moderately low, and some have low and moderate salaries while others have high and very high salaries owing to years of experience and their designations. To identify such distinct cases, we need the pentagonal fuzzy number in different forms. Thus, we use verbal phrases to identify its nature. Figure 15. Non-linear interval-valued PFN with asymmetry for n 1 , n 2 < 1, m 1 , m 2 >1.

Verbal Phrase for the PFN and Interval-Valued PFN
In the case of real-life problems such as the salary of laborers in a factory, everything depends on lots of parameters. For example, there are some laborers whose salaries are very low, some are moderately low, and some have low and moderate salaries while others have high and very high salaries owing to years of experience and their designations. To identify such distinct cases, we need the pentagonal fuzzy number in different forms. Thus, we use verbal phrases to identify its nature.

VL-Very Low, L-Low, M-Medium, H-High, M-Medium, VH-Very High
In this particular game problem, we use different kind of strategies for player A and B. Thus, the members of the payoff matrix are of the pentagonal fuzzy type, and we also use corresponding verbal phrases for different kinds of members. For the details about the problem see Tables 4 and 5.

Verbal Phrase for Interval-Valued Pentagonal Fuzzy Number:
In the case of interval-valued problems, there is a finite range for the interval whose income is low, medium or high. It is specified as very low, low, low mean, large mean, high, very high, etc. Within this finite range the membership function actually varies. Generally, it is denoted by

The Defuzzification Method
The defuzzification process is very important for a problem for two important reasons: Defuzzification is a method for constructing an irrefutable outcome in fuzzy logic, given the fuzzy sets and the equivalent degree of membership functions. There are many defuzzification skills and some of the useful tools are as follows: (1) Centre of area (COA) [30] (2) Largest of maxima (LOM) [31] (3) Smallest of maxima (SOM) [32] (4) Bisector of area (BOA) [33] (5) Mean of maxima (MOM) (6) Regular weighted point (RWP) (7) Graded mean integration value (GMIV) (8) Centre of approximated interval (COAI).

Defuzzification of Non-Linear Symmetric PFN Based on Centroid Method
We proposed a method to compute the defuzzification of non-linear symmetric PFN as, where, After computing we have Hence, And N = k(a 2 −a 1 ) For the linear pentagonal fuzzy number, we have n 1 = 1, For k = 1, n 1 = 1, n 2 = 1, m 1 = 1, m 2 = 1 we have .

Defuzzification of Linear Symmetric PFN Based on Mean of Alpha (α)-Cut Method
The left and right α-cut of a non-linear heptagonal fuzzy number are We proposed the mean of interval method for defuzzification aš For the linear pentagonal fuzzy number, we have n 1 = 1, n 2 = 1, m 1 = 1, m 2 = 1.

Defuzzification of Linear Symmetric PFN Based on Removalof Area Method for Linear Pentagonal Fuzzy Number
We consider different types of areas of the corresponding linear PFN as shown below.   ̌= ( 5 + 4 + 2 + 1 ) 4

Defuzzification of Linear Symmetric PFN Based on Removalof Area Method for Linear Pentagonal Fuzzy Number
We consider different types of areas of the corresponding linear PFN as shown below.

Comparison ofthe Above Three Defuzzification Methods
We already found the analytical result for the defuzzification value for the pentagonal fuzzy number. Then, we compared the two methods numerically as follows in the Table 6: Table 6. Numerical examples.

Comparison ofthe Above Three Defuzzification Methods
We already found the analytical result for the defuzzification value for the pentagonal fuzzy number. Then, we compared the two methods numerically as follows in the Table 6:

Remark 5.
In the numerical study above, we show that calculating the defuzzification value by the centroid method and mean of alpha-cut method gives almost the same result.

Basic Concept of Ranking Fuzzy Numbers
The concept of ranking fuzzy numbers is very important for decision making problems. Researchers have presented many different reasons for finding the ranking of fuzzy numbers . In this section we find a new concept for finding the ranking of PFNs.

Ranking of Pentagonal Fuzzy Numbers
We proposed a new way to find the ranking of PFNs. In Figure 21 the pentagon is divided into three triangles and one rectangle. We can find the ranking by using the centroid formula of triangles and rectangles.

Basic Concept of Ranking Fuzzy Numbers
The concept of ranking fuzzy numbers is very important for decision making problems. Researchers have presented many different reasons for finding the ranking of fuzzy numbers . In this section we find a new concept for finding the ranking of PFNs.
Now, the centroids are in the form: Suppose we consider two different PFNs A and B, which are to be ranked. Then using the above method and after finding the R(A) and R(B) by using the previous formula we can easily say that if The flowchart of the proposed method is given below in Figure 22.

Numerical Computation
Remark 6. By using the above concept of ranking PF's, we can easily compare them (see Table 7).

Application of Pentagonal Fuzzy Number in a Game Problem in a Fuzzy Environment Using the Dominance Method
Let us consider two different players, A and B playing a game of carom, with A 1 ,A 2 , . . . ,A n strategiesand B 1 ,B 2 , . . . ,B m strategies, respectively. We also consider that each player can choose pure strategies. Let, a pq be the elements of the pay-off matrix. The strategy of player A is denoted by A p and the strategy of player B is denoted by B q .
Saddle point: In a game problem, the max-min for A and min-max for B should be equal, therefore, the corresponding game is assumed to have a saddle point or an equilibrium point. The gain at the saddle point, which is the arrangement in the associated matrix where the maximum of the row minima correspond with the minimum of the column maxima, is said to be the value of the game.
We represent the max-min value of the game by gamma(γ), and the min-max value of the game by γ. The game is said to be fair. If γ = 0 = γ.
The game is supposed to be strictly ascertainable. If γ = γ = γ.

Operation for Solving a Game Problem Using Fuzzy
Step1: First, we have to examine whether a saddle point will exist or not. If it exists in the given problem, then we need to find it in a direct way. However, if it doesnot exist, then we need to follow the second step.
(1) In the given pay-off matrix, if the elements of Column A ≤ elements of Column B, that is, Column A strategy will fully dominate over column B strategy, then, according to the rule we need to delete column B strategy from the given pay-off matrix.
(2) In the given pay-off matrix, we tally each column strategy with all other column strategies and omit high strategies as far as possible.
(i) In the given pay-off matrix, if the elements of Row A ≥ elements of Row B, Row A strategy will dominate over Row B strategy. So, omit Row B strategy from the given payoff matrix.
(ii) In the given pay-off matrix, we tally each row strategy with all possible row strategies and omit low strategies as far as possible.
(iii) The game may decrease to a single cell giving an order about the value of the game and optimal master plan of the players. If not, then moves to step 4.
Step4: The dominance rule should not be based on the dominance of pure strategies only. A given approach can be dominated if it gives us a poor result to an usual of two or more other pure master plan.

Numerical Problems
Example 1. Let us consider a fuzzy game pay-off matrix where two different players are A and B (Table 8): We apply the rule of defuzzification using Alpha (α)-cut method, to convert the pentagonal fuzzy number into a crisp number, then we have the modified pay-off matrix as Table 9:  Now, the convex combination of B 3 , B 4 (we take the average), that is, 3, 2, 4 is dominated by B 2 . Hence, we have Table 12, Now, the convex combination of A 3 ,A 4 (we take the average), that is, 2,4 is dominated by A 2 . Hence, we have Table 13, Hence, the optimal solution to the main problem is for A(0,0, 2 3 , 1 3 ) for B(0,0, 2 3 , 1 3 ) and the corresponding game value is 8 3 .
Example 3. Let us consider another fuzzy game pay-off matrix where two different players are A and B for different pentagonal fuzzy numbers as a pay-off matrix member (Table 16). After crispification we have a pay-off matrix as Table 17. Hence, the optimal solution to the main problem is for A(0,0, 5 8 , 3 8 ) for B(0,0, 5 8 , 3 8 ) and the corresponding game value is 15 4 .

Remark 7.
In the above problem we take different three sets of the pay-off matrix with symmetric PFN and after utilizing the concept of defuzzification we solve the game problem. We can see that corresponding optimal strategies are changed, and also the game values are changed. Thus, we can conclude that for different types of pentagonal fuzzy numbers, there will be different kinds of optimal solutions with different types of game value result.

Conclusions and Scope of Future Research
In this paper, we extended the characteristics of pentagonal fuzzy numbers to interval-valued fuzzy numbers and pentagonal fuzzy number. Bearing in mind the symmetric and asymmetric structures of a pentagonal shape, the membership functions were comprehensively investigated for both cases. The linear and non-linear for symmetric and asymmetric membership functions along with three cuts were also added to the discussion. The defuzzification of the pentagonal fuzzy number was carried out by three methods, namely:the centroid method, mean of Alpha (α)-cut method, and the removal of area method. The ranking between two IVFNswas found by the centroid formula method. Efficaciously, the produced solutions exhibited IVPFN structures that contain all the possible outcomes of the governing model. Thus, the following conclusion were made:

•
The expansion of the interval-valued pentagonal fuzzy numbers and functions, adds a new tool for modeling different aspects of science, engineering and other environmental studies.

•
The classifications of linear and nonlinear membership functions provided an appropriate strategy for decision makers. • Detailed illustrations of membership functions, α-cuts and ranking, and defuzzification provide all the required information in one platform to model any real-world problem.
In our forthcoming work, we will apply these definitions to other types of fuzzy numbers such as hexagonal fuzzy numbers, and diamond shape fuzzy numbers and use them to model different dynamics of applied sciences, such as multi-criterion decision making, optimization, networking problem, etc.


CNN-Based Deep Architecture for Health Monitoring of Civil and Industrial Structures Using UAVs †

: Health monitoring of civil and industrial structures has been gaining importance since the collapse of the bridge in Genoa (Italy). It is vital for the creation and maintenance of reliable infrastructure. Traditional manual inspections for this task are crucial but time consuming. We present a novel approach for combining Unmanned Aerial Vehicles (UAVs) and artiﬁcial intelligence to tackle the above-mentioned challenges. Modern architectures in Convolutional Neural Networks (CNNs) were adapted to the special characteristics of data streams gathered from UAV visual sensors. The approach allows for automated detection and localization of various damages to steel structures, coatings, and fasteners, e.g., cracks or corrosion, under uncertain and real-life environments. The proposed model is based on a multi-stage cascaded classiﬁer to account for the variety of detail level from the optical sensor captured during an UAV ﬂight. This allows for reconciliation of the characteristics of gathered image data and crucial aspects from a steel engineer’s point of view. To improve performance of the system and minimize manual data annotation, we use transfer learning based on the well-known COCO dataset combined with ﬁeld inspection images. This approach provides a solid data basis for object localization and classiﬁcation with state-of-the-art CNN architectures.

Introduction
As of 2019, there are 39,671 bridges in road traffic in Germany. In the field of steel and steel composite bridges, this amounts to a total bridge area of 3,999,000 m 2 , 65% of these are much older than 20 years. About 46% of these bridges are in the lower half of the rating scale regarding their status of structure [1]. These figures only concern road bridges. Rail traffic, and, accordingly, the rail bridges are not included. There are further 25,677 (December, 2018) bridges, 4899 of them are steel constructions [2]. The Federal Ministry of Transport and Digital Infrastructure's traffic forecast for 2030 states that there will be an increase of 43% in rail traffic concerning tone-kilometers. The passenger service on rails should increase by almost 39% [3].
These figures illustrate the importance of a substantial healthy and reliable infrastructure. For ensuring the continuously increasing road and rail traffic, there is a regulation for periodic monitoring of bridges [4][5][6][7].
DIN 1076 "Civil Engineering Structures for roads-Monitoring and Examination" [5] regulates the inspection and control of all bridges concerning road traffic. For rail bridges, Ril 804.80 [4,6] is an analog.
Both rules mandate a building examination every six years. This examination has to be carried out visually, at a very close distance to the structure, allowing tactile feedback [5][6][7]. DIN 1076 calls this examination general inspection (H). Inspection is conducted by qualified specialists. To guarantee complete access for the specialist, technical devices are necessary. The use of these big units make (partial) road closure inevitable and are thus time consuming [5].
In addition to the described general inspection, DIN 1076 requires a simple inspection (E) every six years. This should be an extended visual inspection without tools, which is three years shifted to the general inspection (H). The defects found in the general inspection (H) should be found and compared in the simple inspection (E). A change of the existing defects is the focal point of this inspection (E). In those years when there is neither a general inspection (H) nor a simple inspection (E), an observation (B) is required. Further, a special investigation (S) is mandatory by order, or after general storms, fires, or accidents. These considerations demonstrate that currently a bridge inspection in Germany is tied to large expenses in personnel, equipment, and money [5,7].
Based on these challenges, the project InÜDosS (InÜDosS-"Inspektion, Überwachung und Dokumentation von stahlbaulichen Strukturen"-Inspection, monitoring and documentation of steel constructions) was initiated to automate monitoring, inspection, and documentation of steel constructions. Figure 1 shows the interconnection of the different stages in our automatic monitoring approach. The scope of the project InÜDosS covers different aspects that arise in the process of monitoring steel structures with UAVs. This includes data acquisition and 3D model reconstruction, automatic path planning, and image analysis. In this paper, our focus was on the aspect of automatic image analysis (outlined in red). There is much literature on the subject of (semi-)automatic detection of damages on civil and industrial structures. Koch et al. [8] gave a comprehensive overview of current practice in visual inspection and damage types of civil structures including associated severity scales. The authors stated that listed defect classes are detectable with computer vision-based approaches. Further, the state-of-the-art in computer vision defect detection and assessment in the pre-DCNN era is presented. As one of the main open challenges for automatic defect retrieval, an automated process of image and video collection is identified. Recently, Zhang et al. [9] proposed vision-based damage detection and classification using an adapted YOLOv3 network and Huethwohl et al. [10] used a multi-classifier for reinforced concrete bridge defect detection. Furthermore, automatic detection of cracks is discussed [11][12][13][14].
The novelty of the presented work, besides the definition of a hierarchy of damage classes, is that compliance with the rules and regulations of DIN 1076 is discussed. Especially the observations (B) and the simple inspection (E) should both be performed only visually and without the use of tools (cf. [5,7]).
We discuss a vision based approach in (semi-)automatic damage detection in bridges and steel constructions using UAVs. The main contributions of this work are: • Development of a semantic classification scheme for bridge inspection in public space, which is compatible to the specifications of DIN 1076 • Devising a hierarchy of defect classes • Implementation and evaluation of a corresponding cascaded classifier based on state-of-the-art CNN object detection

Materials and Methods
The special nature of the image data gathered demands a custom approach to tackle the problem. Field inspection images at hand show great variety concerning characteristics of the structures being investigated (cf. Figure 2), as well as the level of detail. Depending on the distance between the UAV and the examined object, the different types of damages only add to the complexity of the problem. Our approach is based on a hierarchy on a conceptual level, as displayed in Figure 3, which we directly transfer to our computational method. The proposed hierarchy consists of two stages. At the first stage, we aim to identify relevant structures. Roughly speaking, this comprises all (coated) supporting steel structures. If such a structure is identified, it will then be analyzed for damages. At the second stage, we consider two main categories of potential damage: coating damages and faulty fasteners. Coating damages are further subdivided into blistering, cracks, missing coating, and corrosion. Fasteners can be intact, defective (e.g., loose nuts), tilted, missing, or corroded.  We transfer this concept directly to the computational domain by using a two-stage convolutional neural network (CNN) approach, based on Mask-RCNN [15]. We use three separate instances of the Mask-RCNN (two in the second stage). All three networks use the ResNet50-backbone [16] and Feature Pyramid Networks (FPN) [17]. Our implementation is based on the implementation by Facebook Research [18]. The proposed two-stage architecture is shown in Figure 4. 1, f a n,k n }}. All results contained in R 1 = R 1 co ∪ R 1 f a are then projected back onto the input image I 0 , giving the final prediction results R 0 .
As a data basis, we used approximately 1000 images of steel constructions, consisting mostly of bridges and supporting steel constructions, many of them showing at least one type of damage we are considering. These images are divided into three categories: steel structures, coating damages, and fasteners. Note that the images of the three datasets are not necessarily disjoint, but the corresponding annotations are. To reduce computational costs while increasing accuracy, we used transfer learning. We initialized all three networks with network weights pre-trained on the COCO (Common Objects in COntext) dataset [19]. We trained each network for 80 epochs, at a learning rate of 2 × 10 −3 and a batch size of 2. Further, an initial warm-up phase of 500 iterations (batches) was used to overcome difficulties which may arise in early phases of optimization [20]. During training, validation tests were performed after each epoch. Data augmentation included horizontal and vertical flip, as well as variation in brightness, contrast, and saturation. Datasets were split into 11 parts of (near-)equal size. One part was used as a test set and the other ten parts for cross-validation (10-fold split). Figure 2 shows examples of the images used in this study. For each validation test, we determined the model(s) with the best AP, AP50, and AP75, respectively. Figure 5a shows the average precision on the validation set of the ten-fold split (cf. Section 2). This image shows the evaluation for the first stage (steel constructions). To make results of the cascaded and plain version comparable, we first applied the trained detector for supporting structures and then applied the same model as in the simple step on all areas suggested by the first detector. The results of this step were then projected back on the original (full) image. This allowed for directly comparing the results of the plain and the cascaded detector. We selected models according to the best performance regarding average precision.

Results
To test our proposed two stage method, we exemplarily performed tests regarding the second-tier category coating in a simple version and a cascaded version. For the second level coating, we additionally trained networks on the same data but with reduced number of classes, in order to test how well the network is able to only recognize and spot damages without further classifying them. In these tests, we used the classes intact and defective.
For the simple version, we applied the trained model of the given category on the corresponding test dataset as is common practice. In this case, the detector was simply applied to the whole image. Figure 5b shows IOUs on image level for a test set of 70 images. The cascaded version shows an improvement of 9% compared to the plain version. Figure 6 shows detection results for coatings and fasteners. Red boxes indicate detection results for the first stage (steel constructions), while green boxes and outlines show results of the second stage (coating or fasteners).

Discussion and Conclusions
Detection of steel structures works very well, while detection and especially classification of damages is more difficult. This is mainly due to the fact that the data and the semantic categories are inherently difficult. Some damages are very subtle and hard to spot from images, even for an expert, while others depend on a significant amount of context information and interpretation. Furthermore, classes are not always clearly distinguishable, and objects may adhere to several classes at once (fasteners may be defective and corroded, for example). Overall spotting works well, and classification may be sensible, even when differing from ground truth annotation. This applies when classes overlap, or classification by a human expert is fuzzy by itself.
An automatic visual detection of defects on steel bridges complies the requirements of DIN 1076 and Ril 804.80. Required inspections are, with the exception of the main inspection (H), only based on visual monitoring. Numerous areas of the bridge cannot be reached and inspected without big units such as aerial work platforms or units to inspect bridges from below. These tools are not intended for observations (B) and simple inspections (E). Thus, a periodic flyover with a UAV could improve and specify both inspections (B and E). Further, existing defects are observed and checked regularly and small changes would be detected immediately [5][6][7].
The presented work shows the potential of the developed approach in designing a DIN 1076 compliant visual inspection pipeline for automated damage detection. Due to more precise annual inspections by UAV, which would replace the superficial and visual inspections without tools (B and E), a process of change of a defect would strike immediately and maintenance measures can be taken. The structure and its durability could be improved and maximixed. This presents a further step towards smart city infrastructures.


A decision-making process for resilience assessment in adaptive reuse

In the field of public heritage valorisation through renovation, resilience assessment is crucial to foresee the impact of each project action, monitor the transformability thresholds and achieve more sustainable restoration projects. Within this framework, methodologies to achieve early evaluation of different project alternatives since the very preliminary design phases are supposed to be a key factor for the success of the building process, as it is highlighted by the Italian public procurement code (D.Lgs 50/2016). Therefore, the role of the contracting authority is crucial, as it is called both to activate the process through the definition of the project requirements and to accomplish the evaluation of the proposed solutions by respecting the principles of public interests, legality, transparency and responsibility. In order to achieve these goals, the research proposes a standard methodology composed of a list of synthetic indicators and the criteria for their combination, aiming to assess the residual performances of the building and evaluate the compliance with new intended uses through the match with a set of conceivable design strategies. In this direction, two assessment tools are defined: the PAV (Performance Adequacy and Vulnerability), which is calculated on the actual state of the building for the evaluation of its residual efficiency with reference to some transformative hypothesis and then integrated in a grid of design strategies, and the RTE (Resilience Threshold Evaluation) though to measure the expected transformative impacts on the building and their positive or negative implications within its resilience thresholds. The methodology is implemented in a BIM-based process with the definition of a simplified informative model that makes it possible to retrieve all the information and to define the calculation rules in order to activate the defined methodology with the advantage of ensuring time-saving, data-safety and transparency in the process.

Introduction
Recent debate on sustainable development increasingly includes resilience among its main topics, implying a more dynamic view of the future, where changing and not stability is the norm [1]. Originally coming from ecology where it is defined as the capacity of a system to react to the changes of its context [2], in engineering, resilience is mainly cited with relation to risk management, stressing that the preservation of variety within our natural and built environments is a key factor for development. While change is essential for evolution, adaptability and transformability are keywords of resilient thinking [3] and therefore heritage valorization through adaptive reuse [4] can be regarded as a successful strategy.
Descending to the building scale, resilience will be here meant as the acceptable transformative load that a generic building system is able to bear without compromising its physical, typological or historical identity. In this term resilience limits will be intended as the thresholds above which undesirable effects would spoil the good, by altering irreparably its consistency and semantic coherence, if not its physical existence. Within the building process, this is translated in a research of the fragile equilibrium between the constraints dictated by the conservation needs and the fulfillment of the requirements of the contemporary users.
In this context, within a performance-based approach, the introduction since the first stages of the building process the evaluation of different design alternatives is believed to be crucial, as it is highlighted by the new Italian Public Contract Code (D.Lgs 50/2016 as modified by D.Lgs. 57/2017). This means the necessity of providing the actors involved (the public administration from one side and a series of competitors on the other) of shared multi-criteria analysis tools able to compare different strategies aimed at the satisfaction of a complex framework of needs.
In this paper the application of an assessment and evaluation methodology able to support and provide continuity to the building process is presented with particular reference to the preliminary stages of adaptive reuse projects that are developed through a design contest aimed at the acquisition of the technical and economic feasibility project.

A resilience-based assessment methodology
An ongoing research within the Laboratory of Science and Technology for construction and Planning (STeP lab) in the Department of Civil Engineering and Architecture (DICAr) of the University of Pavia recently settled the theoretical methodology Re³ (Resilient Restoration and Reuse) for the preliminary assessment of existing buildings [5]. The final aim is to define a repeatable procedure for supporting decision-making in the first stages of adaptive reuse processes and to provide a set of tools able to compare different design hypothesis on the basis of a standardized assessment grid. The evaluation method integrates the performance-based building design approach [6] with the principles of resilient thinking applied to the built heritage, by facing the functional compatibility and the transformative impact control issues. The main goal is to guide the stakeholders in the achievement of a preliminary reliable knowledge framework on the existing building in order to understand both its attitude of transformation and resilience thresholds. Therefore, the workflow is developed through the sequence analysis/diagnosis/evaluation and includes two assessment indices able to estimate the leftover performances of an existing building with regard to different intended uses (Performance Adequacy and Vulnerability -PAV) and the impact of the transformative actions put in relation with the estimated resilience thresholds (Resilience Threshold Evaluation -RTE).
The PAV index is composed by six indicators calculated as the average of several parameters and divided by classes of need. Aiming to pursue a synthetic overview of the state of the building under examination, Usability, Well-being, Safety, Accessibility, Conservation and Flexibility are considered, according to Table 1. Each parameter is measured on a scale that goes from 0 (missing) to 3 (good), with reference to its adequacy in accordance to specific ranges of values coming from the Italian regulatory framework and technical standards.  The RTE index is a tool aimed at the evaluation of the impacts of possible project alternatives, with reference to both desirable and undesirable effects. Each driver of transformation is divided in eight macro-actions, four negative (Material removal, Structural alteration, Spatial alteration and Identity weakening) and four positive (Future adaptivity increase, Design for all, Carbon footprint reduction and Safety increase), which can be meant as Boolean questions in order to define a "decision tree" drawn by eight control variables. However, by considering the complexity involved in the design process, its reduction to a set of yes/no queries seems unsuitable. That's why the final output of the RTE Index involves a multi-criteria analysis, which takes into account the mutual relationships between the control variables. This is achieved through the compilation of a matrix that activates a pairwise comparison [7].
The proposed methodology has already been tested for the assessment of a part of the built heritage owned by the University of Pavia by integrating it within a BIM (Building Information Modeling) based process. The parameters used for the definition of the synthetic indices were retrieved in a semiautomated way from the informative model, thanks to the implementation of a series of scripts designed with VPL (Visual Programming Languages) [8].

The current legislative framework
The system of rules applied to public procurement and concession contracts in Italy has been recently reorganized with the issuing of the Legislative Decree 50/2016 as modified by Legislative Decree 57/2017. According to the texts, the design process is divided into three subsequent levels of planning: technical and economic feasibility project, final project and executive project (Art. 23). The characteristics of each stage of design are described in greater details in the draft of implementing decree called "Definition of the content of the design process in the three levels of planning" issued in December 2016. In particular, the first design stage could be defined into two steps: The DOCFAP shall consider also the so-called "zero option", which is the hypothesis of non-intervention. In the cases of projects on existing building it should also highlight the impacts that each design strategy has on the context and on the historical, architectonic, structural and technological characteristics. b. Elaboration and development of all the investigations and the deliverables aimed at the identification of the dimensional, volumetric, typological, functional and technological characteristics and the relative financial framework of the project.
The attention is here focused in the case of intervention on the existing buildings and, in particular, in the preliminary phase of design, where the strategic choices on the project are taken and, therefore, where the implementation of a decision-making methodology is more effective.
The guidelines for the implementation of technical and economic feasibility project states that it can be developed inside the public administrations where appropriate internal professional skills in the subjects covered by the project are retrievable. Otherwise, as it happens in the vast majority of cases, the procedure of the design contest shall be activated. The procedure in all its stages is described in Art. 152 (D.lgs. 50/2016) and can be summarized as follows: • The contracting authority publishes two tender documents focused on a performance based survey: the framework of needs (Quadro Esigenziale), which describes the specific qualitative and quantitative needs which have to be satisfied through the purchase, and the document of guidelines for planning (Documento di Indirizzo alla Progettazione), which identifies objectives to be pursued, functions to be performed, needs to be satisfied and performance to be achieved, along with the reference to potential technical proceedings or standards that the competitors are required to implement; • In accordance with the tender documents, the competitors are called to draw up the Feasibility Study of the Project alternatives (DOCFAP), wherein a set of possible design solutions is identified and analysed under the qualitative profile by considering the environmental, technical and economical perspectives. In the delivery the alternatives should be identified thanks to descriptive and graphic outputs that highlight the key features of each proposal and a multicriteria comparison based on a cost-benefit analysis is required in order to identify the most convenient solution; • The contracting authority choses the best design solution, by means of a commission, whose judgement shall express on each project on the basis of the criteria set out in the content notice with specific justification; • The winner of the competition accomplishes its proposal and develops the design alternative that was selected by the administration and identifies for that one the volumetric, typological, functional and technological characteristics. In the end, the public contracting authority acquires the ownership of the project. Although the regulation states that the first part of the design contest could take place in single stage, also then it is strongly recommended to perform an evaluation of the possible design alternatives.

The Re³ methodology in the current legislative framework
In the preliminary stage of the design process, corresponding to the publishing of the tender documents by the public administration, a consistent number of the most influential decisions on the project are taken [9]. That's why it become crucial to provide both sides with shared assessment methodologies and evaluation tools, in order to establish a chain of actions characterized by transparency and data integration. About this, with reference to the new Italian Public Contract Code some critical points to be filled with the present methodology are highlighted: • The identification by the public administration of the most suitable framework of needs to be satisfied through the design process. In particular, the definition of the best intended use to be assigned with respect to the requirements of the community and the system of constraints related to the building and its context; • The disclosure by the contracting authority of the constraints burdening on the planning, with reference to the conservation needs, the environmental context and the legislative framework, to be translated into a set of inadmissible design strategies; • The multi-criteria analysis requested to the competitors for the evaluation of their design alternatives within the DOCFAP; • The methods and the criteria of choice for the identification of the best design solution by the designated commission. It is believed that all those findings could be fixed up by the introduction of a unique methodology of assessment and evaluation, able to equate both the investigation practices and the deliveries coming from the contracting authority and the competitors.
In view of the above and with reference to the case of reuse projects, the need for instruments that guides both administrators and designers in the achievement of a preliminary knowledge on the building is evident, in order to help understanding the performance adequacy and the attitude of transformation with its thresholds, with the aim of recognizing the impact that each transformative action could have on the existing building. Figure 1 displays the integration of the Re³ methodology within the Procurement Code, with particular emphasis given to the actors involved and to the outputs of each phase. The numbered boxes along with their content will be examined in the following.

Evaluation of the best function
In the case of an adaptive reuse project, before the design competition, during the meta-design and preanalysis phase, it's necessary for the public administration to identify, among several possibilities with regard to intended uses, the one that appears to be the most suitable with relation to the constraints imposed by the existing building and more respectful of its level of transformability. The pre-evaluation of the impacts that new uses will determine on the existing building is crucial to balance the conservation needs and the transformation requirements expressed by needs' framework.  This first stage is essential to start an effective process aimed at reuse projects, in order to be able to conceive design alternatives aware of their own impact on the building. The assessment on the grade of compatibility between the existing building and different intended uses is achieved by activating the same evaluation methodology for each possible solution (Figure 2). This passes through both new requirements and building assessment. The first is composed by the analysis of the requirements coming from the users, the regulatory framework and the energy saving necessity, while the latter includes the examination of the environmental, spatial and technological systems of building (Figure 3). Through the use of the PAV index, in fact, it is possible to estimate the residual performances of the building in relation to the needs expressed by the new users and relative functions. The comparison of the calculated indices ( Figure 4) is intended to be a support for the decision-making process of the contracting authority, which will identify the most suitable reuse function to be included as an element of the procurement within the tender documents, as it will be described in the next subsections.

Definition of the design contest tender documents
After the accomplishment of phase 1, where the choice of the most suitable function takes place, the administration proceeds to define the set of already mentioned tender documents made up of contest. The new requirements and the building assessments are here incorporated from the previous stage along with the financial limits and a set of possible design strategies [10]. The strategies has been classified in eleven cases divided in three main clusters with regard to the relationship they generate among the existing building and the proposed project: Addition, Insertion and Recladding [11] (Figure 5). Within this phase the identification of some selected strategies could be intended as a further recommendation or a technical procedure specified by the implementing regulation (Art. 3 Section 4). In order to establish the frame of constraints determined with regard to the urban context and to the building itself, a scheme containing the matrix of admissible/non-admissible design strategies will be provided ( Figure 6).
The choice of several design strategies among the whole is strictly related to each specific context and is aimed at the exclusion of those deemed difficult to be developed or not feasible under different profiles. That's why it will takes place in a qualitative way, by considering several factors: the legislative framework (environmental, urban, economic, structural, and energy saving) and the potentialities derived from the building and its performance assessment.
At the end of phase 2, the set of tender documents narrows the domain of the project actions, addressing it towards the achievements of specific objectives by the administration. However, the freedom to decline the single strategy or their combination is left to the designers through one or more project proposals that will be formulated, developed and analyzed during the next phases of the design contest.  Figure 6. Scheme of the composition of the tender documents according to the Re³ methodology.

Design contest
In the design contest, the competitors involved are called to develop and analyze multiple design solutions (including the so-called "zero option") and to expose the assessment of each in qualitative terms, under technical, economic and environmental profiles. Then, they will propose to the contracting authority the design alternative that gains the best ratio between costs and benefits, by considering the needs of users, the performance deficits of the existing building and the budget thresholds expressed in the tender documents. The competitors, with respect to the design strategies admitted in phase 2, develop projects that consider a single strategy or combinations of them. In this phase, the participants are asked to recalculate the PAV for a new assessment in relation to different design alternatives. Subsequentially RTE index is evaluated in order to understand the impacts of the project actions with reference to the transformability thresholds of the building. It is the task of the single designer to choose in the range of the proposed solutions which is the best project alternative, in relation to the two indices and the financial limits ( Figure 7).
The final DOCFAP, which each competitor should deliver at the end of the first phase of the competition, shows all the considered proposals, highlighting the one identified as the preferred one with the relative calculation of the PAV and RTE indices (Figure 8). In this way, the administration has the opportunity to compare quickly and easily the different proposed projects and to recognize which is the best one, with the aim of choosing the winner of the competition.
Thereafter the selected contractor will be called to complete his proposal in the next phases as defined in the implementing regulation within Art.8-14.

Conclusion and further developments
The research here presented allowed to identify and analyze the role that the Re 3 (Resilient Restoration and Reuse) assessment methodology could have when integrated within the Italian Procurement Code (Legislative Decree 50/2016), especially with reference to the evaluation of different design proposals in the cases of adaptive reuse projects. The purpose is to provide the contracting authorities and the competitors involved in the design competition of a shared and standardized workflow in order to realize a more integrated and transparent process within the evaluation of different design alternatives.
Future developments aim to the exploitation of theoretical and applicative research activities for the definition of a repeatable process endorsed by an IT (Information Technology) tool for supporting decision-making in adaptive reuse projects. In this way, it will be possible to transfer the research group know-how to the market thanks to the development of the IT tool and to the consultancy and mentorship in the use of the methodology, in both public and private sector. A possible final goal is to set up a commercial product in the form of either a stand-alone app, or a plug-in interacting with main existing building software already on the market such as BIM (Building Information Modeling) software and FM (Facility Management) platforms.


Flexibility of Indonesian text pre-processing library

Received Jul 27, 2018 Revised Aug 21, 2018 Accepted Nov 18, 2018 This study aimed to achieve and measure flexibility as a software quality factor of text pre-processing libraries with Indonesian text from social media. Library was built based on a review of some text mining applications that did not yet have a special pre-process for Indonesian text. Text pre-processing libraries were designed and built using an object-oriented approach that was modular to achieve flexibility. Flexibility was measured by the Mc Call Cyclomatic Complexity (CC) metric. Flexibility of library was tested by implementing the library into text mining applications. The results of experiment showed that text pre-processing libraries could be flexible and easy to use without much configuration in text mining applications. It was proved by the value of CC of 2.51 which meant the library or software was not too complex, simple enough, and also flexible to use.

INTRODUCTION
Software quality was one of the important parts that needed to be accomplished in software development. Software quality factors to be achieved in accordance with the needs and objectives of software built [1][2]. There were various models of software quality factors, such as McCall which was the beginning of the development of software quality models [3], Boehm software quality model [4], FURPS [5][6], International Organization for Standardization (ISO) [7], CMM (Capability Maturity Model) [8][9], and other models were widely used as the goal of software quality to be achieved [10][11][12]. The software quality model evolved according to the needs of the software, one of which was a library or component-based software quality model [13][14][15].
Text mining technique was a technique to find important values previously unknown automatically by extracting text data so that obtained useful knowledge [16][17][18][19][20][21]. One of the problems in text mining was to represent text that was unstructured data into a structured data representation before the mining process. The process of preparing the data into a structured representation was called the pre-processing stage until the data had ready for the mining process [22][23].
Seeing so rapidly the needs of technology [24], especially on text management with text mining techniques appeared a variety of Text Mining applications such as Weka, Kea, Mallet, LingPipe, GATE, Carrot2, MinorThird, Simmetrics, and so forth. Building libraries that pack preprocessing for text mining could be useful in streamlining and simplifying the mining process. Where libraries or software libraries contained modules that had classes and functions [25], collection of functions on functional programming, or class definitions on object-oriented programming. Library was usually used for code reuse with specific functions that exist in a library reused to streamline program code, so it did not repeat the program code with the same algorithm. The result of the pre-processing library was a structured text representation that could be used for mining processes performed by text mining applications.

421
To meet the needs of text mining applications that vary, in this study pre-processing libraries built must certainly be able to meet the quality factor of flexibility. Flexibility was achieved by providing abstract classes that could be implemented as needed for text mining applications and measured by McCabe Cyclomatic Complexity (CC) Metrics. Where the McCabe CC metrics could measure software complexity, the lower the CC software value the simpler, easier, and more flexible to use, modify, and maintain [26][27]. In addition, library flexibility was also tested by implementing libraries in text mining applications.

RESEARCH METHOD
This research used Research and Development method (R&D) [28]. We studied the literatures about text pre-processing as research phase. We analyzed the needs of text pre-processing, the characteristics of Indonesian language, the needs of library, and built the library as development phase. We used Waterfall Software Development Life Cycle (SDLC) method for built the library. Waterfall SDLC was used because it was simple SDLC [10], systematic, and requirements of text pre-processing library had defined clearly, although today many software development methodology that develop, such as Agile methodology [29]. Then, for modeling the analysis and design, we used Unified Modeling Language (UML), because of UML was suitable for object oriented analysis and design that support modularity to reach flexibility also [30][31].

Flexibility of Software Library
Flexibility was a software quality factor that was part of the category of product revision factor according to McCall, required to support software maintenance activities [3], [10], [13]. Likewise, in software component model, flexibility was sub quality or sub characteristic of maintainability [13]. Components had the ability to be modified in maintenance activities. Flexible components or software were easier to maintain [27]. Flexibility was different from adaptability that was a sub quality of portability [13], [15]. Adaptability was the ability of components to adapt in different platforms or environments without much modification [13]. Both flexibility and adaptability needed to had ease of use in a different environment. However, flexibility required components to be easily modified in the maintenance process, whereas adaptability required the ease of components when mounting in different environments.
Library as it was known as software reuse approach [32], so it needed to be flexible to change, in order to reduce the cost when maintained. Flexibility had sub quality including modularity and simplicity. Flexible software could be achieved by applying the principles of modularity, because the modular software reduced the complexity in the software. Modularity could be measured by evaluating cohesion and coupling software module design [10], [15], [33]. While simplicity would be measured by complexity metrics. One of the metrics for measuring flexibility and maintainability was the McCabe Cyclomatic Complexity Metrics [27]. By measuring the complexity of the library, it would also be measured libraries were easy to maintain or not.
McCabe cyclomatic complexity measured software complexity based on graph theory it was cyclomatic number, V(g)=e−n+ [26]. Where, V(g) is a cyclomatic complexity graph, e is the sum of edge, n is the sum of node, and p is saparated component or graph. The McCabe cyclomatic complexity used to calculate component complexity with V(g)=e−n+2 formulation. Where 2 was the addition of an additional edge from the exit node to the entry node on each module component. McCabe value was right if always that mean the value of component was 1. The limit of complexity value of cyclomatic complexity according to Software Engineering Institute among others 1-10 was a simple module without many risks, 11-20 mean more complex modules with multiple risks, 21-50 was a complex module with high risk, and more than 50 including programs which was untestable with a very high risk.

Indonesian Text Pre-Processing and Feature Extracting for Structured Text Representation
Doing text mining techniques was highly dependent on the language. This was because each language had unique characteristics, rules, spelling, and grammar. Especially for texts derived from social media, many natural languages, abbreviations, even slang languages were used so needed special handling. Therefore, it was necessary to understand well the grammar, in this study the Indonesian language, so that the result of the preprocess of the text was a good representation as well. Based on previous research, the text pre-processing produced structured text representation. Structured text representation data was ready for use in the mining process. Structured representation of a text in general there were two types, namely the form of single word or better known as the bag of words and the form of multiple words (n-gram). Bag of word was a structured representation of the text by collecting all the words in a text document without seeing the interrelationship between words [23], while the representation of multiple word was a text representation that collect words in text document by considering the interrelationship between words so that semantic meaning in the text  [34][35]. Sequential pattern was one form of representation of multiple words, so that compared with single words, the meaning or knowledge of text documents can be maintained better [19].
Sequential pattern used in the text was called Sequence of Word (SOW) with a sequence of words that consider the order of occurrence of words. There were 3 structured text representations in SOW form that could be used [33][34], among others, Frequent Word Sequence (FWS) which was the set of FWS with respect to the order of occurrence of each word. Then, the set of Frequent Word Sequence (Set of FWS) which was a set of FWS set that not only pay attention to the sequence of word appearance but also pay attention to sequence of occurrence of sentences. The last, Frequent Word Itemsets (FWI) which was a FWI set that did not consider the order of occurrence of words. There was also FWI association (Set of FWI) as the development of SOW which had been proven able to maintain the meaning of Indonesian slang language well [36][37][38]. The preprocessing library in this study used Set of FWI as a structured text representation. Flowchart of Indonesian text pre-processing library shown nig Figure 1.

Modul Design of Indonesian Text Pre-Processing Library
Based on the needs of Indonesian text pre-processing described in the third section, we built a text pre-processing library with the architectural design shown in Figure 2. There were three main modules, including the Text Preprocessing module, the Feature Extraction module, and the module Featured Selection. Each module had an abstract classes as a prototype that could be implemented flexibly. The text pre-processing module was a collection of classes that performed tokenizing processes, removing non-letter characters, manipulating regular expression, case folding, converting abbreviations to their original form, removing stopwords, and stemming processes. The Feature Extraction module is a collection of classes that function to extract features in the form of structured text representation, while the Feature Selection module contains a set of classes that serve to select the extracted features because a feature may have sub-features that can be removed. Because in this research, Indonesian text is taken from social media so it is more appropriate to use Set of FWI as representation of structured text, so algorithm used is frequent pattern algorithm. One of the frequent pattern algorithms is the FP-Growth algorithm that does not generate feature candidates, but builds tree structures, making it more efficient [39][40].  Figure 3. The value indicated the library was not too complex, quite simple and flexible (indicator value was gotten from Software Engineering Institute). However, there were some classes that had a cyclomatic complexity value greater than 10, even the PorterAlgoGaul.java class had a value of more than 50 which means it was very complex and inflexible.

The Result of Testing on LingPipe 4.1.0
LingPipe was an application for text processing by using linguistic computing that could perform functions such as searching for names of people, organizations, or locations in the news, automatically classifying search results in categories, and correcting the correct spelling of a statement. LingPipe 4.1.0 could accept input data in plain text and xml. In the LingPipe app, the function that was invoked from the preprocessing library was readDir() from the class FrequentItemsetFromMultipleDir to generate the getFwi() feature of the FrequentItemset class to used features in the model development process, as well as PreprocessingTestFileToString() from the TextPreprocessing class to perform preprocesses on the data for testing. LingPipe received input data .txt, so library preprocessing could be directly used without changing the input type first. Examples of library tests for classification with DynamicLM algorithms were available in the LingPipe application. We used data from Twitter and Blog with three scenarios, among others: From 170 testing data, 136 data was classified correctly, among others 115 data in "kampanye" class, and 21 data in "tahunbaruislam" class; from 242 testing data, 87 data was classified correctly, among others 9 data in "agama" class, 15 data in bisnis class, and 63 data in "komputer" class; and total test data from blog was 48 with only 4 data correctly classified.

The Result of Testing Kea 5.0
Kea was a text-processing application that could perform tokenizers by extracting phrases, performing stopword removal and stemming, and modeling the process of document training and model testing. Models were built using the Naïve Bayes algorithm. Kea contracted the phrase of a document and the phrase was used as a keyword to train document data and as a label for classification. Kea had no algorithm for clustering process. The input for the data train process was a .txt file containing text for the train data and a key file with the same name as the .txt file. This .key file contained keyphrases for each .txt file. To prepare a phrase.key file containing phrases it could use the readDir() function of the FrequentItemsetFromMultipleDir class to generate features that could be used as keyphrase, getFwi() from the FrequentItemset class to use features in the model development process, as well as PreprocessingTestFileToString() from the TextPreprocessing class to perform preproces on training or test data. So that the test and training data had been done tokenize process, stopwords removal, and stemming. Because Kea did not provide stopword and stemming Indonesian language. Kea was very dependent on the language used, so if setVocabulary had a "none" value the resulting model could not be tested. Kea accepted only English, Spanish and French input. This was why the model with Indonesian produced could not be tested, but the model was successfully established.

The Result of Testing Mallet 2.0.6
Mallet (MAchine Learning for LanguagE Toolkit) was a Java-based package for the processing of natural language statistics, document classification, clustering, topic modeling, information extraction, and other machine learning applications for text. Mallet included powerful tools for document classification: efficient routines for converting text to "features", various algorithms (including Naïve Bayes, Maximum Entropy, and Decision Tree), and code for evaluating classifier performance using some commonly used metrics. As with LingPipe and Kea, preprocessing libraries could be added to Mallet. Same as the LingPipe application, the function was called from the preprocessing library was readDir() from the FrequentItemsetFromMultipleDir class to generate the getFwi() feature of the FrequentItemset class to used the features in the model development process, as well as the PreprocessingTestFileToString() of the TextPreprocessing class to perform preprocesses on the data for testing. Mallet also received input data .txt, so the preprocessing library could be directly used without changing the input type first. There were several examples of library testing with the same train and test data as in the LingPipe library test, for the process of classification with the Naïve Bayes algorithm. The classification results had an average accuracy of about 96.5%, this result was certainly influenced by good text data pre-process results.

CONCLUSION
Software needed to achieve quality in order to continue to be used, and easy to maintain and develop. There was a quality factor software that could be used as a measure of what quality to be achieved. Componentbased software, including libraries, which were collections of modules, classes, or functions, have quality factors that needed to be achieved. One of them was the quality factor of flexibility. The pre-processing stage in text mining was an important stage that could affect the mining results. However, no many text mining applications had a complete pre-processing phase, especially for Indonesian text. Because, in the text mining needs and characteristics of each language on text data to be processed differently.
This research built and measured the flexibility of text pre-processing libraries for Indonesian text, ranging from tokenizing stages to forming structured text representations. The flexibility of library 425 measurement using McCabe cyclomatic complexity metrics showed that text pre-processing libraries for Indonesian texts were flexible enough, simple, and easy to modify. This was proved by implementing library in 3 text mining applications, including LingPipe 4.1.0, Kea 5.0, and Mallet 2.0.6. Each application utilized the pre-processing library as a pre-process stage for document classification techniques. In addition, the library provided some abstract modules that could be implemented according to the needs of text mining applications. However, of course the programming language used affects the use of libraries, because the library could not be used in applications with different programming languages. Therefore, libraries must achieve with other quality factors, such as portability, adaptability, or interoperability that could be implemented in different environments (both languages and operating systems).


Sentence transition matrix: An efficient approach that preserves sentence semantics

Sentence embedding is a significant research topic in the field of natural language processing (NLP). Generating sentence embedding vectors reflecting the intrinsic meaning of a sentence is a key factor to achieve an enhanced performance in various NLP tasks such as sentence classification and document summarization. Therefore, various sentence embedding models based on supervised and unsupervised learning have been proposed after the advent of researches regarding the distributed representation of words. They were evaluated through semantic textual similarity (STS) tasks, which measure the degree of semantic preservation of a sentence and neural network-based supervised embedding models generally yielded state-of-the-art performance. However, these models have a limitation in that they have multiple parameters to update, thereby requiring a tremendous amount of labeled training data. In this study, we propose an efficient approach that learns a transition matrix that refines a sentence embedding vector to reflect the latent semantic meaning of a sentence. The proposed method has two practical advantages; (1) it can be applied to any sentence embedding method, and (2) it can achieve robust performance in STS tasks irrespective of the number of training examples.

Introduction
Sentence embedding, the task of transforming a sequence of words in a sentence into a fixed-dimensional vector form reflecting the intrinsic meaning, plays an important role in the field of natural language processing (NLP). It can be considered as an essential preprocessing step that transforms unstructured textual data into structured and continuous-valued vectors that can be used as input to machine learning algorithms to conduct various NLP tasks such as machine translation (Sutskever et al., 2014;Wu et al., 2016), document classification (Kim, 2014;Conneau et al., 2017c), and sentence matching (Hu et al., 2014;Wan et al., 2016). Because performances of many NLP tasks heavily rely on the word/sentence/document embedding methods, a large number of studies have been conducted since the advent of Doc2vec (Le & Mikolov, 2014), and progressive performance improvements have been reported whenever a new embedding method was proposed.
A good sentence embedding model should yield a vector value that closely captures the intrinsic semantic meaning of the sentence. Therefore, diverse experiments predicting the similarity score of two embedded sentences have been designed to evaluate how well the embedding model satisfies such requirements. These experiments were conducted in many researches, such as SIF (Arora et al., 2017), Sent2vec (Pagliardini et al., 2017), InferSent (Conneau et al., 2017a), and Fast-Sent (Hill et al., 2016a), and models trained through supervised learning generally showed far better performances than those based on unsupervised learning. However, supervised models have a limitation that they require a sufficient number of labeled training data to learn the model. For instance, Stanford Natural Language Inference (SNLI) dataset, which contains a collection of 570k manually labeled human-written English sentence pairs (Bowman et al., 2015), was used to train the InferSent model. Because it is an English dataset, one must construct a new dataset to train the embedding model for other languages, e.g., Russian, Korean, and Japanese, which is a heavy time-arXiv:1901.05219v1 [cs.CL] 16 Jan 2019 and resource-consuming work. Therefore, to increase the general usability of an embedding model, it is necessary to reduce reliance on labeled training data while closely preserving the intrinsic meaning of the sentence.
In this study, motivated by the studies that obtained successful results in cross-lingual embedding (Mikolov et al., 2013a) and word-level translation (Smith et al., 2017) by training a simple transform matrix, we propose an approach to learn the transition matrix that refines the sentence vector generated by other sentence embedding models. The contributions of this research are as follows: • We propose an efficient method that successfully reflects sentence semantics through training the transition matrix.
• The proposed method is independent of the size of training data and can be applied to any kind of sentence embedding model.
To reflect the intrinsic meaning of a sentence completely, we define a fundamental property that a good sentence embedding model should satisfy: the property of semantic coherence (Jang & Kang, 2018), which implies that paraphrase sentences should be closely located to each other in the sentence embedding space. Next, we derive an objective function to learn the transition matrix to closely satisfy the formulated property and train the matrix using the MSCOCO caption dataset. Experimental results show that the proposed approach is practically advantageous in that it outperforms existing sentence embedding models in various semantic textual similarity (STS) tasks.
The rest of this paper is organized as follows. In Section 2, we briefly review the past researches on sentence embedding. In Section 3, we introduce the objective function for training the transition matrix and the training method. In Section 4, the effectiveness of the proposed method is demonstrated with the experimental results on STS tasks, along with their performance comparison with benchmark models. Finally, in Section 5, we conclude our present study, which leads us to some future research directions.

Related work
Recent researches on sentence embedding models are diverse for models based on unsupervised learning to those based on supervised learning. Unsupervised embedding models can be divided into two categories based on whether sentence sequence information (not word sequence information within a sentence) is required during training.
Doc2vec (Le & Mikolov, 2014) is a representative model that does not need sentence sequence information. Paragraph vectors-distributed bag of words (PV-DBOW) and paragraph vector-distributed memory (PV-DM), two distinct learning methods of Doc2vec, train sentence vectors based on the same objective: maximizing the probability to predict words constituting the sentence. The probability is defined as the dot product between a sentence vector and a word vector. PV-DM considers sequential information of words by employing a moving window. In this method, a sentence vector is learned to predict a word appearing after the moving window using the words within the window and the sentence vector. However, in the PV-DBOW method, words included in the window are arbitrarily selected. Therefore, this is incapable of reflecting sequential information of words in a sentence. Hill et al. (2016b) proposed the sequential denoising autoencoder (SDAE), which slightly corrupts the input sentence by adding random noise. The noise is added in two different ways. First, each word is randomly dropped in a sequence according to the probability p 0 . Next, for the bigrams that are not overlapped, the order of two adjacent words is permuted according to the probability p x . Then, the embedding model, which is a recurrent neural network (RNN) with a long short-term memory (Hochreiter & Schmidhuber, 1997) (LSTM) cell, updates its parameter to generate the original sentence from the corrupted one. If p 0 = p x = 0, the model becomes a sequential autoencoder (SAE), which does not add noise to the input. Hill et al. (2016b) also proposed a variant of the SDAE model that employs fixed pre-trained word vectors. This model is notated as "S(D)AE + embs" in our study. Arora et al. (2017) proposed a simple embedding model named SIF, which computes a sentence vector as a weighted average of fixed pre-trained word embedding vectors. Despite its simplicity, SIF accomplished improved performance in STS tasks and outperformed many complex models based on RNNs if word weights are properly adjusted. Sent2vec (Pagliardini et al., 2017) has a similar characteristic with SIF, which computes a sentence vector as a weighted average of word embedding vectors. However, Sent2vec trains not only the embedding vector of words that are unigram, but also that of n-gram tokens. It is different from SIF in that it employs n-gram embedding vectors to generate sentence vectors.
Contrary to previously demonstrated sentence embed-ding models that exploit information from corpora to learn sentence vectors, C-PHRASE (Kruszewski et al., 2015) requires external information. C-PHRASE uses information from the syntactic parse tree of each sentence. This additional knowledge is included in the training objective of C-BOW (Mikolov et al., 2013b).
SkipThought  is a sentence embedding model in which sentence sequences are mandatory during the training. It has a sequence-to-sequence structure and expands the training objective of Skipgram (Mikolov et al., 2013b) for learning word embedding vectors to a sentence level. Similar to the Skipgram model, which updates word embedding vectors by predicting the surrounding words when the center word is given, the training objective of SkipThought is to generate the preceding and following sentences when a sentence is given.
FastSent (Hill et al., 2016a), similar to SkipThought, is a sentence embedding model aimed at predicting the surrounding sentences of a given sentence. Fast-Sent learns the source word embedding u w and target word embedding v w . When three consecutive sentences S i−1 , S i , S i+1 are given, s i , the representation vector of S i , is calculated as the sum of the source word embedding vectors: Then, the cost function is simply defined as follows: In addition, Hill et al. (2016a) also proposed a variant model (FastSent+AE) that predicts not only the adjacent sentences but also the center sentence S i . Fast-Sent with such a simple structure, takes much less training time than SkipThought.
Siamese C-BOW (Kenter et al., 2016) shares a common concept with SIF and Sent2vec: defining a sentence vector as the average of word embedding vectors. In addition, it is similar to SkipThought and FastSent in that it is also trained to predict surrounding sentences when the center sentence is given. However, Siamese C-BOW employs a Siamese neural network (Koch, 2015) structure, which is significantly different from the sentence embedding models described above.
InferSent (Conneau et al., 2017a) is a sentence embedding model trained through supervised tasks. Inspired by previous researches in computer vision, where a large number of models are pre-trained through a classification task based on the ImageNet (Deng et al., 2009) dataset, Conneau et al. (2017a) performed a research to determine the effectiveness of supervised tasks in the learning of a sentence embedding model in the field of NLP. Through experiments, Conneau et al. (2017a) concluded that a sentence embedding model having a bidirectional LSTM structure, trained on the SNLI dataset, yielded state-of-the-art performance in various NLP tasks.
All sentence embedding models using sentence sequence information, described above, require a specific dataset to train the model. For instance, SkipThought and FastSent used the Book Corpus dataset  and InferSent used the SNLI dataset to train the embedding models. In the case of sentence embedding models independent of sentence sequence information, a large document dataset, such as Wikipedia sentences, was also used. Furthermore, these models have a limitation in completely reflecting the traits of paraphrase sentences conveying the same meaning but having a different word usage, because these models generate sentence vectors based on word embedding vectors. In this study, we propose an efficient approach that successfully preserves sentence semantics by employing only a small amount of paraphrase sentences.

An efficient transition matrix for generating sentence embedding vectors
In this section, we first define a fundamental property that a good sentence embedding model should satisfy. Next, we describe the method to learn the transition matrix from the derived property. Finally, we demonstrate the processing method for training data when learning the transition matrix.

Property of semantic coherence
Semantic coherence refers to a fundamental property that a good sentence embedding model should satisfy: if two sentences have similar meaning, they should be located close to each other in the embedding space. The semantic coherence of a sentence embedding model can be evaluated as follows: Definition: The degree of semantic coherence of a sentence embedding model is proportional to the similarity between the embedding vectors of paraphrase sentences generated by the sentence embedding model.
The above definition can be expressed mathematically. Assume that the set I is a vector set of input sentences and the set P is a vector set of paraphrase sentences corresponding to the input sentences: where (i k , p k ) is a pair of paraphrase sentence vectors. Then, the similarity matrix of elements belonging to each set can be computed as follows: The normalized similarity matrix N can be obtained by the element-wise multiplication of N withŜ as follows: The tupple (i i , p j ) is a pair of sentence vectors sharing a similar intrinsic meaning when i = j. However, when i ̸ = j, the sentence vectors have different meanings. Therefore, the similarity of two sentence vectors i i and p j should be close to 1 when i = j and should be minimized when i ̸ = j. Hence, the similarity matrix S of two vector sets I and P should satisfy the following two conditions if the sentence embedding model fulfills the property of semantic coherence.

Sentence transition matrix
In this study, we attempted to devise an approach satisfying the semantic coherence by learning a minimal number of weights. To achieve this, we trained the transition matrix, which elaborates the pre-generated sentence vectors to meet the conditions described above. Assume that I M and P M refer to the sentence vector sets generated by the sentence embedding model M of the input sentence and the corresponding paraphrase sentences, respectively.
The main purpose of this study is to train the transition matrix W using I M and P M . First, each set of sentence vectors is multiplied by the transition matrix as follows:Î Then, the similarity matrix of the sentence vectors to which the transition matrix is applied is computed as follows:Ŝ Next, from the formulated conditions of the semantic coherence property, the diagonal and non-diagonal losses are defined as follows: (9) Finally, the final training loss is defined as follows: where λ is the user-specific hyperparameter that controls the trade-off between the two losses. The sentence vector for sentence x after training the transition matrix is computed as follows: where M(x) is the vector value of the sentence x generated by model M.
Model M can be any kind of sentence embedding model. However, we defined the average of pre-trained word vectors as a sentence vector to show that the performance improvement could be obtained by a simple method. We used two kinds of pre-trained word vectors: GloVe vectors (Pennington et al., 2014) and Google Word2vec 1 .

Training transition matrix
Composing training dataset: To train the transition matrix, we employed the MSCOCO 2017 training dataset (Lin et al., 2014), which is widely used as a paraphrase dataset in many researches (Prakash et al., 2016;Gupta et al., 2017). This dataset contains a minimum of five captions per image for 118,284 images. The total number of unique captions is 591,753.
To train the transition matrix proposed in this study, i i and p j should be mutually exclusive except for the case when i = j. In other words, in a mini-batch, only one paraphrase sentence should be included for one corresponding sentence when training the transition matrix. Hence, we constructed the training dataset as explained below.
First, note that it is possible to create ten unique sentence pairs for each image because at least five captions are provided for each image. Therefore, for all images, if we include only one sentence pair per image in one sub-training set, then the sub-training set does not contain sentence pairs with overlapping meaning and totally ten sub-training sets can be constructed. Training option: Both GloVe vectors and Google Word2vec have word vectors of 300 dimensions. Therefore, the transition matrix has a size of 300 × 300. We employed Xavier initialization (Glorot & Bengio, 2010) and weight updates were performed with a mini-batch size of 512. We trained the transition matrix for five outer epochs using the RMSprop optimizer 2 .

Textual similarity task
Data set: To evaluate how well the proposed approach reflects the intrinsic meaning of sentences, we conducted STS tasks (2012-2016) (Agirre et al., 2012;2013;2014;2015; and SemEval 2014 semantic relatedness task (SICK) (Marelli et al., 2014). The objective of these tasks is to predict the similarity score of two given sentences. Their performance is evaluated by comparing the predicted score with the ground truth, which is the similarity score determined by human judgments. The evaluation metric is Pearson's r (Pearson, 1895) and Spearman's ρ (Spearman, 1904). Similar to previous researches, we defined the 2 http://www.cs.toronto.edu/ tijmen/csc321/slides/lecture_slides_lec6.pdf predicted similarity score as the cosine similarity between two sentence vectors.
Experiment design: We conducted the experiments in three different levels based on the number of training data to investigate the relationship between the performance and size of training data. The first level is the case of using all available data. The second and third levels are the cases where only 50 % and 10 %, respectively, of the total training data are used.
In the proposed approach, the hyperparameter λ is a key value affecting the model performance. In this experiment, we empirically determined λ from a sufficient number of candidate values. The λ values for GloVe vectors and Google Word2vec are set to 0.7 and 0.9, respectively.

Effect of the transition matrix on performance
We first evaluated the effect of the transition matrix by comparing the performances before and after applying the transition matrix to sentence vectors that are defined as the average of pre-trained word vectors. The result is summarized in Table 1 and Figure 2. The notation "TM" denotes that the transition matrix is applied, and the number in the parentheses indicates the percentage of training data used. The evaluation metric recorded in Table 1 is Pearson's r.
Experimental results show that the proposed transi-  tion matrix remarkably improves the performance in both the pre-trained word vectors. The training time was about 130, 60, and 13 s when 100, 50, and 10 %, respectively, of the training data was used to train W.In the case of Google Word2vec, the performance improvement is 5 ∼ 16 %, while the transition matrix with Glove vector shows a very high performance improvement of 13 ∼ 49 %. Both the transition matrices do not show significant performance differences according to the amount of training data. Therefore, it can be concluded that the performance of a transition matrix is independent of the amount of paraphrase data used in training.

Comparison with other sentence embedding models
After confirming the effectiveness of the transition matrix, we compared its performance with previously proposed sentence embedding models. The benchmark models are as follows: • Embedding models using sentence sequence information: S(D)AE, PV-DBOW, PV-DM, Sent2vec, C-PHRASE, and SIF.
• Embedding models not using sentence sequence information: SkipThought, FastSent, and Siamese C-BOW.
• Supervised task-based sentence embedding model: InferSent The tasks used for comparison are the STS 2014 and SICK relatedness tasks, both of which were used in the benchmark models. The benchmark results were obtained by Pagliardini et al. (2017), Arora et al. (2017), and Conneau et al. (2017a). For the models discussed above, the result of Pagliardini et al. (2017) contains the model defining a sentence vector as the simple average of word embeddings trained by the C-BOW (Mikolov et al., 2013b) method. It also includes the result of TF-IDF representation, which is the weighted word count of the 200,000 most common words. We underlined the best performance for the dataset, while  Table 2.
The experimental results show that the proposed approach is ranked among the top three for most of the datasets despite its low computational complexity. We further compared the performance ranks of the proposed models with three selected benchmark models: Sent2vec, C-BOW, and C-PHRASE. The selection criteria are: (1) the model was tested for both STS 14 and SICK experiments, and (2) it performed the best for at least one dataset. For each dataset, we recorded the performance ranks of the models and then compared the distribution of the ranks and its average. Figure 3 shows the box plots of the performance ranks for the selected ten models. The triangles denote the average performance ranks. The results show that the model which used Glove vectors and 10 % of the total paraphrase sentences yielded the lowest average rank in both Pearson's r and Spearman's ρ. In addition, in the case of Pearson's r, most of our proposed models resulted in lower average ranks than those of Sent2vec, which showed the best performance among the benchmark models.
Comparison with SIF: SIF has characteristics similar to our approach, i.e., it also computes a sentence representation vector from fixed pre-trained word vectors in a simple manner. In Table 2, we were not able to compare the performance with SIF in detail because Arora et al. (2017) did not specify the experimental result for the individual datasets of the STS 2014 task. However, because Arora et al. (2017) performed STS 2012-2015 tasks and the SICK semantic relatedness task, we compared the performance of our approach with SIF in more detail, as shown in Table  3. We recorded the Pearson's r of the best model for each pre-trained word vector. The experimental results reveal that our proposed approach showed a better performance than SIF for all the datasets. We also observed that the average performance improvement  ratio was 6 % and the performance was enhanced to a maximum of 16.5 % compared to that of SIF.

Conclusion
Sentence embedding, which transforms an unstructured textual data into a structured vector form, is a fundamental and imperative method in the field of NLP. Generating sentence vectors that preserve the semantic meaning of a sentence is the key component to achieve an improved performance in various NLP tasks. Hence, various sentence embedding models have been proposed, which show an enhanced performance in various NLP tasks such as document classification, sentiment analysis, and semantic similarity task.
In this study, we first defined the property of semantic coherence that closely preserves the sentence semantics. Subsequently, we derived the objective function to train the transition matrix, which refines sentence representation vectors to satisfy the derived property. Finally, the transition matrix is trained by employing the MSCOCO caption dataset, which is widely used as a paraphrase dataset.
The proposed approach was evaluated through various semantic textual similarity tasks. Despite its low computational complexity, our approach showed significant improvements in various STS tasks. In addition, compared to the previously proposed benchmark models, our approach showed the best performance in many datasets and one of the best performances in almost every dataset.
Nonetheless, the proposed method has a limitation that paraphrase data is required to train the transition matrix. Although the experimental results show that the performance of a transition matrix is independent of the amount of training data, it can be a critical issue when analyzing the languages which are difficult to obtain paraphrase data. Therefore, similar to researches on unsupervised machine translation (Artetxe et al., 2017;Lample et al., 2017; and unsupervised cross-lingual embedding (Wada & Iwata, 2018;Xu et al., 2018;Conneau et al., 2017b), both of which do not require labeled data, a research to generate sentence vectors preserving the latent meaning of sentences without paraphrase data should be developed.