Ubiquitous Computing in Precision Agriculture: A Systematic Review

The applications of ubiquitous computing have increased in recent years, especially due to the development of technologies such as mobile computing and its integration with the real world. One of the challenges in this area is the use of context sensitivity. In agriculture, this can be considered as the context related to the environment, such as the chemical and physical aspects that characterize the different soil types. This scenario periodically changes due to factors such as climate, type of cultivar and soil management technique used, among other aspects. This article presents a systematic review on the research works that explore ubiquitous computing in precision agriculture, including which technologies are being currently applied and which gap scan still be researched. Nine scientific repositories were explored to find articles about precision agriculture and ubiquitous computing. As a result of this search and filtering process, 32 works were reviewed, analyzed and categorized between the years of 2009 and 2019. In general, the reviewed articles concentrate on problems arising from the communication between sensors and the management of context-sensitive data.

Introduction
Precision agriculture is a suitable strategy to increase productivity, which al-lows the rational use of inputs and reduces the environmental impacts caused by agricultural practices. Currently, the inputs are used in a variable way to meet the specific needs of each location, thus optimizing the production process. However, it is necessary to characterize the soil spatial variability to check chemical and physical attributes through several representative sampling (Costa,  Generally, computing research has aimed to develop techniques to integrate information technology into people's daily lives, so that they are proactively assisted by technology while they execute their daily activities (Weiser, 1999). Ubiquitous computing seeks new forms of communication and interaction that are distributed in the environment, either in a perceptible or imperceptible way. Furthermore, through the use of sensors, computers can detect and extract data from the environment, which helps users to perform their tasks (Satyanarayanan, 2001).
Context-aware applications are necessary for this vision to become real-ity (Dey et al., 2001). Context means any information that allows the characterization of an entity situation that is relevant to the interaction between a user and an application, which includes information about the situation, identity and lo-cation of people, groups and physical or computational objects. Through the knowledge of contextual data, an application can adjust its own functioning or even act proactively, such as by alerting users ' to a specific scenario or aiding them to develop activities more efficiently. The generated information will enable the construction of a historical database for posterior decision making (Hong et al., 2009;Ciaramella et al., 2010).
This article uses the systematic review methodology developed by Petersen et al. (2008) to conduct a review of the use of ubiquitous computing in agriculture. Guided by the search and two-phase processing, this article concentrates on discovering the main authors in this area. This article also looks for the most relevant works, in addition to the possible research gaps and their challenges.

Materials and methods
This article uses a systematic review methodology to" identify, analyze and interpret all the available evidence related to a specific research question" (Kitchenham & Charters, 2007), which in this case are relevant to the application of ubiquitous computing imprecision agriculture. This type of methodology not only discusses the conclusion but also examines all of the activities related to the discovery. Hence, a systematic study collects data when the activity occurs and the media in which it was published, and then maps this connection (Cooper, 2016). The methodology consists in the execution of the following steps: a) establish research questions; b) design the process of the research and c) define criteria for filtering results.

Research questions
The research questions led this study to discover works related to the theme. The goal of these questions is to understand how ubiquitous technologies are being used to help in precision agriculture. It is also desirable to find how specific technologies are applied between agriculture and context awareness. Finally, clusters of commonly used terms within selected articles were generated to support the identification of academic research interest trends. Therefore, five questions were established and presented in Table 1.

RQ1
Which technologies support precision agriculture?

RQ2
Where are these technologies being applied in precision agriculture?

RQ3
How is ubiquitous computing being used to support precision agriculture?

RQ4
Which are the main clusters of research that express the terms ubiquitous computing and precision agriculture?

RQ5
What is the number of publications per database and per year?
Source: own processing  Petersen et al. (2008) defined three stages of a research process: specify the search string, choose the databases to apply them and then get the results. The first stage starts by identifying the keywords and their related terms. In this study, we chose the keywords "Ubiquitous" and "Agriculture" and also other related terms, as indicated in Table 2.  These terms generated the following search string to be used in the search databases:((ubiquitous OR context-aware OR context-sensitive OR context awareness OR pervasive OR internet-of-things OR iot) AND (agriculture OR agronomy OR soil)). The term "soil" was inserted in the search string because the term "agriculture" covers other sub-areas of application, such as aquaponic, permaculture, indoors agriculture, organic, subsistence, among others. This enabled this study to filter only the works related to intensive agriculture applied to the soil; that is, that have high productivity, large extensions of land and the use of modern techniques and mechanizations.

Research process
Once the search string was defined, we constructed the research parameters to be used on the databases. In the second step, we selected eight re-search databases relevant to the area of computing, including the ACM Digital Library, Semantic Scholar (Citeseerx), Google Scholar, IEEE Xplore Digital Library, Scopus, Science Direct, Springer and Wiley Library. Research in the ACM Digital Library required the use of advanced search features, where each of our strings was inserted in the "Edit Query" tool. Similarly, this practice was applied in the Semantic Scholar (CiteSeerX), Google Scholar, IEEE Xplore and Scopus databases.
Only in the Science Direct and Google Scholar base was the string applied in a simple search box that is available on the main page of these sites.

Filters application
To filter the most relevant works, we generated the following Inclusion Criteria (IC) for this selection: [5] Ubiquitous Computing in Precision Agriculture: A Systematic Review -IC1: The study must be published in a conference proceeding or journal.
-IC2: The study must be related to the context of use of ubiquitous com-putting in agriculture.
-IC3: The study must be a full paper.
In turn, the Excluding Criteria (EC) were also defined, as follows: -EC1: Studies published before 2009.
-EC2: Studies that are not written in English.
-EC3: Studies related to theses or dissertations.
-EC4: Studies that are not related to research questions.
The inclusion and exclusion criteria helped the filtering process to obtain the most relevant studies and to eliminate any noise generated during the search. The studies obtained in the search process were filtered, and those that did not fit in the inclusion criteria were removed. The articles were stored in the Mendeley Desktop© program and organized in specific folders to each database. The next step was to analyze the works by title and abstract, so that they could be later combined in the same folder. Six articles were added by heuristic because they are relevant for this study, even though they were not found during the search process.
The next filter was based on the first two passes of the three-pass approach introduced by Srinivasan Keshav (Keshav, 2007). The first pass is a quick sweep, consisting of: 1) reading the title, the abstract and the introduction; 2) reading only the heading of the section and subsection, but ignoring all the rest; 3) looking the mathematical contents (if there is any) to determine the underlaying theoretical fundamentals; and 4) reading the conclusions. The second pass consisted of carefully analyzing the figures, diagrams and any other illustrations in the article, giving special attention to the graphics. Finally, the remaining articles were filtered by the analysis of the full text and the observation of the exclusion criteria EC4. Figure 1 presents the filtering process, with IC and EC applied at each stage, with the Scopus and Science Direct databases bringing together more assertive works related to the search string with 46.4% of the filtered results. The Semantic Scholar and Wiley databases presented many unrelated works. Figure 2 shows the result of this processing before the article combination stage. The filtering process also brought works related to subareas of computation, such as hardware architecture and networks applied to precision agriculture.
Source: Own processing [6] Ubiquitous Computing in Precision Agriculture: A Systematic Review

Threats to validity
As in any work involving a systematic review, this research presents some risks that can affect the obtained results. These risks are directly related to the data filtering process. To reduce the risk margin, this research searched data in nine different databases, selected from their relevance in the areas of ubiquitous computing and agriculture.
The research string was constructed considering the main terms and some related words to ensure the greatest possible coverage of keywords in the search, avoiding an ineffective search.
Source: Own processing

Results and discussion
This section details the survey results obtained by reading and analyzing the 32 mapped studies.
In addition, the research questions were answered and additional discussions and analysis on the studies were presented.

RQ1 -Which technologies support precision agriculture?
The technologies that support precision agriculture were analyzed and categorized as IoT & Sensors Applications, Architecture Model, Semantic & Ontology and, finally, Wireless Network Sensor (WSN). Table 3 maps the works with their categorization.
Based on this question, most of articles focused on WSN. There is a big difference between IoT and WSN, which causes these terms to have been categorized separately. In an IoT system, all of the sensors directly send their information to the Internet, such as soil temperature and moisture. In this case, a direct connection to the Internet will be open immediately or periodically to synchronize data. Already in a WSN, the various sensors connect to some kind of router or central node. A large collection of sensors, as in a mesh network, can be used to individually gather data and send data through a router to the Internet in an IoT system. In other words, WSN is a subset of IoT.
Source: Own processing  37.50% [7] Ubiquitous Computing in Precision Agriculture: A Systematic Review

RQ2 -Where are these technologies being applied in precision agriculture?
Analyzing the results according to Table 4, most technology applied in precision agriculture was about improvements of communication between sensors, in the same direction that RQ1 appointed. network communication followed by the soil analysis and context sensitive applications. In this last case, the control centers collect and process data in real time to help the farmers to make the best decisions related to planting, fertilizing and harvesting.
Source: Own processing

RQ3 -How is ubiquitous computing being used to support precision agriculture?
According with Table 4, there is currently a big gap (56.25% of the articles) in improving communication between the distribution of the sensors and the base. Technology such as WSN has boosted agricultural research because there is no longer a need for cables to receive the information but instead solutions are available to avoid the loss of data, to enable synchronization and to improve sensor power efficiency.
A total of 14 articles were identified that used the contexts as defined by Dey et al. (2001). The main goal of these researchers was to improve the use of technology in favor of increased production in the planted area. The major challenge of ubiquitous computing is related to the need of the applications be context-sensitive so that, when appropriate, they respond through decision making. All of these studies of context awareness that were selected in the mapping are presented in Table 5. Figure 3 presents the result of a bibliometric mapping tool known as VOSViewer, which was used to identify relevant works by publication year (Van Eck & Waltman, 2009). It is also possible to verify the common terms within publications and interest relating to ubiquitous computing and precision agriculture. All of the selected studies focus on four terms, in order of relevance: sensor network, context, sensor node and soil. The WSN is one of the most promising technologies for the agricultural sector. WSN enables advancements in ubiquitous computing due to their availability, small size and low price, resulting in an easy and cost-effective implementation. Figure 4 presents the publications grouped according to the year of publication. However, it should be noted that the year of creation of this article (2019) is still on going and other articles could compose this statistic. In the last six years, the number of publications on agriculture and ubiquitous computing increased when compared with the rest of the beginning period, except 2015. This growth shows the interest of researchers in improving agriculture, as well as in improving the quality of products through the better monitoring of production. The number on the top of peaks indicates the total of publications, excluding duplicates.

Discussion
Only 10 studies or 31.25% of the articles used IoT and Sensors to support precision agriculture (Table 3). Among these, the most popular technology was WSN with 12 articles. [8]

Ubiquitous Computing in Precision Agriculture: A Systematic Review
Source: Own processing This work proposed and illustrated the implementation of a new method for delineation of management areas using satellite images to understand spatial variation within a field and to optimize the use of agricultural inputs (seeds, agrochemicals and soil amendments). (Phillips, Newlands et al., 2014) This work discussed a series of key recommendations for monitoring the soil moisture dynamics at a field scale to integrate with remote sensing and decision support models. It also concluded that for integrated sensing to be utilized for long-term operational monitoring and decision support web services, soft-ware tools and analysis tools need to be developed. Thus, data from multiple sources could be analyzed and integrated with various models for crop production, food security and environ-mental change. This work recommends that disease treatments for farmers plants must be considered from a set of related observations. Thus, it developed a theoretical framework for systems to man-age a farmer's observation data. It introduced the representation of observation data, called warn cons, based on the user's context information aiming to create a representation of the advice data  By analyzing the entire filtering process presented in Figure 1 and the Research Question 1 (RQ 1), which presents in which databases the articles were published, it is possible to verify that the most accurate databases that processed the query string were Google Scholar and Science Direct because they had a small number of articles in the initial survey but have a relatively high number when compared to the total articles used in the mapping. The least accurate database was Semantic Scholarof the 81 papers in the initial search, only one study was used after the last filtering.

Conclusion
This systematic review has presented the state-of-the-art in the application of ubiquitous computing in precision agriculture. Furthermore, it also presented different applications Source: Own processing [10] Ubiquitous Computing in Precision Agriculture: A Systematic Review of technologies associated with computing for better results in agricultural production.
Although many of the selected papers aim to solve WSN problem, one of the gaps that was found during the evaluation of this research corresponds to the application of historical data in precision agriculture. In particular, none of the articles mentioned the use of a historical database and how this collected data could effectively improve production with the support of mobile technology.
Contextual data could lead to a three-dimensional spatial variability of soil conditions, such as fertility, moisture, pH, macro and micronutrients, and other soil attributes. This type of visualization was approached by Stojanovic using yield data (Stojanovic et al., 2017).
The history of contextual data could support decision-making on the farm. In this sense, the formalization of a context (Dey et al., 2001) applied to precision agriculture would allow the generation of context histories (Rosa et al., 2015) related to a whole plantation or areas of it. This data could be analyzed to generate context pre-dictions (da Rosa et al., 2016) of soil conditions. Therefore, the right moment for agricultural inputs distribution, such as pesticides and fertilizers, could be determined, in addition to uniformity in productivity.


Applications of GIS for Bridges Maintenance Service

AThe Bridge Maintenance Management System (BMMS) is an application system that uses existing data from a Bridge Management System database for monitoring and analysis of current bridges performance, as well as for estimating the current and future maintenance and rehabilitation needs of the bridges. In a transportation context, the maintenance management is described as a cost-effective process to operate, construct, and maintain physical money. This needs analytical tools to support the allocation of resources, materials, equipment, including personnel, and supplies. Therefore, Geographic Information System (GIS) can be considered as one tool to develop the road and bridge maintenance management system. This research aims to use GIS to create a database for bridge maintenance system. Two study areas have been chosen: the old construction bridge (Al-Qadisiyah) and the newly constructed bridge (Barboty). Both of them are in Al-Muthanna city\Iraq. The creation of database was based on field surveying data. In order to monitor the damages and decline of the bridges, four periodically observations have been achieved. The first analysis of the observations was conducted by graphical analysis which gave the results between all the observations. The second part of analysis included the process of documenting the cracks observed in both study areas via hyperlink tool. This tool can display each of the bridge cracks in the form of window that contains full information for each crack such as, photo of the crack, station number of the crack, coordinates of the crack, type of crack, the crack description, and risk levels of the crack. This information may help to save cost, time and effort to decision makers in the relevant institutions.

Introduction
Geographic Information System (GIS) is a system for plotting and investigating features on the Earth. It is a way to collect, store and manage any type of data with geospatial components. GIS data is typically stored in further than one layer. This is the basic aspect of GIS, working with layers of geographic information commonly known as data incorporation. This makes the GIS unique among the various data collection systems. Fundamentally, this system has the ability to overlap with the map and provide an information query facility that can ramblingly generate a whole new set of communication. At this point, data can be obtained in the form of tables, maps, graphs or a combination of these three. The other strong side of the GIS is its flexibility in modeling spatial objects to suit the particular needs of the user or application. The GIS offers a set of instruments or computer packages that allow the user to perform a specific operation on the map, with the help of a set of attribute data [1]. Therefore, GIS can be used to develop road and bridge maintenance management system. In a transportation context, the maintenance management is described as a cost-effective process to operate, construct, and maintain physical money. This needs analytical tools to support the allocation of resources, materials, equipment, including personnel, and supplies. Bridges are a capability consisting of the means and equipment that are necessary for the movement of passengers or goods, and it is a means of transporting staff, students and employers etc. Bridges are also considered to be the best way to reduce congestion in the city [2]. Damage to bridges or lower passageways can be classified as: damage to the concrete walls, damage to the lower parts of the concrete bridges, damage to the concrete by the rust of the reinforcing steel, damage to the chairs, damage to the concrete from iron rust, damage to collisions with high trucks, damage to expansion joints of bridges etc., and the most significant damage is the passage of heavy duty vehicles carrying more than the specified weight [3]. The causes of these damages are due to several reasons, including: • Absence of supervision and follow-up and weak penalties applied to violators.
• These bridges are located in the city centers or important transit areas, leading to the crossing of many vehicles on them. • Ease of granting licenses and may be followed by non-specialized sectors.
• Lack of GIS applications in road management facilities and the mechanism of distribution and licensing of restrooms and stations based on data, also geographical areas such as village areas, lengths of roads, places of paintings, sites of gas stations and other statistics such as the density of pedestrians, information restrooms and others. Geographic Information System (GIS) can be considered as a tool that can be used to develop bridge maintenance management for its ability to represent, process, and analyze all types of bricks on earth using spatial components. GIS data can be usually used to aggregate data in more of a single layer called "shapefile or feature class." These layers can also be combined with other layers of geographic information in a process called data integration. This makes unique GIS between different information gathering systems [4]. Following are some of previous researches which related to the topic of this research.
[5] Illustrated that the roads networks in India are enormous with in excess of 3.01 million kilometer of roads lengths with 34608 km of Nationalized Roads, 128622 km of State Roads and unofficial networks of 2737080 km, managed in immensely dissimilar social, economic, and climatic surroundings. The maintenance management of the enormous networks in the India was principally done at two types of levels: local and national level. The first level is problem particular and limited to a few municipal capital cities, while the second level in the India is comprehensive grounded and is done utilizing some macro level data such as area, uncultured national products etc. The main management in diverse phases of roads networks can be obtained to the absence of obtainability of huge capacity of data obligatory. At the time, no organization can use this information for the adequacy of the network in the regional context or to produce a large-scale scheme for the primary network such as highways and major roads. Therefore, the observable data obtained in a large number of locations in all possible formats is not useful, and the sources spent on collecting and maintaining such data drain the economy as a non-useful regular and do not achieve the desired aims. The main objectives of GIS use are data integration and map / display. The main aim of this research is to reveal the role of GIS technology in the enhancement of bridge maintenance management system components such as the output results through creating a comprehensive database (Geodatabase) for both study areas.
[1] Stated that the development and economic increase of a nation is closely associated to its accessible transportation system. Efficient road transport infrastructure capabilities will promote industrial and socio-economic development. To deliver security and comfort to road users, a comprehensive road maintenance agenda must be formulated and assumed to safeguard those roads are in good condition at all times. Preventive maintenance works, like road rehabilitation will assist to decrease the main road repairs and expenditure. Previous to this, a suitable database, collected through GIS will be necessitous. The GIS is one of the effective tools that can be used to manage database in road maintenance engineering. [6] Showed the relation among the Web-GIS and remote sensing as tools to bridge maintenance management system BMMS. GIS-Based TRIMS offer a mechanism for data integration, management, development, and production generation in its spatial environment. The objective of this study is to develop a GIS-Based roads transportation information maintenance management system for Adamawa Central. In order to realize the objective and aims of this study, Satellite image, roads transport maps, roads transportation brochures, picture of; bridges roundabout and their coordinates were obtained. The road transport infrastructures were mapped utilizing the satellite imageries. File geodatabase was generated in ArcCatalog. The appearance dataset was populated utilizing the road transport documents that served as an inventory of the dataset and the database was lastly generated. RTIMS was built by means of ArcGIS 10.1. Query analysis was transmitted out on the results. The paper disclosed that the federal road covered 7.94% length, state road 22.27% while the LGA road was the longest of (77.73%). It also discloses that 78% of the LG roads ware untarred while only 22% were tarred, the tarred roads were typically localized in the Yola north which is the state capital. It also there were twelve roundabouts in the study area and three of them were under construction. Also, there were six bridges in the area, with (Gerei LGA) having two with one under structure.

Study Areas
Two study areas have been chosen: Al-Qadisiyah bridge and Barboty bridge. The reason behind this choice is the importance of these bridges for the Samawa city, in addition they are affected by the heavy loads, precipitation in soil, rain water, and cracks.

Al-Qadisiyah Bridge
Al-Qadisiyah bridge is located in the southwest part of Samawa city. It is connecting the city center with Najaf city and it is passing over Al-Qadisiyah river which is one of Euphrates river branches. This bridge also leads to Salman town which is located near Saudi Arabia kingdom's borders. The bridge laying between (523522.786m E, 3458868.649m N) and (523677.602m E, 3458995.62m N), (Directorate of Roads and Bridges in Al-Muthanna, 2018). Figure 1 represents a satellite image of Al-Qadisiyah bridge.
The total length of the bridge is 200 meters, 50 meters on each approach. The width of the bridge is 12 meters. What is worth mentioning is that this bridge is connecting between cement plants in Al-Muthanna province. Therefore, most of vehicles passing through it are heavy loads trucks. These heavy trucks may cause cracks and geometric issues; thus, these issues will be investigated in this article.

Barboty Bridge
Barboty bridge is located in the western part of Samawa city. It's one of the important bridges which is constructed recently. It's laying between (524266.642m E, 3465978.518m N) and (524277.446m E, 3465658.394m N). The length of Barboty bridge is 154 meters long without its approaches, and 319 meters long with its approaches. The width of the bridge is 13.76 meters. It contributed to reduce the traffic jam which is usually happening in the city center. This bridge is the longest one on the Euphrates, it's connecting Al-Rumaithaa city and Al-Warkaa city together alongside with Samawa city's entrance from the northern side, Directorate of Roads and Bridges in Muthanna, 2018. Figure 2 shows a satellite image for Barboty bridge.

Data Collection
Data collection is an element of research in all fields of survey, counting social sciences and physical, business, and engineering as well. There are many methods which are depending on the tools or instruments (e.g. total station or level instrument) in which the data is collected. The focus on ensuring the accurate collection makes the results that are intended to reach it with high precision [7]. In this paper, the bridges have been divided into six layers in order to observe and compare different locations of points on the bodies of the bridges. These layers included: the side walk (s), fence (f), border (b), electricity (e), joints (j), and cracks (c). The meteorological process is repeated every 3 months, in 4 different annual seasons, to determine the variation in levels, the amount of settlement that occurs for each bridge and its causes and the possible solutions. There are two instruments have been used in this study: total station and level. The total station is Nikon-Nivo total station 5.C. 5". ± (3Mm + 2PPM), and the level is Topcon AT-B4 which is a new standard to measure the elevations.
Al-Qadisiyah bridge (Q) was observed using total station to determine (Easting and Northing) coordinates and the level to determine (Elevation). The first three layers were observed are border, side walk, and fence layer. After that, the electricity pole layer which containing 21 columns for every side. Every electric pole was observed at the base of it. The joint layer was the last layer which was observed by 3 observations (left, right, and center). Finally, the cracks layer was observed at the centre for documentation purposes. All these cracks have been identified with its numbers and discretion. Barboty bridge (B) was also observed using the same instruments to determine (Easting, Northing and Elevation). The first three layers were observed are border, fence, and side walk. After that, the electric poles layer which containing 13 columns. Every electric pole was observed at the base of it. The joints layer was the last layer which was observed through 3 observations (left, right, and center). Finally, the cracks layer was observed at the centre for documentation purposes. All these cracks have been identified with its numbers and descriptions. Table 1 shows the time period, date, and the temperature of the observations period.

Results of Graphical Analysis
A graphical analysis is a tool for science researches to collect, draws, and analyze the collected data. It can be defined as the analyses of data to determine the optimal output. One of the powerful tools utilized for data valuation are the graphs. Utilizing graphical techniques can represent equations or tests of statistics and mathematics more easily and clearly [8]. The output of this research was presented as graphics as will be explained in the following sections.

Results of Al-Qadisiyah Bridge
The graphical representation was applied to compare the differences between observations one, two, three, and four. The output of this analysis can be seen in figure 3 to figure 7.   According to the graphs shown in figure 3 to figure7, it is extremely clear that Al-Qadisiyah bridge suffers from very highly decline in the centre. This decline is about 19 mm in station (1+10). It can be also noticed that the least change value which is to be zero at the ends of the bridge. The concrete spaces of the bridge suffer from decline which is about 8 mm at the aspects of the bridge. The joint layer is the most important layer of the bridge because it is the only one capable of connecting the bridge spaces. The decline in this layer has been observed at the centre of the bridge which was found16 mm. This may be causing a strong defect in the structure of the bridge.

Results of Barboty Bridge
The differences between observations one, two, three, and four of Barboty Bridge have been also presented graphically. The output of this analysis can be seen in figure 8 to figure 12.       According to the graphs shown in figure 8 to figure 12, it is extremely clear that Barboty bridge suffers from very highly decline in the steel space which is equal to 16 mm at station (2+00) in the side walk layer. In the last observation, it reached to the highest change in the decline which is equal to 18 mm. However, this girder extends from station (1+70) to (2+24). The decline in the elevation at every concrete space is 2 to 3 mm in the second observation, and it doesn't exceed 6 mm in the third observation, while the last observation showed decline 8 mm. The decline in the elevation of the northern and southern approaches of the bridge was 1 to 2 mm. That's expected because it is linked to the soil and it is low elevation compared with the structure of the bridge.

Outputs of Hyperlink Tool for both Study Areas
Hyperlink tool is the item in a document or electronic document that leads to another section of the same document, or to a specific section of another document. It's mainly files location pointers that allow to the users to access related objects through specific actions (e.g. clicking on geographical features with a hyperlink). The hyperlink features of a GIS allow users to access numerous types of qualitative information related with particular geographical features in a GIS. Furthermore, numerous actions can be started though hyperlinks, such as a wep page can be began in the default web browser though a universal resource locator (URL) [9]. In this study, the hyperlink tool has been adopted as a process of documenting additional information in spatial layers that have saved in database. This included information about cracks located in the study areas such as cracks style, their images, and coordinates. Once clicking on the hyperlink tool and go to any certain crack, full information will appear such as the coordinates of the crack and the station number of it. Also, a photo shows the crack type and its description such as lowest, normal, and the most dangerous level of this crack. This process is very important for the bridges to management and maintenance. The following figures illustrate some of the results of cracks which observed for both study areas:  Figure 13. Shows the hyperlink output to the alligator crack of Al-Qadisiyah bridge. Hyperlink tool has been adopted to document the information of the cracks of the bridges for two study areas. By making the structure themes active, the hyperlink tool is utilized, the selected crack in green pops out an image of the discretion of the crack above by clicking it out. This analysis was achieved to display a physical nature of the crack for decision making in terms of maintenance and to see whether the crack is due for maintenance. And these are the pictorial representation of the bridge's cracks. Therefore, there is no requirement of physical site visitation or inspection. Most of the cracks that obtained at Al-Qadisiyah bridge are: alligator, transverse, longitudinal, potholes, rutting, block, and reflection cracking at joints. In contrast, the cracks that obtained at the Barboty bridge are: alligator, longitudinal, rutting, potholes, and reflection cracking at joints. The detailed information of each crack can be seen on the window of the hyperlink tool.

Conclusions
1. The graphical analysis showed that the maximum decline in joints layer of Al-Qadisiyah bridge at station (1+10) and equal to 16 mm which may cause a strong defect in the structure of the bridge. On the other hand, the maximum decline at the joints layer of Barboty bridge at station (2+49) and equal to 11mm. 2. The steel part of Barboty bridge has shown highly change (decline) in elevation which was equal 16 mm in side walk layer at station (2+00), therefore it is necessary to have more attention and consideration from directorate of roads and bridges in Al-Muthanna city. 3. Through the hyperlink outputs, most of the cracks that have been observed in Al-Qadisiyah bridge are Alligator cracking. In contrast, most of the cracks observed at Barboty bridge are Longitudinal cracking. The areas affected by these cracks must be maintained as soon as possible in order to not affect the surrounding areas. 4. The geodatabase which was created in this research can save time, cost, and efforts for the directorate of roads and bridges to identify the problems of bridges, which can take several days or months using traditional methods.


s2fieldsofstudy:
Computer Science
Engineering

text:
Monitoring software risks based on integrated AHP-ANN method

Software risk management refers to systematic process for analyzing and identifying the project risks. The present paper provides a hybrid method for IT software risks identification. Software projects possess different features which increase the project failure possibilities. Therefore, the present work integrate the Artificial Neural network with the Analytic Hierarchy Process (AHP-ANN) in order to solve the problem of software project estimation in early stage. The questionnaire developed to find out the risk functional model and provide the proposed method with proper data. The results observe a major common risk in software projects is the insufficient knowledge based on different software project life cycle stages. Also, there are some other important factors in software projects such as lack of good estimation in project scheduling, poor definition of project requirements which cause human errors.

Introduction
Risk management can be defined as a method to identify the software project threats in order to enhance the software firms organization. The risks sources can be the erroneous strategic in project management or the external challenges. Therefore, there is a need for operation enhancement of software project in order to develop the software efficiency and flexibility [1].
Most of studies investigated the risk factors and provided some useful techniques to specify the effectiveness of them. The ranking based on risks importance is made in light of analysis, planning, maintenance, design and implementation [2]. Classifying risk factors can be considered risk attributes as the main issue in developing risk project.
Development of risk management software can be classified into scheduling risks and quality risks. Also, it can be grouped into performance risks, cost risks support risks and schedule risks [1] [3]. These classifications were very helpful in monitoring and controlling risks in software projects. More importantly, the top ten software risk factors in developing software were chosen and utilized for analysis [4].
Some authors apply Artificial Neural Networks to identify the risks and to develop an application for risks management during software development [5]. Many other techniques have been used in this field such as regression analysis, expert systems, stochastic models, Monte Carlo Simulation, Decision Tree and Analytic Hierarchy Process AHP [6]. finally, theresearchers develop some techniques to tag the same goal such as Singular Value Decomposition SVD technique [7]. In this work, Artificial Neural Networks have been integrated with Analytic Hierarchy Process (AHP) method for risk control as a tool for risk management.

Software Risk identification
Software risk identification is considered the activity of the potential risks which can effect on the project development and determination. The risk check list can be creadt based on the identified risk concepts [5]. It occurs when the organization faces uncertainties from limited capacity and costs in its pursuit for opportunities. In this regard, an effective risk management initiative coupled with suitable risk management strategies can help mitigate the cost and stress brought on by risk issues [8]. Risk identification is a critical process, the risk management mostly depends on identifying all possible risks that may face the project during development [1]. The result of software risk identification is the risk factor list. The identification of risk factors will be followed by risk analysis. The quantitative risk analysis simulates each critical risk effect. Elzamly in 2014 brought forward new methods using quantitative and mining methods to conduct comparisons among risk management methods in the lifecycle of software development [4]. prior to the project implementation and the steps taken to mitigate them ensures higher rate for successful projects [5]. Hojjati and Noudehi in 2015 applied Monte Carlo simulation for risks assessment. The study evaluated project risks in the IT domain and utilized the Primavera Risk Analysis software to quantitatively analyze management [9]. Paraschivescu in 2016 brought forward integrated quality and risk management concepts resulting in an integrated management system risk that sheds light on new dimensions and perspectives. Also, Elzamly et al. in 2016 identify software risks and software development controls [10]. The study ranked the risk factors in software based on their importance and how often they occurred in a data source. The ANNs applicability was examined in Andreas"s study in an attempt to analyze survey data concerning risk management practices effectiveness in the context of product development (PD) projects and forecasting of project outcomes [7]. They explained the relationships between risk management factors that influence successful PD project (e.g., cost). Salman in 2018 apply the maintenance risk factors in Singular Value Decomposition (SVD) correlated with the traditional risk factor calculations to estimate the software maintenance projects [7]. Based on the present review, the researchers specifies the main risk factors that can be used in this work as in the next section.

Selection of Risk Factors
Risk management is a process to develop strategies for identifying and estimating their impact. The steps taken for risk management process in the present work are as follows; 1. Risk identification, it represents the activity of detecting the effected potential risk in the project that affected the project development. In the present work, the researcher developed a questionnaire using the taxonomy based risk identification presented by Marvin J. Carr [11]. 2. Risk analysis, it represents the process of understanding of where, when and why the risk appear. This process take place based on direct queries about the impact and probability of the risk elements. Traditional risk analysis focuses on the potential impacts to a human population due to the presence of an introduced substance or event, for example the presence of pesticides in a body of water used for human consumption, or an oil spill. A broad variety of techniques are used to evaluate risk in these situations. Risk analysis typically involves four steps: hazard identification, risk assessment, determining the significance of the risks, and risk communication. The traditional risk management in the present paper focuses on pure risk and refers to individual risks as if they don"t interact (Simona-Iulia, 2014). Based on the present two bases in the software project risk management, the researchers listed the risk factors. The "Top 10 software risk factors" lists differ to some extent from author to author, but some essential software risk factors that appear almost on any list can be distinguished. These factors need to be addressed and thereafter need to be controlled. Consequently, the list consists of the 10 most serious risks of a software project ranked from one to ten, each risk's status, and the plan for addressing each risk [10] [4]. However, the software risk factors listed in Table 1 below are considered in this study. In addition, these factors are the most common factors used by researchers and experts when studying the software risk factors in software development lifecycle.
Hoodat and Rashid classify the software risks and specify the relations between these risks. They used the risk tree structure correlated with the probabilistic calculation. The analysis helps qualitative and quantitative assessment of risk of failure. Also, its help software risk management process [2]. Therefore, this classification used in this research as a base of study which is correlated with the Software Development Life Cycle. Figure 1 shows the project scheme.

Risk Factors Evaluation
The study developed a questionnaire that comprised of questions relating to chosen 34 risks maintenance risk factors adopted from Lopez and Salmeron (2012). The questions were chosen with the hope of the works of Marvin (1993) and Webster (2006) and the risk factor values calculated two types of questions, positive and negative [11] , [13] , [6]. The former type represents the questions that had yes answers, while the latter type represented those that had no answers. The sum of the questionnaire list of questions for every type of risk can be represented by the following formula [5] , [7] (1) Where RF = Risk Factor Value, Q =value of each question, W = weight Thus, the boundary condition is represented as:

{
(2) Collection of data was conducted using questionnaire to determine the commonly occurring risks in majority of software projects in the software companies. The respondents were then presented with the 20 software risk factors. The study sample comprising of 150 persons worked in specific IT organizations in Iraq. These peoples represent the Software Life Cycle user areas. The collected data reflect the selected software risk factors which developed to be used in ANN.

Methodology of Risk Factor Specification
In order to specify the risk factors in software project life cycle, the researcher integrate the AHP technique with ANN as in the following:

a) Artificial Neural Networks (ANNs)
The neural network can be defined as a parallel distributed processor. The main processing unit is inspired by the way of biological nervous system, such as the process information of human brain. The potential system of ANN involves several layers developed by computing elements and called nodes. The system operation of neural network depends on the signal transmission. When the nods receives the input signal from input representation of the system, it will transfer the signal to the next step node. This process will mining the transferred the data in order to find out the specific correlation in input data. The first layer represents the input layer and the last layer considered the output layer. The input layer is received the data of the case study which represent the statistical data. The last layer produce the solution of the problem which represent the predicted or identified data. In between, there are hidden layers which operate the complex data to identify proper pattern using system of specific formulas. The reason for using the neural network ire as in follows: 1. It must have the ability to learn the neural system how to do tasks. The tasks done based on the given tanning data.
2. It must have the ability to generalize the internal system operation. It must produces reasonable outputs without paying attention how deal with the internal processes.

b) AHP
Analytical Hierarchy Process (AHP) methodology has been applied to the evaluation of risk related to software project. Five risks are evaluated and defined in each project stage as presented in table 1. The criteria weights can be more precisely defined by the AHP methodology using "Saaty scale" than using the digital logic method. However, subjectivity is playing a great role in both of methods. Subjectivity is included to the comparison of alternatives by the original AHP methodology, also.
Contrary, by using other method there is no subjectivity concerned of alternatives comparisons because of dealing with transformed values of criteria. The ranking of all alternatives can be performed, by obtaining the priorities. The weights present the relative importance of each criterion compared to the goal. Finally, alternatives present the group of feasible solutions of the decision problem.

Experimental results
The methodology of the present paper is to integrate the ANN with the AHP technique. The AHP will present a pattern to the ANN. Based on the results, the software project risks were important in the perspective of the project managers, whereas all controls are used most of the time, and often. The risks were ranked on importance in light of analysis, planning, design and implementation. In particular, top of software risk factors in software development Lifecycle were very important, aggregating the responses resulted in the following ranking of the importance of the listed risks. The AHP model in this study is formed to prioritize the various risks within the software project. The result observe the factor priority, for instance the software requirement results can be seen in table 2 and 3.   The plotted results shown in Figure 3 indicate that results observe the same responses of the risk factor effects.
The yellow bars represent the ANN results, while the blue bars represent the integrated method. The bar graph shows that the risk identification due to the present method. These valid results highlight the largest problem on IT software risk factors which represents inadequate knowledge/skills, insufficient expertise and Insufficient/inappropriate staffing. The results observe a big effect by the insufficient expertise in the applied software management as shown in figure   3.

Conclusion
In this work we apply AHP technique with ANN to support Risk Management.
The results show that that AHP technique is simple and efficient for total variance in common questionnaire of each of the software risk factors to model if they are effective in mitigating the occurrence of each risk factor. The result of AHP is presented as a pattern to ANN. As a conclusion, this method can be used effectively to identify the risk effect in all project phases. The used method specify the Risks in three reasons. It specifies the root of risk problem and the effective phase of project.


Improvement of Chatbots Semantics Using Wit.ai and Word Sequence Kernel: Education Chatbot as a Case Study

Designing interactive question-response systems has become an important challenge in Artificial Intelligence which aims to build a computer program, referred to as Chatbot, able to manage an online humancomputer conversation with natural language. The number of chatbots continues to increase these recent years using different languages, tools and platforms and has been used in several domains, such as marketing, education and medicine. One of the most important issues in designing chatbots is its degree of interoperability with human natural language. In this context, we propose in this work a new messenger chatbot design approach based on Wit.ai and Word Sequence kernel in order to improve semantics. Wit.ai is used to detect contexts and concepts while the Word Sequence Kernel is used as a similarity measure between textual conversations taking into account the order of appearance of words in the conversation. A testing educate chatbot has been build which aims to provide FAQBot system for university students and acts as undergraduate advisor in student information desk. The performance of the proposed chatbot was compared to conventional messenger chatbots and showed better results.

I. INTRODUCTION
Virtual assistants are helping customers with day-today tasks like maintaining to-do lists, setting reminders and getting daily news updates among a panoply of required tasks. Digital assistants like Alexa, Google Home and Echo are increasingly becoming popular with consumers. Seasoned brand marketers have started taking notice of the power of conversations to drive marketing activities even though it is just the early days of messenger marketing. According to Facebook, 20 million out of the 70 million brands/pages are actively responding to consumer messages [15].
This trend is not surprising as popular sources suggest that consumers love messaging and prefer interacting with brands via popular social media applications. Customers report the on-demand access to brands whenever and wherever provides a much needed and preferred convenience. It's estimated that nearly twothirds of consumers expect to increase their messaging applications usage in the next two years. Seeking to enhance their two-way conversational capabilities, brand marketers recognize the immense built-in audience. The most glaring shortcoming of today's chatbots is that do not really understand what you are querying. They will often misinterpret typed messages and then give answers which do not much with user's queries. We propose to design and develop a "chatbot" to respond to search queries for products using Natural Language Processing techniques where semantics are improved. The proposed chatbot improves semantics by looking for entities in the user query using "wit.ai" then compares the similarity of the query of the customer to the other queries in the database and returns the response assigned to the most similar question. We propose a new method to compute the similarities of textual messages based on the word sequence kernel.
The rest of this chapter is organized as follows: Section II describes structures of chatbots and gives examples of existing works implementing chatbots. After that Section III gives steps and techniques for preparing unstructured text to numerical analysis while Section IV gives our proposed chatbot design approach which includes Wit.ai and word sequence kernel to improve semantics. The evaluation of the performance of the proposed chatbot in the field of education is then described in Section V. Finally, Section VI gives conclusion and future works.

II. CHATBOTS : DESIGNING METHODOLOGIES AND HISTORY
It is necessary to build a dialogue system or computer program called a Chatbot [11] in order to be able to answer phrases or keywords that have been taken from textual conversations. A chatbot is able to manage interactions between humans and computers because it can both ask and respond to questions, therefore having the ability to influence and understand user behavior. Natural language text is input into the chatbot so that it is able to provide both questions and answers that are the most appropriate response to what has been asked. This process is repeated throughout a conversation, as illustrated in 1. In the subsequent sections, design methodologies and history of chatbots will be explored.

A. Designing methodologies
Designing a chatbot requires highly developed programming skills and experienced developers to achieve even a basic level of realism. There are several platforms and languages that can be used to build a chatbot. Usually, the Artificial Intelligence Markup Language (AIML), a derivative of XML, is used as a standardized language to build chatbots. This language was first used in the project A.L.I.C.E. (the Artificial Linguistic Internet Computer Entity) [16] and is able to use AIML to characterize and define various types of data objects as well as being able to describe the partial conductance of the programs it performs. AIML is effectively centered on the units of "categories" and "topics", with its aim being to make conversational modelling a simplified process, particularly with regard to the system-response process. In AIML, the data object class is defined as an AIML object, with the responsibility of these objects being to model conversational patterns. AIML objects have a standard structure, which is described as follows: (1) There exist several AIML tags, the most frequent are "category", "pattern" and "template". The "category" tag defines the knowledge unit of the conversation and the "pattern" tag is able to identify input by the user, while the "template" tag allows the chatbot to respond to particular input from the user. These tags facilitate the design of AIML chatbots, which are able to make intelligent responses to textual natural language conversations [12].
Although the complexity of designed language of chatbots, their performance is usually related to the knowledge data base and the manner the program choose the right answer. The data base must be very large and must contain answers to all predicted queries. Also, the selection method of the right answer among thousands or millions of answers must be optimized in order to choose the response that best answers the user's query. Many research works have focused on improving recognition rates of textual conversations and the technology is now approaching reality for conversation based human computer interactions. The shown improvement of chatbot is essentially related to the improvement of NLP and learning techniques. Learning phase in chatbot means saving new questions and then using them later to give appropriate answers for similar questions [2].
Chatbots keywords-pattern matching techniques can be divided into two categories. First is rather similar to human brain incremental parsing technique (Pickering et al., 2000) where an input sentence is being analyzed in a word-by-word basis from left to right by sequence. Keywords can be one-word keywords or many-words keywords but each word in many-words keywords must be attached to one another, forming a long keywords pattern (cannot be separated as e.g. one word in prefix and one word in suffix separated by several words in the middle).

B. Chatbots history
When considering textual messages, a specific toolkit allows the text to be organized into sentences and then split into words in order to consider and provide the most suitable response. The chatbot can be "trained" using NLP to allow it to be familiarized with the interactions it will go through. The more examples the chatbot is given, the better it will be able to provide users with the right answers and responses to their questions.
The idea of chatbots first appeared in the 1950, but it was to be more than 50 years before the world was ready to see them implemented into real life. This was facilitated by rapid progress in artificial intelligence (AI) and natural language processing (NLP) as well as the existence of text messaging applications on a global scale. A chatbot is also known by other terms, including talkbot, interactive agent, chatterbot, Bot, IM bot and Artificial Conversational Entity. Essentially, it is a computer program that conducts a conversation in either speech or written natural language and is able to both understand the user intent and then respond in a way that is appropriate to the particular organization that it has been designed for. Due to the rise of communication technology in recent years, text communication has become a socially acceptable way to interact with others and as a result, many people now prefer to chat in this way rather than making personal telephone calls or other direct contact. Turing was the first to ask the question Can the machine think? [12] where thinking is a human ability of deciding the right answer for a question. The imitation game was suggested by Turing as a method to directly avoid the question and to specify a measurement evaluating the degree of similarity between the machine and a human [15]. Chatbots continue to grow in popularity with 80% of businesses expecting to be using one by 2020 [15]. There are now chatbots for multiple uses such as Customer Service (Twyla), time-sheets (Nika), Expenses (Acebot) and feedback (Wizu). We give in the following a brief history of intelligent machines which were able to make conversation with humans.

A. Preparing texts for numerical analysis: VSM and n-Grams
Text is normally considered to be unstructured data, for which it is impossible to conduct numerical analysis. Therefore, a Vector Space Model (VSM) representation is used to prepare the text to be numerically analyzed or calculated and in this VSM model, the text j is represented by a vector of terms: where T is the whole set of words T = (t 1 , ..., t |T | ), T is the size of the vocabulary and w kj represents the weight of the term w k in the text t j . When the vectors are close to each other, the text is considered to be similar in content and the VSM representation assumes that the relative position of words is of minimal importance. However, this may lead to a loss in terms of the correlation of adjacent words and information regarding the positioning of words is lost.
Meanwhile, the "n-Grams" representation of text [18], is an independent language text representation technique that attempts to solve the problem of the loss of information about the positions of words by considering each textual input as a sequence of n characters, syllables or words. Using an extraction of all the possible ordered subsequences of consecutive n characters, a set of "n-Grams" is built. The similarities between the text that has been input are then measured, according to the number of contiguous and non-contiguous sub-sequences that are shared between them. However, this representation also presents issues as a high number of dimensions is needed to represent each input text.
Several embedding techniques have been considered in order to tackle the issues of high dimensionality. One example of this is the application of Random Projections (RP) [7] to text representation as it reduces the dimensionality of the term space. It does this by projecting it to a lower-dimensional subspace, which is formed by a set of random vectors. Latent Semantic Analysis (LSA) [8] has also been used in text representation [14] and uses Singular Value Decomposition (SVD) to project the original, high dimensional vector space to a lowdimensional concept vector space. In this new, lowdimensional concept vector space, the dimensions are statistically uncorrelated, or orthogonal.

B. NLP and chatbots
The integration of NLP into chatbots allows a more human element to be added. When a chatbot is built for public use, it is common for users to ask it some difficult questions, as human nature tends to dictate that they will try to beat the bot or stump it with difficult questions. The developers can try to solve this issue or at least mitigate it by adding some default answers, although it is almost impossible to provide a suitable answer to everything or predict what may be asked because there are so many variables [12].
The chat client can include natural language processing so that users can be helped to perform data queries or access content. The chatbot should contain a set of sample phrases that have been mapped with a specific intention in mind in order to determine intents and entities. An administrator who has been specifically trained to manage this process will provide sample phrases for each of the intents. In addition, to identify existing entities in the user queries, natural language processing can be further exploited and machine-learning techniques can facilitate the establishment of similar phrases for each intent. The administrator can also review any new phrases that users provide and identify the correct entities and intents so that the platform's performance can be further developed and enhanced [12]. Given that not all interactions can be predicted, several strategies need to be implemented in order to build chatbots able to give an answer and, thus, simulate intelligence. A first possible solution is the association of a prior personality to a chatbot allowing to justify some answers to be considered inappropriate. A second solution is to direct the conversation, by asking questions inciting the user participation. This fact will keep the conversation with little contribution from the program.

IV. PROPOSED APPROACH
We propose to improve the performance of chatbots using semantics and the word sequence kernel as a similarity measure. The proposed approach can be described by the following stages as illustrated in Fig.2 : the first stage is devoted to pre-processing and preparing data, the second to improve semantics and the final one is devoted to the similarity evaluation and the selection of the best response. The main algorithm of the proposed method is described in algorithm 1.

A. Phase 1: text preprocessing and filtering
Text preprocessing is usually considered as an important process in text mining applications. This process is used to transform input textual queries to meaningful information for the next step of processing. We present in the following the different text preprocessing techniques which will be used to prepare users textual queries for the next proposed phase: • Tokenization: which consists in dividing text into linguistic units, words, phrases, terms and symbols named tokens. In this process, we use the separator space to break text into k words. • Stop words elimination : Given the k obtained words after tokenization, we remove common terms which appears frequently in the text and do not help to find the right meanings. This process decreases the size of data and makes the system more efficient. We propose to combine three stop words lists, the first is "SMART" [9] 1 built by Gerard Salton and Chris Buckley for the experimental SMART information retrieval system at Cornell University which contains 571 English words, the second list is "SNOWBALL" 2 containing 175 stop words for multiple languages. Most of these have been ported from the Quanteda stop word list. The last list is "ONIX" 3 which contains 429 words. As a result, our stop words list contains 1175 multi-languages stop words.

B. Phase 2: Semantic improvement
To improve semantics, we propose to use a new NLP platform, referred to as "Wit.ai" adopted by Facebook since January 2015 and is behind the NLP capabilities of Facebook Messenger platform. This platform has rolled out major updates. One of the best capabilities of Wit.ai is the sophisticated tool that can be used to train the platform in new conversation models as well as monitoring the interactions between users and the platform. Wit.ai allows using entities, intents, contexts, and actions, and it incorporates natural language processing (NLP) techniques. It supports about 50 free languages and offers for developers an easy interface to build applications and to connect devices in or-der to change textual or voice messages.
Wit.ai contains two major components: Understanding and Inbox. The Understanding is the page where you can teach your bot sentences and words using intents and entities while the Inbox component contain the user's messages sent to the chat bot. The bot works with confidence. If the value of confidence is less than a threshold, it sends the messages to the inbox. The developer should look at the phrases and validate them or gives answers. Our goal of using wit.ai is to find entity from users question. For example, to ask for the temperature a user can formulate his question with different syntax terms: • What's the current temperature? • Is it me or it is hot here? • could you give me the current temperature please? • temperature please ...
Using wit.ai we can detect that all these questions are about the entity "Temperature". We note also that one question can contain several entities.

C. Phase 3: Similarity calculation and selection of the best answer
In order to improve semantics when calculating textual similarities of users queries and existing questions in the data base, we propose to use the word sequence kernel which keeps order of words as they appear in the message. Many kernels known as String Kernel are proposed in the literature to solve the problem of high dimensional features in n-Grams representation of Text. The n-Grams are not computed for each input, but only dot products between n-Grams of each pair of inputs are computed. An example of these kernels is WSK (Word Sequence Kernel) [3]. The advantage of using words as atomic unit is to keep information regarding words positions in order to maintain linguistic meaning of terms. For example, the terms son-in-law have a special meaning that can be lost if they are broken. The WSK has also the advantage of reducing the number of features per text because it uses sequences of words rather than sequences of characters. The time complexity of computing WSK similarity between two messages m 1 and m 2 is evaluated to O(n m 1 m 2 ) [3] where n is the length of the used subsequence and m i is the number of words in message m i .
Let Σ the alphabet consists of the set of words that exist in the text. Let S = m 1 m 2 m 3 ...m |S| a sequence of words where S is the length of S. Let u = s[i] a subsequence in S with s[i] = t i 1 ...t i j ...t i n where t i 1 and t i j in this subsequence are not necessarily contiguous in S and n is the length of the subsequence u. The feature mapping φ for the sentences S in the feature space is provided by defining φ u for each u Σ n as: where the decay factor λ is used to penalize noncontiguous sub-sequences and l(i) is the length of subsequence s[i] in S and defined by : These features are used to calculate the number of times that sub-sequence u in the sentences S occurs and these are then weighted according to their length. Therefore, when there are two strings S 1 and S 2 , the feature vectors' inner product is gathered by the sum being computed across all the common sub-sequences.
V. EXPERIMENTS AND EVALUATIONS RESULTS: EDUCATION CHABOT We choose education chatbots as a challenging domain to evaluate the performance of the proposed chatbot's designing approach. Recently, messenger chatbots has been designed for revolutionary universities such as Georgia State University, the University of Memphis, West Texas and Arizona State University. Education chatbot aims to improve communication, minimize ambiguity from interactions and increase productivity by facilitating information access and retrieval. Students can ask questions about general topics such as financial aid or exam issues more specific to the college they are inquiring about. We present in the following two challenging use cases demonstrating the motivation of education chatbots to create a better cam-pus experience to students: • Given a new student having no idea about scholarships that can potentially reduce cost of his course. He can visit the university website and looks for details when he finds time. This may take several minutes since the student is not familiar with the university website and may not find response to his query. Education chatbot can accelerate and facilitate this process by simply typing the question "Where can I find information on your scholarships?" and the chatbot gives an instantly answer Here's a link to all our current scholarships.
In addition, the chatbot can go further by making a conversation with the student in order to find only scholarships that can be applied to his situation. The chatbot can ask for example "Which department are you in? (Please select one.) " or which "Which course are you studying? (Please select one.)" • Given a student that has missed an exam and would like to know what he must to do ? is there any possibility that he pass the exam later ? This is an extremely use case where the student may not find the response in the website and time looking for the response is also important since he can have the right to do the exam only after a short period. The education chatbot may facilitate the action for this student by simply asking "What do I do if I miss an exam?" and the chatbot gives him options or responses like You can apply for a re-examination within 2 days from the date of the missed exam".
We Recent Photo and all copies of diploma " > 2. <"How apply certificate Attendance", "To request a certificate of attendance you must go to the university website in this link:'... ' and click on the certificate section and then fill in your details"> 3. ...

etc.
An example of conversations between our educate messenger chatbot and a test user is shown in Fig. 3. We show the ability of the proposed chatbot to give answers for students which are querying about missed exams and steps to obtain a certificate of attendance for an event organized in the university. This may help students to rapidly find responses to their questions without losing time to search this information in the website or contacting professors and university administrators.
In order to objectively evaluate the proposed chatbot to give correct answers for students, we prepared 100 users queries related to 50 questions in the simulated dataset. For each query, we evaluate if the response of the chatboat corresponds to the expected answer. We calculate the percent-age of correctly identified responses by dividing the number of correctly identified responses by the total number of queries. We compare the obtained results of the proposed chatboat with and without using Wit.ai to those obtained using Jaccard and Cosine textual similarities. Table 1 reports the percentage of correctly identified responses for each evaluated method. We show that the proposed approach gives better results than those obtained with Jaccard and Cosine. We obtained 89% of correctly answers from a set of 100 queries on 50 questions for the proposed chatbot while only 60% using Jaccard textual similarity. Another interesting result reported in Table 1 is the important improvement of quality when using Wit.ai which increases quality of correct answers about 24%.  In order to better demonstrate the high semantic improvement of Wit.ai compared to classic messenger chat-bots, we prepared 50 questions related to only 10 responses. We formulated these questions in such a way that the same question is asked using different syntactic terms. Table 2 reports obtained scores. We show the high improvement of correctly identified responses equally to 50% compared to Jaccard and Cosine.
From the scores given by our evaluations the following inferences could be taken: • The proposed building approach of the chatbot outperforms those designed based on standard textual matching such as Jaccard and Cosine. We proposed in this work a new approach of building Facebook messenger chatbots integrating Wit.ai and word sequence kernel to improve semantics. Wit.ai is used to efficiently detect contexts and concepts while the Word Sequence Kernel is used as a similarity measure between textual conversations taking into account the order of appearance of words in the conversation. A testing educate chat-bot has been build which aims to provide FAQBot system for university students and acts as undergraduate advisor in student information desk. The performance of the proposed chatbot were compared to conventional messenger chatbots and showed better results.
As future works, it would be interesting to evaluate the proposed approach in other interesting domains in order to confirm its showed performance in the education domain. We also plan to improve the computational complexity when looking for the best answer. For example, we can generalize the context of conversations by clustering similar topics and grouping similar answers in order to reduce the search space when looking for the best response.


Pun-GAN: Generative Adversarial Network for Pun Generation

In this paper, we focus on the task of generating a pun sentence given a pair of word senses. A major challenge for pun generation is the lack of large-scale pun corpus to guide supervised learning. To remedy this, we propose an adversarial generative network for pun generation (Pun-GAN). It consists of a generator to produce pun sentences, and a discriminator to distinguish between the generated pun sentences and the real sentences with specific word senses. The output of the discriminator is then used as a reward to train the generator via reinforcement learning, encouraging it to produce pun sentences which can support two word senses simultaneously. Experiments show that the proposed Pun-GAN can generate sentences that are more ambiguous and diverse in both automatic and human evaluation.

Introduction
Generating creative and interesting text is a key step towards building an intelligent natural language generation system. A pun is a clever and amusing use of a word with two meanings (word senses), or of words with the same sound but different meanings (Miller and Gurevych, 2015). In this paper, we focus on the former type of pun, i.e., homographic pun. For example, "I used to be a banker but I lost interest" is a pun sentence because the pun word "interest" can be interpreted as either curiosity or profits.
An intractable problem for pun generation is the lack of a large-scale pun corpus in which each pun sentence is labeled with two word senses. Early * Equal Contribution. 1 The code is available at: https://github.com/ lishunyao97/Pun-GAN. researches (Hong and Ong, 2009;Valitutti et al., 2013;Petrovic and Matthews, 2013) are mainly based on templates and rules, thus lacking creativity and flexibility.  is the first endeavor to apply neural network to this task, which adopts a constrained neural language model (Mou et al., 2015) to guarantee that a pre-given word sense to appear in the generated sequence. However,  only integrates the generation probabilities of two word senses during the inference decoding process, without detecting whether the generated sentences can support the two senses indeed during training. Promisingly, Word Sense Disambiguate (WSD) (Pal and Saha, 2015) which aims at identifying the correct meaning of the word in a sentence via a multi-class classifier, can help the detection of pun sentences to some extent. Based on the above motivations, we introduce Generative Adversarial Net (Goodfellow et al., 2014) into pun generation task. Specifically, the generator can be any model that is able to generate a pun sentence containing a given word with two specific senses. The discriminator is a word sense classifier to classify the real sentence to its correct word sense label and classify a generated pun sentence to a fake label. With such a framework, the discriminator can provide a well-designed ambiguity reward to the generator, thus encouraging the ambiguity of the generated sentence via reinforcement learning (RL) to achieve the goal of punning, without using any pun corpus.
Evaluation of the pun generation is also challenging. We conduct both automatic and human evaluations. The results show that the proposed Pun-GAN can generate a higher quality of pun sentence, especially in ambiguity and diversity.

Model
The sketch of the proposed Pun-GAN is depicted in Figure 1. It consists of a pun generator G θ and a word sense discriminator D φ . The following sections will elaborate on the architecture of Pun-GAN and its training algorithm.

Generator
Given two senses (s 1 , s 2 ) of a target word w, the generator G θ aims to output a sentence x which not only contains the target word w but also express the two corresponding meanings. Considering the simplicity of the model and the ease of training, we adopt the neural constrained language model of  as the generator. Due to space constraints, we strongly recommend that readers refer to the original paper for details. Compared with traditional neural language model, the main difference is that the generated words at each timestep should have the maximum sum of two probabilities which are calculated with s 1 and s 2 as input, respectively. Formally, the generation probability over the entire vocabulary at t-th timestep is calculated as is the hidden state of t-th step when taking s 1 (s 2 ) as input, f is the softmax function, and x <t is the preceding t − 1 words. Therefore, the generation probability of the whole sentence x is formulated as To give a warm start to the generator, we pretrain it using the same general training corpus in the original paper.

Discriminator
The discriminator is extended from the word sense disambiguation models (Kågebäck and Salomonsson, 2016;Luo et al., 2018a,b). Assuming the pun word w in sentence x has k word senses, we add a new "generated" class. Then, the discriminator is designed to produce a probability distribution over k + 1 classes, which is computed as where c is the context vector from a bi-directional LSTM when taking x as input, U w is a wordspecific parameter and y is the target label. Therefore, D φ y = i|x, i ∈ {1, ..., k} denotes the probability that it belongs to the real i-th word sense, while D φ (y = k + 1|x) denotes the probability that it is produced by a pun generator.

Training
We follow the training techniques of Salimans et al. (2016) which applys GAN to semisupervised learning. For real sentence x, if it is sense labeled, D φ should classify x to its correct word sense label y, otherwise D φ should classify x to anyone of the k labels. For generated sentence x, D φ should classify x to the (k + 1)-th generated label. Thus, the training objective of the discriminator is to minimize: where p data denotes the sentence which only supports one word sense.
To encourage the generator to produce pun text, the discriminator is required to assign a higher reward to the ambiguous pun text which can be interpreted as two meanings simultaneously. For pun sentence, the probability of the target two sense D φ (s 1 |x) and D φ (s 2 |x) should not only have a small gap, but also account for the most. For example, (0.1, 0.5, 0.4) and (0.1, 0.8, 0.1) are two probability distributions outputted from D φ . The former is more likely to be a pun with the second (0.5) and third (0.4) meaning, while the latter is mostly a generic single sense sentence with the second meaning (0.8). Based on the above observations, the reward is designed as where 1 is a coefficient that avoids the denominator being zero. Then, the goal of generator training is to minimize the negative expected reward.
where x (k) is the k-th sampled sequence, r (k) is the reward of x (k) . By means of policy gradient method (Williams, 1992), for each pair of senses (s 1 , s 2 ), the expected gradient of Eq. 6 can be approximated as: where K is the sample size. Similar to other GANs (Salimans et al., 2016;Yu et al., 2017), the generator and discriminator are trained alternatively.

Dataset
Training Dataset: To keep in line with previous work , we use a generic corpus -English Wikipedia to train Pun-GAN. For generator, we first tag each word in the English Wikipedia corpus with one word sense using an unsupervised WSD tool 2 . Then we use the 2,595K tagged corpus to pre-train our generator. For discriminator, we use several types of data for training: 1) SemCor (Luo et al., 2018a,b) which is a manually annotated corpus for WSD, consisting of 226K sense annotations 3 (first part in Eq.4); 2) Wikipedia corpus as unlabeled corpus (second part in Eq.4); 3) Generated puns (third part in Eq.4).
Evaluation Dataset: We use the pun dataset from SemEval 2017 task7 (Miller et al., 2017) for evaluation. The dataset consists of 1274 humanwritten puns where target pun words are annotated with two word senses. During testing, we extract the word sense pair as the input of our model.

Experimental Setting
The generator is the same as . The discriminator is a single-layer bi-directional LSTM with hidden size 128. We randomly initialize word embeddings with the dimension size of 300. The sample size K is set as 32. Batch size is 32 and learning rate is 0.001. The optimization algorithm is SGD. Before adversarial training, we pre-train the generator for 5 epochs and pre-train the discriminator for 4 epochs. In adversarial training, the generator is trained every 1 step and the discriminator is trained every 5 steps.  (Mou et al., 2015) 2.0 2.1 2.0 CLM+JD  3.4 3.6 3.5 Pun-GAN 3.9 3.7 3.8

Baselines
We compare with the following systems: LM (Mikolov et al., 2010): It is a normal recurrent neural language model which takes the target pun word as input.
CLM (Mou et al., 2015): It is a constrained language model which guarantees that a pre-given word will appear in the generated sequence.
CLM+JD : It is a state-of-theart model for pun generation which extends a constrained language model by jointly decoding conditioned on two word senses.

Evaluation Metrics
Automatic evaluation: We use two metrics to automatically evaluate the creativeness of the generated puns in terms of unusualness and diversity. Following Pauls and Klein (2012) and He et al. (2019) 4 , the unusualness is measured by subtracting the log-probability of training sentences from the log-probability of generated pun sentences. Following , the diversity is measured by the ratio of distinct unigrams (Dist-1) and bigrams (Dist-2) in generated sentences.
Human evaluation: Three annotators score the randomly sampled 100 outputs of different systems from 1 to 5 in terms of three criteria. Ambiguity evaluates how likely the sentence is a pun. Fluency measures whether the sentence is fluent. Overall is a comprehensive metric.    Table 1 and Table 2 show the results of automatic evaluation and human evaluation, respectively. We find that: 1) Pun-GAN achieves the best ambiguity score. This is in line with our expectations that adversarial training can better achieve the aim of punning; 2) Compared with CLM+JD which is actually the same as our pre-trained generator, Pun-GAN has a large improvement in unusualness. We assume that it is because the discriminator can promote to generate more creative and unexpected sentences to some extent via adversarial training; 3) Pun-GAN can generate more diverse sentence with different tokens and words. This phenomenon accords with previous work of GANs (Wang and Wan, 2018).

Results
In addition, Table 3 shows the A/B tests between two the models. It shows that Pun-GAN can generate more vivid pun sentences compared with the previous best model CLM+JD. However, there still exists a big gap between generated puns and human-written puns. To conclude, both automatic evaluation and human evaluation show the effectiveness of the proposed Pun-GAN, especially in ambiguity and diversity.

Ablation Study
In order to validate the effectiveness of adversarial learning, we fix the discriminator after pretraining. Table 4 shows the results, from which we can conclude that adversarial leaning can help improve the creativeness of generated puns. Pun-GAN It is a touch with the red sox.

Human
The massage which came with the spa treatment was a nice touch. state s1: an organized political community forming part of a country.
s2: mode or condition of being.

CLM+JD
According to the state, he was the first time in the united states.
Pun-GAN In the state, the national assembly was established.

Human
Many people need to learn to be happy with the state they are in. GAN. Human-written puns are also given. It demonstrates that, compared with CLM+JD, Pun-GAN can generate puns which are closer to the funniness and creativeness of human-written puns. However, both CLM+JD and Pun-GAN may sometimes generate short sentences. Since too short sentences lack sufficient context, they always tend to ambiguous. More analysis can be found in Section 3.8.

Error Analysis
We carefully analyze the generated results of Pun-GAN with low overall scores in human evaluation. Fig 3 shows the proportion of different error types. The most common type of error is generating a sentence which only supports a single word sense. This accords with our expectations since generating a sentence which can support two word senses without any labeled corpus is very hard. Another common type of error is generating over generic sentences. For example, "It is a bank". In most instances, these generic sentences are always very short and they begin with a pronoun like "It is" or "He can". The reasons are two-fold. One is that these type of sentences can get a high generation probability since the generator is actually a language model. The other is these type of sentences can even get a not bad reward since they are indeed ambiguous. Moreover, grammar error also accounts for about 1/5. We hypothesize that it is caused by the joint generation process in Eq.2.

Conclusion and Future Work
In this paper, we propose Pun-GAN: a generative adversarial network for pun generation. It consists of a pun generator and a word sense discriminator, which unifies the task of pun generation and word sense disambiguation. Even though Pun-GAN does not require any pun corpus, it can still enhance the ambiguity of sentence produced by the generator via the reward from the discriminator to achieve the goal of punning. Pun-GAN is generic and flexible, and may be extended to other constrained text generation tasks in future work.