Exploring the self-service model to visualize the results of the ATLAS Machine Learning analysis jobs in BigPanDA with Openshift OKD3

A large scientific computing infrastructure must offer versatility to host any kind of experiment that can lead to innovative ideas. The ATLAS experiment offers wide access possibilities to perform intelligent algorithms and analyze the massive amount of data produced in the Large Hadron Collider at CERN. The BigPanDA monitoring is a component of the PanDA (Production ANd Distributed Analysis) system, and its main role is to monitor the entire lifecycle of a job/task running in the ATLAS Distributed Computing infrastructure. Because many scientific experiments now rely upon Machine Learning algorithms, the BigPanDA community desires to expand the platform’s capabilities and fill the gap between Machine Learning processing and data visualization. In this regard, BigPanDA partially adopts the cloud-native paradigm and entrusts the data presentation to MLFlow services running on Openshift OKD. Thus, BigPanDA interacts with the OKD API and instructs the containers orchestrator how to locate and expose the results of the Machine Learning analysis. The proposed architecture also introduces various DevOps-specific patterns, including continuous integration for MLFlow middleware configuration and continuous deployment pipelines that implement rolling upgrades. The Machine Learning data visualization services operate on demand and run for a limited time, thus optimizing the resource consumption.

Introduction
Cloud computing is perhaps one of the most important game changers in the IT business industry in the last 10 years. With the advent of container engines and complementary cloud orchestrators, the entire industry moved forward to a new era -an era of portability. Therefore, the software can now be bundled together with dependencies and runtime and run on various heterogeneous systems. In addition, aspects like scalability, security or high availability can be outsourced to a cloud orchestrator and fully provided by this software solution.
Unquestionably, Kubernetes remains a relevant technology when talking about modern cloud computing. It is a highly customizable orchestration solution that can address a wide range of production scenarios and configurations [1,2]. It is mainly used to manage containers via Pods and it also facilitates the development of native cloud applications. Led by the Cloud Native Computing Foundation, the Kubernetes project is used as a foundation for multiple commercial and community-driven orchestrators. An important Kuberentes flavor available on the market, is OKD, an open-source version of the Openshift Container Platform, currently maintained by the Redhat community. OKD combines the versatility of Kubernetes with some production-graded automation mechanisms and security admission controllers, to provide a fully clusterized solution, ready for production activities. In terms of DevOps methodologies [3], the OKD supports the construction of continuous integration, delivery or deployment pipelines, aiming to support the entire application development lifecycle. Similar to Kubernetes, the OKD exposes a complex API, offering the possibility to interact and integrate with 3rd party solutions.
In the context of ATLAS Distributed Computing [4], Openshift OKD [5] can serve as an extension to BigPanDA [6]. BigPanDA is a complex monitoring tool that provides information about distributed analysis processing [7,8]. Developed as a pilot project, the current case study focuses on demonstrating that Openshift OKD can take over part of the workload of BigPanDA. The proposed solution will give the BigPanDA service the possibility to delegate the governance of the Machine Learning [9,10] data visualization microservices to a Kuberentes solution. Thus, if a tenant would like to see the results of a machine learning activity previously processed by the dedicated infrastructure of the ATLAS Distributed Computing [11], a corresponding catalog service can be accessed, on request, from the BigPanDA interface. BigPanDA will further locate the results, and it will spin up a new MLFlow [12] service container in the OKD Openshift cluster to display the required data. MLFlow is an open source platform that aims to support the entire Machine learning life cycle. It has several features very useful for scientists working with Machine Learning data. Among these features, we can find tracking capabilities for experiments, packaging mechanisms for projects, models management and versioning. For our particular purpose, we selected MLFlow to view experimental data in an intuitive format. Each MLFlow instance will be exposed to the outside world, and it will be accessible from the browser. The web service will be discoverable for a limited time, after which the BigPanDA core controller will delete the MLFLow instance and, consequently, will automatically recover the associated computational resources. Openshift OKD implements multi-tenancy, therefore, multiple MLFlow instances can run in parallel, serving multiple stakeholders.
The aim of this pilot project is to probe whether Openshift can supplement BigPanDA on a daily basis for activities that are not the main purpose of the platform. Furthermore, by taking this approach into account, some doors can be opened for a transition from the current architectural model to a cloud native or hybrid model, able to run in any public or private cloud. The goal is to expand the platform's capability with loosely-coupled features, managed by 3rd party cloud orchestrators. The logic of governance will also be outsourced, taking advantage of the orchestration routines and all the abstractions provided by Kubernetes and Openshift OKD.
In the current paper, we will formally discuss aspects related to the implementation of a new capability of the BigPanDA platform which consists in visualizing the results of Machine Learning jobs in an on-demand system. In Section 2, we will present the concept architecture of our hybrid solution, including the architectural decisions based on the constraints encountered. In Section 3, we will reiterate through the functional objects and present the way we approached each particular aspect from a technical perspective. In addition, we will present some aspects that can be optimized in a future release. In Section 4, we will run a quick functional test suite in order to validate end-to-end usability. In Section Figure 1: Concept architecture of a self-service system for data visualization services resulting from the processing of machine learning jobs/tasks 5, and the last section, we will conclude the development and integration process and provide the results of our tests.

Architecture
In the current case study, the authors will focus on defining the interaction between BigPanDA and Openshift OKD. The context of such an interaction is related to the need to view Machine Learning analysis data in a friendly format. At the same time, a management model detached from the core ATLAS orchestration is being tested. As can be observed in the concept architecture and interaction flow ( Figure 1), a tenant, which has previously submitted a Machine Learning job or task to the ATLAS processing infrastructure, can also request a visualization service to display the results. For each demand, the BigPanda controller triggers the creation of a web service in Openshift OKD. As part of this routine, BigPanda's role is to locate the results and download [6] them from the ATLAS distributed infrastructure, via Rucio [13]. Once data is stored and indexed within BigPanDA, the controller calls the Openshift OKD API and triggers the creation of an MLFlow web service along with all the Openshift (Kubernetes) communication and configuration objects. The MLFlow pod downloads the Machine Learning artefacts from BigPanDA and stores them locally, in a temporary/volatile location.
Following such an approach, the BigPanDA solution outsources data visualization management to an external cloud platform, optimized for this type of interaction. The principle on which we based our architectural model is "segregation of duty". Therefore, instead of having one solution that fits all, various auxiliary routines can be detached from the main solution and executed through specialized platforms. Openshift OKD is a viable platform for various scenarios, especially when it involves creating on-demand services, multi-tenancy and intelligent container management. To take advantage of OpenShift and also to increase the portability of our solution, we deliver our MLFlow instances in containers and pods. Following this cloud native model, our solution is extremely portable and can be easily adjusted to run on most public and private clouds. Openshift OKD is a Kubernetes implementation, therefore, it inherits the same base architecture as the one implemented by the core project. A service that needs to be exposed to the outside world will most often be accessible through the ingress-managed load balancer. Therefore, if a client would want to connect to a service running inside an Openshift OKD cluster, the client will be able to use a specific domain name to reach that service. Furthermore, an ingress controller can also manipulate internal routes to forward traffic toward various services based on subdomains or paths selectors. As we will further discuss in the next section, BigPanDA will use the routing mechanism (ingress controller) to enable multi-tenancy by creating unique fan-out definitions ( Figure 2) and communication primitives for each MLFlow instance, through the OKD API.
In addition, we observed that both OKD 3.11 and MLFlow v1.9 (versions selected for our proof of concept) do not support target rewriting. This concept means that if an HTTP request path does not correspond to an existing resource or application endpoint, the request will end up generating a resource-not-found error. Since we will be using one DNS domain for all MLFlow instances running in parallel, we will identify each instance by a random string embedded within the resource path. Some web servers or web applications support remapping a non-existing endpoint to the root path. However, both out-of-the-box OKD 3.11 with HAProxy Ingress Controller and MLFlow v1.9 do not have support for such configuration. To work around this issue, we adopted a multi-container design pattern for pods -the adapter pattern [14]. An Adapter object is located in front of the main MLFlow application service, and it handles the requests coming from the load balancer. Each HTTP request will be translated and sent to the MLFlow service through the localhost interface ( Figure 2).
Finally, and also part of the architectural vision, we decided to create our own base container. It includes the MLFlow middleware and the assembly scripts and is built using the native build capabilities available in Openshift OKD. Therefore, we separated the preparation of the base MLFlow image from the actual application deployment, respecting the DevOps methodology and practices ( Figure 3). All the configuration items are currently stored separately from the BigPanDA code base, and everytime a change is detected, a trigger executes the Continuous Deployment pipeline.

Implementation
In order to spin up a data visualization process for Machine Learning analysis tasks in Kubernetes, a series of steps has to be performed in order to make this service available for consumption. Each configurable item in Kubernetes and Openshift can be generated through native or custom object definitions, processed by specific internal controllers and manipulated from outside via Kubernetes/Openshift API. In our case study, BigPanda needs to create four such objects, two defining the web-service that includes the MLFlow application and Machine Learning artefacts and other two abstracting the network configuration ( Figure 2). The access to data visualization service is provided through Routes (ingress definitions). When a new MLFlow service is requested from BigPanDA, the platform generates a seven byte unique id per request. BigPanDA uses this string to create a mapping between the workload id of a job/task submitted in the ATLAS processing infrastructure and the corresponding Openshift request. As this string uniquely identifies a MLFlow service instance, it is also used to compose the unique URI for external access. Therefore, the string is concatenated as an endpoint (path) to a shared, pre-provisioned DNS domain (fan-out definition) and configured internally to reach the corresponding MLFlow instance. This approach facilitates the multi-tenancy character of the ensemble, as each MLFlow service will be obfuscated and reachable by a specific, randomized URI. However, once a request passes the ingress controller and it is forwarded to the MLFlow web service, the HTTP request keeps the original path (unique id) in the HTTP request header. When an HTTP request reaches the MLFlow service, the middleware tries to find, in the application structure, an endpoint corresponding to the path selected. Since there is no such resource, MLFLOW will return an HTTP 404 error -resource not found.
Because the purpose of having a random endpoint in the URI is to select the appropriate MLFlow instance, the need to have an intermediate HTTP proxy arises. Therefore, if the application itself is not able to deliver resources from custom endpoints, an HTTP request adapter must handle the original HTTP call and alter the original content. Therefore, the adaptor must remove the random path string and replace it with the root (/) path. In the vanilla Kubernetes implementations, the most common ingress management solution adopted is the NGINX ingress controller, which, by design, supports target rewriting via annotations. However, Openshift OKD 3.11 comes with an out-of-the-box HAproxy Ingress Controller for managing routes which does not have support for these particular use cases. Also the MLFLow version 1.9 which has been selected for the current Proof-of-Concept, is also unable to handle custom endpoints for serving data. In this regard, as described in the previous section, we adopted an adaptive pod design pattern. Therefore, instead of having only one container exposing the Machine Learning job results, our solution proposes a two container model, one acting as a proxy (adapter), filtering all requests and staying in front of the MLFlow service and the other, the web service itself, responsible for data presentation.
In programming, this approach is called the adapter model/pattern. As shown in Figure 4, the adapter entity works as a reverse proxy, altering the HTTP header of the initial request and replacing the random path sting with the root path. Since a Pod, in Kubernetes terminology, shares the same network namespace among the containers, the communication between the MLFlow service and the Proxy-Pass NGINX server is handled by the localhost. In more extensive scenarios, an adapter service can define more complex models, acting on the entire spectrum of input data. In our particular case, the main role is to remove any identification information from the request and to expose the MLflow capabilities to outside on behalf of the webservice.
Machine learning analysis data must be located within the MLFlow container. For this initial iteration of our pilot project, the data is downloaded from BigPanda and temporarily stored inside the container. When a new Pod is created, the initial step performed by the main container is to connect to the BigPanDA and to obtain the specific machine learning results. The source of data is encoded in an environment variable and set in the pod definition at the time of creation. At this point in time, the BigPanDA openshift controller sends a request to the Openshift OKD API in order to create the application service and all the communication primitives. Since the underlying pod volumes are volatile, data downloaded from BigPanDA will be conserved as long as the web service is up and running. In case of a failure, the pod may be recreated, and the data may be lost during this process. Therefore, a new container will have to re-download all the artefacts. This approach respects the principle of idempotency, essential when developing cloud applications for Kubernetes. Thus, one has to ensure that consecutive executions of the same initialization routine will not affect the results. A more optimal solution would be to use init containers and maybe also to mount directly the distributed ATLAS storage system [15] in the container. By implementing the latter feature, our solution will avoid duplicating the date and it may contribute to the sustained effort to ensure a smaller disk footprint in the organization. For the former idea, the approach will help in cleaning the architectural model by detaching the initialization function from the actual execution.
Opting for the OKD Openshift could also provide the opportunity to adhere to a safer and more restrictive operating model. OKD enforces several security policies over the container context, including non root execution and UID randomization. Therefore, in the context of an on-premise cluster implementation, a vulnerability in one of the components running inside a container will not expose the entire hosting system. Moreover, for our particular toolkit, all the artefacts selected were imported from official certified sources, which are constantly updated and actively maintained. An essential process to ensure a quick reaction in the event of a major security problem is the use of the OKD automation pipelines (Figure 3 and Figure 5). In addition to the existing Kubernetes primitives, OKD Openshift brings several other solution-specific objects to support the creation of an application development lifecycle pipeline. Thus, we benefit from the existence of such a feature to manage the construction and configuration of our own MLFlow middleware container image, using OKD's native BuildConfig routine. Moreover, once the middleware image is reconstructed and stored in the embedded registry (artefacts repo), the build routine notifies the DeploymentConfig controller about the changes. This process will further trigger the upgrade routine, and it will redeploy all the existing running MLFlow instances with the new middleware version. The update strategy selected for our pilot is the rolling-upgrade model. This strategy significantly reduces the downtime by running two application instances at the same time -one with the new version and the other the current one. Once the new application instance is initialized and ready to serve content, the network will automatically configure the routing service to point to the new instance.
All the middleware configuration items and build artefacts are stored in an external GitLab repository. Once a PUSH event is detected, Gitlab notifies Openshift via a predefined webhook. When the endpoint is reached, Openshift OKD starts a new build process, which imports the latest version of the artefacts and assembles them according to the instructions. During this process a new container image is built. Once this process is successfully completed, the rolling-upgrade routine is also triggered. Therefore, all the existing MLFlow instances, running on the cluster, are automatically updated one by one ( Figure 5). As can be seen in Figure 3, the entire pipeline is developed in accordance with DevOps methodologies and practices and it is fully generated with native OKD Openshift routines.
The last component developed for the integration between BigPanDA and Openshift is a service deletion routine (garbage collector). The adopted model proposes a retention period of 24 hours for both services and data. After this period, BigPanda will clear all resources associated with a previously submitted request. In order to partially detach this routine from the BigPanDA controller, one way would be to use privileged Kubernetes cron jobs. For this particular approach, we have developed a Python procedure that interacts with the OKD API and scans for DeploymentConfig objects within the project namespace. This procedure retrieves the creation timestamp from the objects attributes (deploymentconfig.metadata.creationTimestamp) and calculates the number of hours elapsed since initialization. If the 24-hour deadline has passed, the python routine will also collect the instance ID, encoded in the labels section at the time of creation, and delete all associated objects (all four Kuberentes/Openshift objects identified by the same instance id). Kuberentes uses user-defined labels (key value pairs) to identify and manipulate objects. For our solution, we decided to attach the MLFLow service identification request string in the label section of each object created in OKD. Thus, when the garbage collector identifies an old instance running within the cluster, by just retrieving the identity string, our Python procedure should be able to identify all the objects corresponding to an old request (Fig. 02) and delete them. The Kubernetes cron job can be set to run once per hour and therefore, to continuously monitor the cluster.

Opportunities for improvement and future work
For a second release, we have already identified several optimization development tracks aiming to improve both the provisioning process and architectural model. A way to better segregate the responsibility for certain processes is to work with micro services, small routines that accomplish only one atomic activity [10]. Currently, the main application container processes two important tasks at bootstrap time: download Machine Learning analysis data from BigPanda and start the MLFlow web service. In order to separate responsibilities and to improve the overall experience, especially regarding recovery policies applied by Kubernetes, the data retrieval task can be delegated to an initialization container entity. An init container is an application container that runs during the initialization stage and must run to completion before starting the main application containers (MLFlow and Adapter). Once such a segregation model is applied, if a health check probe fails for example (eg: liveness probe detects an anomaly), Kubernetes will only try to restart the application service and not the entire initialization process. The artefacts can be shared through emptydir volumes and attached to both sequential containers. Furthermore, in order to keep the idempotency and to follow the best practices, BigPanDA will also have to compute and provide the checksum of the Machine Learning artefacts. This will allow the init process to compare what version of data is available upstream and what is already downloaded locally.
As previously mentioned, another tangential scenario could be the configuration of Kuberentes pod-specific health checks (liveness, readiness and startup) for each MLFlow Figure 6: The interaction between the source code management platform and the cloud orchestrator at middleware configuration change events instance. Although MLFlow service doesn't have a special / health endpoint to be queried, one can use the tracking services ( mlflow. set_tracking_uri () ) to check the consistency of the web service.
Another possible improvement would be to combine all OKD object definitions into a single jumbo object (template), applied through a single REST API call from BigPanda to OKD Openshift. However, in a modern approach, the previous capability can be further enhanced through operators, a design pattern and development framework that extends the existing Kubernetes / OKD API and can delegate the provisioning management of MLFlow instances to a custom developed controller. Thus, instead of sending one or more HTTP calls describing four native Kubernetes/Openshift objects, we will need to create a single custom object description that instructs our custom controller how to provision a new ATLAS MLflow instance.
In addition, the migration to OKD 4 could also improve the current architectural model, as since its official launch, this flavor provides an enhanced HA proxy, capable of handling target rewrites. Therefore, we can reduce the footprint of the main application pod by removing the adapter function.

Functional testing (end-to-end testing)
As mentioned in the previous sections, we are using GitLab to store all the configuration items for middleware (MLFlow). GitLab supports event monitoring and notification and it has native integrability with 3rd party solutions through several interfaces including webhooks. Therefore, when a PUSH event occurs, GitLab runs the webhook routine and it sends an HTTP POST message to the Openshift OKD, announcing that a change has been detected. As can be seen in the Figure 6, this event triggers a build process that concludes by pushing a new container image to the internal registry. Following the DevOps methodology, this chain of events defines a continuous integration pipeline.
During the build process, the entire image configuration repository is cloned. The build pipeline uses the Docker Build strategy, a method that searches for a Dockerfile in the repository and creates a docker container image based on the instructions written there. In the Figure 7, it can be seen that the previous statement is correct. Once the build pipeline is triggered, the entire repository is cloned with the last commit, the Dockerfile is located within the root folder and the build process runs to completion.
The link between a build and a deployment can be provided by triggers, an Openshift OKD native mechanism that monitors a build process and notifies the upgrade routine about the existence of a new base container image version. Thus, we have used a trigger to constantly check the base image fingerprint (MLFlow docker image) and to initiate a rolling upgrade process when the base image has changed. Following this strategy, a new pod with the new version is created and executed in parallel with the current one. Once the initialization stage is concluded and all the health checks are passed, the networking is switched to the new pod, and the old one is killed. In Figure 8, this process can be seen in action, as we captured the moment of transition between pods. Figure 9: Data visualization service exposed as a web application and uniquely identified by the request ID Each MLFLow instance will have a unique ID generated by BigPanDA and configured in the Openshift object definitions. This random string is applied as a fan-out definition in the ingress router and also as part of the proxy-pass config in the Adapter component. As can be seen in the Figure 9, the fan-out definition is properly configured and the randomized URL is pointing to the corresponding MLFLow instance. Also the service is accessible from the browser, meaning that the service has been properly exposed to the outside world.

Conclusion
In the current paper we have developed a new feature for BigPanDA that offers the ability to visualize the machine learning results, produced in the ATLAS Distributed Computing infrastructure. A tenant (PhD student, scientific groups etc.) can request such a service directly from BigPanDA, and the workload will be further delegated to an Openshift OKD cluster via REST API Calls. Openshift will spin-up a MLFlow pod per request, download the Machine Learning artefacts from BigPanDA, expose the web service to the outside world and maintain high availability through the native healing mechanisms. Following this delegation model, we demonstrated that BigPanDA can easily adopt a cloud native approach and also that it can function as a catalog of scientific services, in the context of the ATLAS Distributed Computing at CERN. Moreover, we also followed several DevOps methodologies and practices to facilitate the build of the base MLFlow container from scratch. Therefore, we implemented continuous integration and continuous deployment pipelines using the native mechanisms of Openshift OKD. These pipelines are triggered automatically each time a PUSH event occurs in the external configuration items repository. Furthermore, as a good practice, the deployment strategy is using the rolling-upgrade model, a method that minimizes downtime inevitable for an upgrading process. Finally, we also implemented a garbage collector, a Python procedure that identifies old service instances and deletes them if they passed the 24 hours expiration time.
The results obtained during the testing phase certifies the end-to-end functionality of the integrated solution. For new iterations, we have already identified a few other paths of development and optimization and here we include: the use of initialization containers, a migration to the operators pattern, use of health probes and the delegation of the routing


Developing contextually aware ship domains using machine learning

Abstract Developing risk models to predict where collisions between vessels might occur is hindered by the relative sparsity of collisions. To address this, vessel encounters and near-misses have been used as a surrogate indicator of collision risk, referred to as ‘domain analysis’. When constructed empirically, using historical information, previous work is challenged by the multitude of factors which influence the passing distances between vessels. Within this paper, we conduct data mining of big vessel traffic datasets to determine the encounter characteristics across different waterways, vessel types and speeds, weather conditions and other exploratory variables. To achieve this, we utilise a novel approach of machine learning through a random forest algorithm to predict the critical passing distance between vessels in a multitude of conditions. We contribute a far greater range of influencing factors on domain size and shape than previous studies. Finally, we investigate the potential advantages of this approach to assess the spatial risk of collision across a large region. The results help to establish the factors that influence collision risk between navigating vessels and enable empirical maritime risk assessments.

Introduction
Collisions at sea are a leading cause of ship casualties. Analysis by the European Maritime Safety Agency (EMSA) shows that 26% of accidents are collisions, with between 500 and 700 incidents per year recorded between 2011 and 2018 and representing a leading cause of fatalities (EMSA, 2019). Furthermore, between 2009 and 2018, 46 vessels greater than 100GT were lost as a result of collisions (Allianz, 2019). These events can, therefore, result in significant loss of life, damage and pollution to the marine environment.
In an effort to reduce these hazards, many researchers have proposed models to predict and quantify the likelihood of collisions. First, statistical analysis of accident data and aggregated vessel traffic data to derive incident rates (Bye and Almklov, 2019). Secondly, the use of expert judgement in the form of Bayesian networks or event trees (Hanninen, 2014). Thirdly, the development of models to represent navigation safety through geometric route models (Pedersen, 1995;Mazaheri and Ylitalo, 2010;Li et al., 2012).
Maritime risk modelling is challenged to some degree by the relative infrequency at which collisions occur relative to the volume of traffic within an area. For example, in the South-West Lane of the Dover Straits, which is one of the busiest waterways in the world, of approximately 38,000 annual transits, there were an average of 1·2 collision per year between (MAIB, 2014, a rate of 1·05 × 10 −5 per transit. With a sample size of six collisions over a five-year period, the validity of predictions using statistical models is limited. By extension, in other waterways, where fewer vessels transit, and there are fewer collisions, these problems are exacerbated. To overcome this, many authors have proposed the use of non-critical accident situations, near misses or encounters as a proxy measure for collision risk (Du et al., 2020). Within this body of work, the underlying premise is that for a collision to occur, two vessels must meet, and by determining where two vessels meet more often, the likelihood of a collision increases. This concept of meeting vessels has been interpreted as a ship domain, the surrounding effective waters which a navigator wishes to keep clear of other ships or fixed objects (Goodwin, 1975). Where the domain of a vessel is breached, a threat to navigational safety has occurred (Pietrzykowski and Uriasz, 2009). Such a model has good utility for application in collision avoidance and waterway risk management (Szlapczynski and Szlapczynska, 2017). When a vessel is moving faster or is less manoeuvrable, a larger domain may be desired.
Methods of determining the size and shape of the ship domain has been a topic considered by many authors, and within the academic literature a plethora of domain shapes and sizes have been proposed (Pietrzykowski and Uriasz, 2009;Wang et al., 2009;Szlapczynski and Szlapczynska, 2017;Bakdi et al., 2019;Fiskin et al., 2020). Whilst we do not wish to repeat the extensive reviews of ship domains contained in other studies (see for example Fiskin et al., 2020), we highlight some of the key ship domains in the following. An elliptical domain (Fujii and Tanaka, 1971) is shown in Figure 1(1); a segmented domain is shown in Figure 1(2) (Goodwin, 1975); and Figure 1(3) shows a more complex domain with four axis, which lengths may vary depending on certain conditions (Wang, 2010).
Authors have generally acknowledged that the size and shape of the ship domain can change depending on several factors, including the physical characteristics of the vessels, the COLREGS situation, manoeuvrability, the human element, metocean conditions and fairway characteristics, amongst many others (Tu et al., 2017). Each model is fitted to the local conditions and research question of the particular study, and it should not be expected that domains in constrained waterways would be similar to those in open waters (Wang and Chin, 2016). The diversity of proposed domains is in some ways reflective of the lack of agreement on their structure and, therefore, would impact their predictive capability at determining collision risk (Goerlandt and Kujala, 2014). Szlapczynski and Szlapczynska (2017) and Fiskin et al. (2020) both draw a distinction between the methods through which domain models may be developed, such as analytically or empirically. We may also add ship domains that are developed using real-world trials, such as through the work of Yim et al. (2018). Empirically derived ship domains have some attractions, able to represent how vessels actually behave, arguably reducing the risk of biases during domain definition and increasing validity as opposed to other methods. In addition, provided enough encounters are analysed, different influencing factors can be taken into account (Tu et al., 2017). However, as is described in the following section, there are significant challenges with this approach, which mean that only a few studies have attempted to derive a ship's domain empirically (Szlapczynski and Szlapczynska, 2017;Fiskin et al., 2020), and the authors are aware of none that take into account multiple spatial and environmental factors.
Within this paper we propose and implement a method by which the critical passing distances between vessels can be mined, and the factors that influence those passing distances determined. We demonstrate that through the implementation of a machine learning algorithm to predict the domain size, based on the conditions of millions of historical interactions between vessels, many of the aforementioned limitations can be addressed. As a result, our work allows a flexible and intelligent ship domain which is evidence based. We then apply this domain to analyse collision risk across a large region. This paper is structured as follows: the remainder of Section 1 describes the key literature for empirical domain analysis and collision risk. Section 2 describes the datasets, methodology and model implemented within this project. Section 3 describes the results of the analysis and accompanying discussion, and conclusions are contained in Section 4.

Developing empirical domains
As defined by Fiskin et al. (2020), one of the methods to define the size and shape of a domain is empirical, using historical data of previous encounters. Perhaps the earliest example of this is Fujii and Tanaka's (1971) work, which used photographs of radar screens in Japanese waters to establish the separation between vessels, identifying an ellipse of approximately 500 m by 300 m. This ellipse varied depending on ship length, but other factors were noted as likely having an influence. Similarly, Goodwin's (1975) analysis of the Dover Straits conceptualises domains in terms of three circular segments with fixed radius of 0·85 nm, 0·7 nm and 0·45 nm, rather than smooth elliptical shapes. Such a significant difference is reflective of the local nature of each survey, with different navigational dispositions (Fiskin et al., 2020).
With the advent of the vessel tracking system AIS (Automatic Identification System), other authors have employed more extensive data-mining algorithms to determine domain size and shape. The workflow typically consists of processing AIS data, conducting a pairwise comparison of vessel positions, plotting these encounters and then deriving some form of a ship domain based on the results. Within previous studies, a number of different factors have been identified as likely influencing a domain's size or shape (Tu et al., 2017;Fiskin et al., 2020), yet analysis is generally limited to only three, namely vessel size, vessel speed and waterway type.
The EfficenSea (2012) project in the Baltic Sea tested and supported Fujii and Tanaka's (1971) work in several locations by producing heatmaps of passing distances. Whilst they observed from the heatmaps that the distributions are influenced by vessel size, speed and study area, with for example restricted channels having closer passing distances than more open waterways, these effects are not quantified. Gucma and Marcjan (2012) also used AIS to analyse the distributions of passing distances between different types of vessels and encounter situations. Their results suggest an asymmetric shape and that vessel type had little impact. Like many studies, their analysis focuses on an area of the Baltic Sea, specifically excluding ports, which might have different encounter characteristics. Hansen et al. (2013) analysed encounters at three locations in Danish waters, thereby providing some differentiation of spatial factors in encounters. They concluded that vessel domains are best described as ellipse shapes, similar to the results obtained by Fujii and Tanaka (1971). The dimensions of their ellipse are eight ship lengths by 3·2 ship lengths but were derived visually on the resulting heatmaps, that were approximately on the 7·5% threshold. Magaj (2016, 2017) compared domains within a traffic separation scheme (TSS) precautionary area, again defining an elliptical domain through determining the distances of the 7·5th percentile of encounters. Their work also notes that vessels on average transit closer together within precautionary areas as they would within traffic lanes.
The aforementioned models have used relatively simple techniques to derive domain shapes. A more analytical approach is proposed by Wang and Chin (2016), who conducted domain modelling within the Singapore Straits, analysing more than 250,000 vessel interactions. They employed a genetic algorithm, utilising a weighting of distance, to determine the angular, size and speed coefficients of domains under different circumstances, quantifying for example the significant influence of vessel speed on domain size. Zhang and Meng (2019) also study the Singapore Straits, developing a novel probabilistic ship domain, concluding that vessel speed and waterway characteristics had a significant influence on domain size.
A recent study measured ship domains by mining the characteristics of more than 600,000 encounters at 36 specific locations around the Swedish coast (Horteborn et al., 2019). They also find that the type of waterway, whether restricted or open sea, influences the domain size, as vessels must necessarily navigate closer together. Other outcomes suggested that the type of encounter influences the domain shape, but that vessel type and size had little influence.
The use of empirically calibrated vessel domains has several advantages. They enable an evidencebased assessment of vessel interactions, without relying on subjective inputs or defining a priori what collision situations are. Yet, determining empirical models of ship domains have three key challenges. Firstly, proposed methodologies are unable to capture or separate the impact of multiple, interrelating variables and, therefore, previous studies typically utilise only a few factors (Wang and Chin, 2016). Secondly, such studies are isolated to single case studies of specific waterways (Szlapczynski and Szlapczynska, 2017;Zhang and Meng, 2019) and, therefore, lack generalisation to other study areas, such as between ports and the open sea. Thirdly, the vast volumes of data required to be reflective of different conditions and waterway characteristics makes this approach computationally expensive. Furthermore, others have proposed that additional factors should be included, such as weather, daylight and fairway conditions (Horteborn et al., 2019). Within this paper, we seek to address these research gaps.

Vessel traffic data
The Marine Cadastre (2020), a joint project by the Bureau of Ocean Management and the National Oceanic Atmospheric Administration, publish AIS data collected from the US Coast Guard's national network of AIS receivers. Two seasonal samples were extracted for all regions between 1 January 2018 and 31 January 2018, and 1 June 2018 and 30 June 2018. The data is approximately one-minute resolution containing dynamic positional data, such as latitude, longitude, timestamp (UTC), course and speed and vessel identification attributes, such as MMSI number, vessel type and vessel size. The data covers a variety of different environments and waterway configurations, with offshore, coastal and inshore waters all represented.
It should be noted that the Marine Cadastre data does not provide AIS offsets, and only the vessel GPS position. As such, it is not possible to perform a correction to find the centroid of the vessel and, therefore, the domains may be off centre, introducing an error in the vessel position in the subsequent calculations. Given the size of vessels in the study, this error would rarely be greater than 100 m.
The data was extracted and processed following a number of steps. Firstly, we undertook linear interpolation of the vessels position, speed and direction in order to standardise the data at set oneminute intervals to allow for direct comparison. Secondly, where a vessel's heading is unavailable, we utilised the vessel's course to calculate interaction angles. Thirdly, any missing attributes in the data were filled in using the median values.
It should be noted that a one-minute resolution interpolation is coarser than that used in previous studies, such as one-second (Horteborn et al., 2019), ten-seconds (Rawson et al., 2014) or 30-seconds (Hansen et al., 2013;Zhang and Meng, 2019). However, we compensate for this through volume, which allows us to analyse many encounters under a greater variety of conditions, with 77 million and 125 million vessel positions in January and June 2018, respectively.

Approach
Three key methodological issues need to be addressed; firstly, how can we efficiently derive a dataset of encounters and associated temporal or spatial factors? Secondly, how should an encounter between two vessels be judged as a significant encounter? Thirdly, how can a machine learning model be used to predict the significant encounter distance in any situation? The methodological approach undertaken to achieve this is shown in Figure 2 and described next.
Firstly, a pairwise comparison of all vessel positions within 3 km of each other is conducted. The choice of this distance is critical; greater distances will include more encounters, increasing both processing and the average passing distance between vessels. Previous studies have used varying distance thresholds, in both Wang and Chin (2016) and Magaj (2016, 2017), a 2 nm (3·7 km) search radius is chosen without justification, whilst in Horteborn et al. (2019) a 5 km distance is utilised. Hansen et al. (2013) use a 3·5 km distance, justified as 10 times the longest ship in the dataset. Arguably the distance should be greater, such as at 6 nm, the usual observation range of ship-borne radar in coastal areas (Zhang et al., 2016).
Secondly, we determine that 1 in 10 encounters would be critical for determining domain distance, such that 90% of encounters are considered safe. To determine this, we need to identify the closest 20% of encounters for each combination of circumstances (vessel type, location, speed and others), and develop a regression model to create a best fit that is approximate to the 10% threshold. Other thresholds have been suggested by other authors, such as 5% (Horteborn et al., 2019) or 7·5% (Hansen et al., 2013;Pietrzykowski and Magaj, 2016).
The determination of both the critical search distance and percentage threshold for domain size are arbitrary to an extent, and the results of the analysis are sensitive to the chosen values. For example, Horteborn et al. (2019) use 5 km and a 5% threshold, whereas we use a 3 km and 10% threshold, which would be similar, but we require significantly less computations between ship pairs. We have sought to utilise logical and practicable thresholds but invite further work to assess the sensitivity of the results to other thresholds.
The workflow to identify encounters is similar to those proposed by previous authors in the literature review. Firstly, the AIS data is interpreted to one-minute resolution using a linear method to standardise the AIS dataset temporally. Secondly, a pairwise comparison at every timestamp was conducted, with only those records retained where the haversine distance between the vessels was less than 3 km and the average speed of the vessels greater than 1 kt. The retained results are spatially indexed to enable joining with metocean and other spatial datasets.
During data exploration, it was determined that some encounters occur frequently and are the result of normal operational behaviours. For example, in ports, tugs and pilot boats will come alongside navigating commercial vessels, yachts might be racing in groups, or fishing boats may operate in pairs. Each of these situations would generate encounters, and whilst a collision might still occur, they do not reflect the normal navigational practice between vessels. To account for this, we limit the encounters to the following pairs which are unlikely to exhibit such issues: Thirdly, a machine learning algorithm is utilised to predict the critical passing distance between vessels under different conditions through regression. To achieve this, the 20% closest values for each encounter circumstance are extracted, split into a training and testing dataset for evaluation, before random forest regression is utilised to predict the passing distance for all of the input conditions.

Feature selection
For each of the 5 million resulting encounter pairs, a set of features are identified and used to categorise the type of encounter. Table 1 shows the features used in this assessment and these will be discussed in turn. In order to identify the closest 20% encounters of each configuration, it is necessary to reduce some of these continuous features into categorical scales to support computation. Increasing the number of categories for each feature would increase the resolution of the analysis but reduce the sample size within each category. It should be noted that the choice of these categories would have some influence on the results, and, therefore, the rationale has been provided where this has been undertaken.

Bearing
For each encounter, the relative bearing ( ) from vessel 1 to vessel 2 can be calculated as the difference between the heading of vessel 1 and the geographic bearing with vessel 2 (see Figure 3). In Horteborn et al. (2019), 5°bearings are used, whilst Fiskin et al. (2020) use both eight and 16 nodes. In our case, we also use eight nodes at 45-degree groupings, such that each value is ±22·5°, i.e. 090°includes 067·5°t o 112·5°(see Figure 3).

Encounter type
The heading differential between the two vessels can be calculated and used to define the encounter situation in line with the International Collision Regulations. The categories are as follows (Montewka et al., 2010): • Head On -170-190 degrees • Overtaking -350 to 10 degrees • Crossing Stand On -190 to 350 degrees • Crossing Give Way -10 to 170 degrees 2.3.3. Distance from shore Distance from shore was calculated using a high-resolution vector world landmass shapefile, converted into a 500 m resolution raster and the ESRI ArcGIS Euclidian Distance function performed to calculate the distance from each raster cell to the closest cell containing shoreline. This has been categorised between coastal and offshore waters using the 12 nm limit, and 5 km used to differentiate port and inshore waterways.

Vessel speed (SOG)
Provided by AIS data, categorised into five knot bands up to 15 knots.

Vessel size
Provided by AIS data, categorised into three groups of less than 50 metres (including the majority of fishing and recreational craft) and greater or less than 250 m. This separates commercial vessels into two size groups based on the approximate size of an Aframax or Panamax vessel. This enables a comparison between the domain characteristics of the largest commercial vessels from the majority which are less than 250 m.

Vessel type
Provided by AIS data, categorised into commercial cargo or tanker vessels, fishing vessels and recreational craft, as described in Section 2.2.

Near TSS
Some in the literature have proposed that ship routeing measures effect encounter characteristics. To test this, an additional feature was included which determined whether the encounter occurred within 2 nm of a TSS, including traffic lanes and precautionary areas. Spatial data files of these features are provided by the MarineCadastre (2020).

Near port
Whether the encounter was within 20 nm of a major US port -this includes approach channels and port limits. These details are available from The National Transportation Atlas Database (USDOT, 2020). A more accurate delineation could be achieved by marking the waterways of each individual port; however, at a national scale this is not practical.

Wind speed
Wind speed at time of encounter was derived from the EU Copernicus Earth Monitoring Service. A NetCDF file containing time, location and wind speeds was extracted for June 2018 from dataset WIND_GLO_WIND-L4_NRT_OBSERVATIONS_012_004. This dataset contains wind speeds at six hourly temporal resolution and 0·25-degree spatial resolution. For each grid cell, the data was interpolated to one hourly resolution and joined to the encounter dataset based on its time and location.

Day/Night
Based on the UTC, latitude and longitude of the encounter, a spatial query was conducted to determine the local time zone. Any record between 06:00 and 18:00 is categorised as a day encounter and between 18:00 and 06:00 categorised as a night encounter.

Random forest prediction of domain distance
From the 1 million encounters that account for the 20% closest encounters in each group, a regression problem can be framed to predict the passing distance given the characteristics of that encounter. Given the multiple factors which influence ship behaviour and risk perception, we propose that a machine learning algorithm can be utilised to learn the domain size and shape of a given vessel in a given situation. In this case, we utilise the random forest algorithm in the scikit-learn package of Python. Random forests (RF) (Breiman, 2001) are a popular ensemble-tree learning algorithm which have been applied to many regression and classification problems in maritime contexts (Jin et al., 2019). Whilst many other regression algorithms are available, we utilise RF due to its widespread popularity and availability in numerous languages as well as inherent properties, such as training speed and robustness when using high-dimensional and unbalanced datasets (Breiman, 2001). RFs consist of many (k) decision trees, which themselves consist of a root node, child nodes and terminal nodes. The root node consists of training data, which is split into a child node based on attribute variables and maybe further split into subsequent child nodes. The algorithm seeks to ensure that each child node is as homogenous as possible, and when this is reached the node becomes a terminal node. Decision trees are prone to overfitting, as they are capable of replicating the structure of the training data perfectly. To overcome this, RFs introduce bagging and random subspace into the algorithm. Bagging (bootstrap aggregating) involves the training dataset being sampled with replacement, and random subspace involves randomly selecting attribute variables when splitting the dataset. Figure 4 shows a schematic of RF, with the training dataset sampled and then decision trees constructed, with the average of the predictions of each tree used as the resultant prediction. As a result, RFs are an ensemble of decision trees, avoiding overfitting by producing a diversity of trees.
In addition, RF allows for the calculation of the relative importance of each feature in producing the predictions. Feature importance, also referred to as Gini importance, is calculated by comparing how much the tree nodes that use each feature reduce the impurity on average (Breiman et al., 1984).
The data is split into a training and testing dataset, with an 80:20 ratio, and a RF trained on the training dataset only. Hyperparameter tuning was performed using a random search of a hyperparameter grid using cross-validation, with the best performing parameters selected based on the 'r 2 ' scoring, which achieved 0·74 on the training set. The selected regressor was then applied to predict the passing distance on the test set, with an r 2 of 0·69. It should be noted that by segmenting the dataset in Section 3.3 into the top 20% of each category, we ensure that the dataset is not imbalanced by the encounter types that are most common and therefore avoid overfitting

Mined encounter densities
In order to establish how encounters change in given conditions, we first plot the densities of interactions between commercial vessels in Table 2. Within these plots, one ship is located at x = 0/y = 0, heading due north, and the densities show the most frequent positions where encounters are recorded. Therefore, to interpret this plot, if an area of high density is seen at for example, x = −1000 and y = 0, this means that a greater propensity of encounters occur for a transiting vessel, on the port (left) side as it transits.
The densities demonstrate that the distribution of encounters changes significantly with encounter type and distance from shore, supporting previous work (Horteborn et al., 2019). For example, in constrained waterways, vessels must necessarily pass each other with less separation than they might otherwise choose offshore.
For head-on encounters, it can be seen that the highest density of encounters occurs on the port side. This reflects Rule 14 of the collision regulations that requires two vessels meeting on reciprocal courses to both turn to starboard to pass each other on the port side. This results in an asymmetric domain that has been previously highlighted (Goodwin, 1975).
Overtaking encounters are described under Rule 13 of the collision regulations, such that the overtaking vessel avoids the other vessel. Within our data, in the coastal locations, a clear ring can be seen as vessels overtake one another, leaving sufficient sea room to keep clear. For both the inshore and to some extent the coastal regions, the high density approximately 2 km ahead and astern indicates the natural separation. At the centre of the overtaking offshore plots, two areas of high-density are vessels moored alongside one another conducted ship-to-ship transfers (STS), are removed from the subsequent modelling as they are outliers for our purposes.
Crossing situations have been compared for stand-on and give-way situations as defined under Rule 15 of the collision regulations. These state that when two vessels cross one another, the vessel on her own starboard (right) side shall keep clear of the other, avoiding crossing ahead. Therefore, in the context of our stand-on plots, the other vessel is necessarily crossing left to right, and for the give-way plots, the other vessel is crossing right to left. Taking the coastal region as an example, the data shows that a giveway vessel would cross a stand-on vessel's course approximately 2 km ahead, or alternatively would cross astern at a similar distance, a pattern similar to that shown in Hansen et al. (2013). Offshore, a greater proportion of vessels choose to cross astern rather than ahead of a stand-on vessel. Inshore, with linear channels, the crossing potential for commercial ships is limited, and therefore a similar but less well-defined pattern is evident.

Influence of factors on domain size and shape
Understanding the reasons why passing distances and domain shapes vary has been a focus of several previous studies (Horteborn et al., 2019). Figure 5 shows a correlation matrix of each of the features investigated in this study, in order to determine whether there is correlation between the different independent features. Weak positive correlations indicate that vessel speed, length, weather and traffic schemes have a minor impact on domain size, whilst the passing distance is inversely correlated with the presence of ports. Individually these correlations are weak and without statistical significance, which has been concluded by others (Horteborn et al., 2019). However, in Section 3.1 we have demonstrated that by combining factors, much clearer patterns emerge, which are demonstrable through our implementation of machine learning.
Through application of RF, we can measure the feature significance towards the model predictions across our exploratory features, which is shown in Figure 6. In each case, we will examine the impact of these features on certain domain shapes.
The most significant feature is distance from shore, with vessels navigating naturally closer together when they are in confined waterways. This is demonstrated in Figure 7 for head-on and overtaking commercial encounters. The circumstances of an encounter in a narrow channel would likely be seen Table 2. Commercial encounter density -speed greater than 5 kts.
Vessel is orientated north-up at x = 0, y = 0. Distances in metres.
as typical in that situation, whereas were the same encounter distance to occur offshore, it might be seen as a near miss. It is also of note that there is a difference between the two encounter types shown in Figure 7, with overtaking encounters exhibiting a more elliptical shape (such as described by Fujii and Tanaka, 1971), with little difference between coastal and offshore waters, whereas head-on domains are offset to starboard, with a much greater size difference, perhaps due to a greater perceived risk of collision in head-on situations. These biases to starboard are evident in some of the segmented domains proposed, for example by Goodwin (1975).
The second most significant features are vessel speed and vessel length for the two vessels involved in the encounter. With respect to speed, it is logical that the faster a vessel is travelling, the greater the safety distance the master wishes to keep as the opportunity to either stop or manoeuvre clear reduces. Figure 8 shows commercial ship overtaking encounters at various speeds, demonstrating a clear increase in the natural domain size as the vessel increases. This effect is approximately a doubling between less than 5 kts and 10-15 kts, an effect which has been observed by others (Wang and Chin, 2016;Zhang and Meng, 2019). Figure 8 can be directly compared to Wang and Chin (2016) showing a similar overtaking forward distance of between 1 km and 1·5 km for a commercial ship travelling 15-20 kts. However, our domains are less elliptical in nature, possibly due to the greater range of locations under assessment.
Vessel size was also shown to influence domain size and shape with larger vessels generally maintaining a greater separation from other vessels (see Figure 8). Previous work has shown that this effect is limited (Horteborn et al., 2019); however as previous work has typically considered only commercial shipping, it is likely that our inclusion of recreational and fishing vessels would increase this effect.
The third group of factors include wind speed, proximity to ports and traffic schemes are shown to be less significant than the aforementioned variables. In significant weather conditions, vessels are affected by windage and other phenomenon that impact the safe handling of a vessel, and as such, it would be sensible to increase the passing distance between one another. Figure 9 shows overtaking commercial vessel encounters in coastal and offshore waters and suggests that this effect is only significant once wind speeds exceed 10 m/s (Beaufort Force 6), providing empirical evidence to support similar conclusions based on ship motion modelling . It should be noted that as the metocean data is averaged at six hourly intervals, isolated squalls and gusts are  not captured within this analysis and may result in more significant variations than we are able to describe. Figure 10 compares the domain shapes on overtaking situations between commercial vessels within traffic schemes and near ports. It is surprising to see TSS as one of the least significant factors shown in Figures 5 and 6, this being one of the key risk control measures that can be put in place by regulators and governments. Traffic lanes seems to slightly reduce the domain width, perhaps indicating that the predictable linear flows in traffic schemes increase the comfort of the bridge team in overtaking at closer distances. With this said, on average the presence of a TSS has little impact, as the circumstances of the individual encounter are more significant. This might partly explain why more elliptical domains are derived when studies are performed in areas with a TSS, such as Wang and Chin (2016).
Port approaches and waterways have a more significant influence at reducing the domain size due to the constricting effect of buoyed waterways that necessitates closer interactions, a similar effect as the one shown in Figure 7. The significance of this difference is such that some authors have chosen to either exclude these waters (Gucma and Marcjan, 2012) or concentrate on them entirely (Rawson et al., 2014).
Whilst it has been suggested that daylight might be a contributory factor (Horteborn et al., 2019), within our exploratory data analysis there was no clear evidence and, therefore, it has not been included within our modelling. The greatest effect of this factor is the temporal variation in the volume of fishing and recreational traffic throughout the day.    These results demonstrate that domain size and shape, and therefore the passing distances between vessels, varies based on numerous input conditions, including vessel size, speed, waterway conditions and weather. Therefore, attempting to utilise vessel domains that do not account for some of these conditions risks producing results that are skewed either in exaggerating or under-representing the frequency of interactions between vessels. For example, Figure 10 shows that the same ship domain shape or size cannot be applied both to ports and traffic lanes. By utilising machine learning, we have constructed a dynamic ship domain which can account for many more features than previous models and, therefore, better reflects the natural behaviour of vessels in meeting situations.

Application for collision risk assessment
Having trained and calibrated our empirically derived ship domain, we can then utilise it to study where critical encounters are more likely to occur. Many previous studies have utilised analysis of the frequency and spatial variation in domain encounters a measure of collision risk (Du et al., 2020). Within this section we seek to assess domain encounters between commercial vessels in the Puget Sound between Washington State and Vancouver Island. Figure 11 shows the density of commercial shipping in the region, clearly identifying the major routes as vessels enter the Strait of Juan de Fuca and the TSS, before proceeding north to Vancouver or south to Seattle. Figure 12 compares the frequency of critical encounters as derived from our model. These are where two vessels encounter at a closer distance than the critical encounter distance predicted for those circumstances, as defined in the previous sections. The results show that certain regions have a far greater number of encounters than would be expected given the level of traffic. For example, the Haro Strait to the east of Victoria necessitates a constriction in the flow of traffic, increasing the number of encounters. The most significant areas of high encounters are near to ports and terminals, where both the density of traffic is greater, and the available sea room is less.
Whilst several studies have presented analysis of collision risk through the use of domain analysis (Goerlandt and Kujala, 2014; Rawson et al., 2014), we argue that this approach has several advantages. Firstly, as our domain is empirically derived from previous transits, it reflects the natural behaviour of vessels in meeting situations, as opposed to other methods, such as expert judgement which might be subject to bias. Secondly, by utilising machine learning to construct the ship domain, we are able to integrate a far greater number of different features than other methods, allowing the algorithm to learn the domain shape in hundreds of different permutations of circumstances. This would not be possible utilising traditional methods. Thirdly, such an approach has good scalability; with more training data it would be possible to include other features, such as wave heights, visibility or extents pilotage districts, in order to further develop the model.

Conclusions
The results of the analysis presented in this paper support the wide body of literature concerning ship domains. Based on mining the passing distances between navigating vessels across a wide range of locations, environmental conditions and other circumstances, the size and shape of this domain can be derived. These domains clearly vary between locations, speeds and encounter types, supporting conclusions reached in previous work, but are influenced by other factors, such as weather, which have not previously been assessed. The analysis demonstrates that encounter distance is significantly less for inshore as opposed to offshore waters, low-speed to high-speed encounters and small vessels rather than large vessels. Whilst there is some evidence that higher wind speed and encounters outside of TSS lanes result in larger domains, this effect is less significant, and no evidence was found for the influence of day and night on domain size.
To capture all of these interrelating factors, we have developed a novel approach of utilising a machine learning regression algorithm to learn the passing distance between vessels in each circumstance. This ship domain is dynamic to ship speed, size, encounter type, weather and waterway characteristics, having trained on significant quantities of ship encounter data. To our knowledge, this is the first time that this proposed methodology has been applied to characterise ship domains. These learnt domains can then be applied to determine where critical encounters most frequently occur and inform maritime risk assessments.
Several other aspects of this research are open to further work. Firstly, we have discussed the apparently arbitrary method at which key parameters in empirical ship domain mining are derived, namely the maximum filtering distance and the ratio of critical to noncritical encounters. This gap is bridged when using nonempirical methods, such as expert judgement, but further work is needed to understand the sensitivity of empirical domains to parameter selection. Secondly, whilst we utilise a large amount of AIS data across a wide geographic area, we do not take advantage of any big-data processing technologies. The sheer volume of AIS data, and the computational complexity of calculating pairwise interactions between vessels can make scalability a challenge. Although some approaches have been proposed to streamline this process, such as reducing the raw AIS to simpler line segments (Horteborn et al., 2019), this limitation remains to some extent. Optimising domain calculations in bigdata processing technologies such as Apache Spark would overcome this limitation by expanding the volume of data and variety of factors that could be investigated, but we are not aware of any previous research for this purpose.
The results show that it is possible to develop intelligent ship domain models, that better represent the usual navigation practice than traditional approaches. Through the use of a machine learning algorithm, a far greater number of influencing factors can be included in the subsequent domains than would be possible with traditional methods. Using models ill-suited to the local conditions under study can result in unrealistic results and therefore this has implications for the wider risk modelling literature.


DNA ; Digital Data Storage Device

The growth of various types of information happens day by day. The increased production of data, in turn, led to the invention of various data storage tools. The idea of storing data in DNA is now one of the major focuses. There are so many methods for storing data in DNA, the method put forward by Nick Goldman and Evan Birney has so many advantages over the other methods. They had converted data in the binary form into ternary by using improved Huffman coding of base 3. This helped to reduce the error rate that had produced in the earlier works.

Introduction
There is a huge increase in the amount of data day by day. Our storage necessity increases by 50% every year. Data produced from various sources like important documents, data from social media all are increasing in a few times. To store the increasing amount of digital data, we should find the appropriate storage mechanisms. From the old days, we had used bones, rocks, and paper to store data. The evolution progressed after that through punched cards, magnetic tapes, floppy discs, gramophones and reached to CD"S, blue ray discs, DVD"S, and flash drives. These devices can easily get damaged by temperature, moisture and magnetic variations in the surrounding. All the above-mentioned devices are strong and nonbiodegradable, it will lead to environment pollution. And also, it will loss efficacy with time. This led the scientists to discover more efficient DNA storage model.

Advantages from other digital data storing devices
An unbelievable amount of enormous data can be stored on DNA. The long shelf life and robustness are the properties of data. Even after thousands of years the information stored on DNA can be recovered by providing optimum conditions for keeping DNA in dry, dark and cold conditions [7]. Current data storing devices are more prone to data loss. The power usage by other data storing devices is very high, while for DNA, it is much more efficient than these devices. DNA is the genetic material of all living beings and is made up of polynucleotides, it contains a pentose sugar, a phosphate group and nitrogenous bases like adenine (A), guanine (G), cytosine(S) and thiamine (T). DNA has a double helical structure, the two separate strands are connected together by hydrogen bonds between purine and pyrimidine, and also by some base stacking interactions. Here "A" invariably pairs with "T", and "G" invariably pairs with "C. It is possible to store the world's whole data in a few grams of DNA. [5]. It is possible to get many copies of DNA by the usage of PCR (polymerase chain reaction). The durability and the long-term storage option making this to retain information for centuries. A huge amount of data can be stored in a very small space due to the high density of DNA. As in practical, the data is kept in long virtual DNA molecule but the encoding is done using short oligonucleotide strands, synthetically prepared, short strands allow to easily manipulate the data. Since data can store in DNA, it is secure as invisible to the human eye, and also, it can withstand extreme environmental conditions [8]. There are so many problems also arise with this idea-storing data in DNA-like the cost of synthesis of oligonucleotide strands, and the inability to synthesise long DNA strands. Also, there may be errors in DNA synthetic machines during the synthesize of oligonucleotides.

Related Works
The possibility of DNA data storage was executed by Clell, Risca and Bancroft in 1999. They encoded data in short strands of DNA called microdots [2]. This technology had used in World War 2 to communicate data. The next research was published by Wong et al in 2003. They made small oligonucleotides in a manner similar to the arrangement of 1s and 0s in the ASCII code of computer language code. And also, they encoded data in E.coi cells. In 2011, research led by Church and Kosuri led to a more convenient method of DNA data storage. They assigned 0s as A or C, and 1s as T or G. But it made homopolymers throughout their oligonucleotide chain that made problems during DNA synthesising and sequencing. [3]. The method used by Church had several advantages over the past proposed models. They used purely in vitro approach that avoids cloning and stability issues in in vivo approach [6]. Later in 2013, work led by Goldman and E. Bierney created small pieces of oligonucleotide strands carrying image in jpg format and a video clip in mp3 format. This is an almost error-free way of encoding data in DNA. The conducted by Zhong and team had used base 4-digit method [10].

Encoding Strategy
The encoding used in DNA nucleotides breaks them into segments. Breaking them into segments provided a redundancy up to four folds. The data can be stored in DNA by mainly five steps. nthesis III.Storage IV.Retrieval V.Decoding The data, present in the binary form, first converted to the nucleotide code, the corresponding nucleotides are synthesized by using DNA synthetic machines, and it can be stored for long time, data can be retrieved by sequencing the DNA and the sequenced material can be decoded in order to regain the data stored on DNA [8].

The Proposed System
There are so many proposed systems for storing data in DNA. One of the most prominent and the latest system is the method proposed by Goldman and et.al. They converted data, in the binary form, into a ternary format using specialized encoding format called "the improved Huffman code". This "improved Huffman code" is coding the method derived from the old "Huffman code" method, which converts data in the binary form into more compressed binary format (one method used for compression of data in the binary format in order to reduce the storage space). But in improved Huffman method, it converts data into a ternary format instead of binary format. Binary letters contain only the 1s and 2s, but in ternary form, the code is a three-number format, it contains 1s, 2s, and 0s. Goldman used this version of Huffman code and converted data in the binary format into a ternary format, the code containing only 1s, 2s, and 0s, for that he used a Huffman tree. Data compression is achieved by using a Huffman tree for encoding. No one can decode it without the original tree; this makes the data more secure. A lot of specialized equipment is needed for sequencing of DNA strand. In order to reduce data loss, there are two copies of data provided; in case of any data loss the other copy can be the alternate. Flexibility is considered as an advantage of this method; data manipulation can be done by the researcher to meet his criteria [4]. As we know, all the data can be converted to ASCII (American Standard Code for Information Interchange) format. ASCII format is nothing but a universal code used to store data in the digital format. Only data in the binary form can be transferred via digital means. ASCII code is an eight-letter binary code, which uses only 1s and 0s, and there are about 256 characters (standard) having ASCII code for each of them. Since there are eight digits in an ASCII code for a particular character, it reduces the storage capacity of a file, and it takes more space, time and energy to transfer data across the internet or other digital means. File capacity is expressed in the bit, byte, kilobyte, and megabyte form. One byte equals eight bits. One ASCII code has eight characters; hence it is a onebyte character. Researchers think about the need for a binary code which takes less space and transfers a huge amount of data. This leads to the evolution of so many coding schemes, one of which is the code formed by Huffman. Huffman had used a tree (Huffman tree) to generate the code by using the frequency of each character in the data. Goldman had used an improved version of Huffman tree to translate the data into codes, but it is not for digital purposes, he converted data into a ternary format using improved Huffman coding scheme and then into a base -3 for the format of nucleotides, and by using that he synthesized a new single-stranded DNA. The single-stranded DNA didn"t contain any nucleotide repeats since it might confuse the DNA synthetic machines [1]. Goldman created a new method strategy (in the year 2013) by modifying Church"s one bit per base system with the help of the improved base 3 Huffman coding"" (trits 0, 1 and 2). Here, he generated a triplet DNA code from the original binary code (0, 1) through a ternary code (0, 1, 2), all the four steps involved in this method is shown in the figure. Usually, binary digits have respective ASCII codes, when we convert binary to base 3 Huffman code that replaces each byte with base-3 digits (five or six). In order to avoid error created (homopolymer formation) during synthesis of DNA, each trit used here encoded with one of the three nucleotides different from the previous one used.

Encoding
Encoding is done by forming a data character"s frequency table. New nodes are created for encoding the non-repeating nucleotides of the Huffman tree. These nodes will have three sub nodes (or children) as well as the incoming weight of each generation"s parent is determined by the children"s branches weight. For example, C is considered as the leftmost child, G in middle, and T as the rightmost if the weight of the incoming branch of a parent is A. like this, G is considered as the leftmost child, T in middle, and A as the rightmost if the weight of the incoming branch of a parent is C. The whole information split into overlapping segments of 100 nucleotides with an offset of 50 nucleotides from previous and create pairs of segments starting from the first segment [8].

Fig.3. A rotating encoding to nucleotides avoids homopolymers, which will create errors
DNA strand is divided into chunks each of length 117 base pair (bp). 75 bases for each DNA information chunks were overlapped with four-fold redundancy to recover the data loss that occurred during synthesis and sequencing DNA. For the data security each redundant chunk is converted to reverse complement of the strand in every alternate chunks. Each DNA chunk is appended with data address blocks of 117 bases to determine the location of segment in overall data. One parity check bit is added for the intra file location and error detection. Total 153,335 DNA strings were generated. 33 nucleotide base primer was added to facilitate synthesis process and amplification. For details reader is referred to [Goldman et al. 2013]. As proof of concept, they used four different file types (739 kilobytes file size) and achieved 2.2 PB/g DNA storage density.

Decoding
The decoding the process is the simple reversal of the encoding. Whether the DNA belongs to the 1st or 2nd segment of the pair or whether the data is reverse complemented or not, can be identified from the first nucleotide. Remove the 1st nucleotide if it is A and the next four nucleotides will tell us about the segment number, and the next 100 nucleotides will be the data. For the confirmation of the type of segment the last nucleotide can be used. If the 1st nucleotide is C, then reverses the whole complement and then remove the first nucleotide, next four nucleotides will tell us about the segment number, the data will be the next 100 nucleotides and the last a nucleotide can be used for confirmation of the type of segment. If G represents the first nucleotide, then first reverse the whole segment and remove first nucleotide, the next four nucleotides will tell us about the segment number. Then reverse complements next 100 nucleotides, these 100 nucleotides will be the data and the last nucleotide defines the segment"s confirmation. If T represents the first nucleotide, then remove the 1st nucleotide, next four nucleotides will tell us about the segment number. Now, reverse complements next 100 nucleotides, these 100 nucleotides now will be the data, and the last nucleotide again defines the segment"s confirmation. The "TTTT" sequence will define the end of each file and after that new character will start from next nucleotide. Now the data can be converted back into the original file (Characters) by using the same Huffman tree. A single Huffman can be used for the whole data. To code different files different Huffman trees can also be used [8]. Now the DNA strands split into chunks. Each of the chunk has length approximately 117 base pairs as well as 75 bases in each DNA information chunks are overlapped with four-fold redundancy. This helps to recover data loss that will occur during synthesis and sequencing of DNA. In every alternate chunk, each redundant chunk gets converted to reverse complement of the strand, to secure data. Next, the determination of location of segments in overall data, each chunk was appended with data address blocks of 117 bases, for intra file location and error detection one parity check bit was added. Primer was added to facilitate synthesis process and amplification. Goldman & co-workers used four different file types; Shakespeare's sonnet in ASCII format, Watson & Crick's DNA double helical structure 1953 paper, a medium resolution colour photograph, a part of the audio of Martin Luther King's speech, which is in mp3 format; and they achieved 2.2 PB/g DNA storage density [9]. This will compress the whole data. The decoding process requires the original tree, without it no one can decode the data [8].

Conclusion
DNA is extremely dense and durable and thus it has the potential to be the ultimate archival storage form. By using DNA as a storage medium, storage of information in a very less space for the long-term purpose is possible. Similar to all revolutions in technology, it also has to face challenges in order to know its full potential. Since DNA has higher density, robustness, stability, and energy efficiency, it is the best archival device so far known. A minute amount of DNA can store all the information produced in the world, even though several breakthroughs will be required before it comes to the commercial sector. Future work could include compression schemes, parity checking, correcting errors in order to enhance density and safety.