extfieldsofstudy:
Physics
Computer Science

s2fieldsofstudy:
Computer Science
Mathematics

text:
Generating efficient basis sets for unbounded domains

The treatment of the problems involving unbounded domains (UDs) with vanishing boundary conditions is always challenging. For spectral methods, in particular, very limited basis sets are commonly used for such domains, in which the ranges of the decay rates with acceptable computational efficiency, are very small. Furthermore, maintaining high level of analyticity becomes burdensome. Developing efficient mapped basis tailored for such problem is one of the main strategies to overcome these difficulties. In this work, we present a technique to generate efficient basis sets for UDs. This approach allows using basis sets defined for bounded domains (BDs) for problems in UDs, and hence, providing more freedom to choose from a variety of basis sets. To ensure computational efficiency, the designed transformations cover a wide range of decay rates and allow solving integrals analytically. The method is applied to solve many differential equations encountered frequently in many physics related problems. The results illustrate the efficiency of the developed technique and mapped basis sets.

Introduction
Many physical problems involve unbounded domains (UDs) with vanishing boundary conditions. Solving such problems is challenging. In spectral methods, various basis sets are commonly used for such domain such as Hermite and sinc function. However, the ranges of decay rates in which these basis sets are efficient are very small, and hence cannot be used in approximation of functions with wide range of decay rates [1,2,3,4]. The other possibility is to transform the UD to a BD. The advantage of transformation is that we can exploit the basis sets which only applicable to the BDs. Hence, there is more freedom and flexibility in the choice of basis sets. The simplest way to convert an UD to a BD is by chopping the domain. Such method introduces a truncation error, and hence it is not a good option where higher accuracies are required. Of course the truncation error can be reduced by increasing the domain size, but at an increased computational cost. Another possibility is to convert UD to a BD through domain mapping. The main advantage of domain mapping is that there is no truncation error and a variety of basis sets which are only applicable to BDs can now be implemented for the problems involving UDs.
The idea of domain transformation may seem simple, and although there are unlimited ways to transform an UD to a BD [1], however, transforming the domain, and at the same time 2 not making the problem complex (i.e keeping integrals simple and analytically solvable) and approximating a wide range of decay rates is challenging.
In this work, we present an efficient approach to map a UD to a BD using using generalized coordinate transformation [5]. After the transformation, any basis set defined for any BD can be implemented to solve the problem. The developed method ensures orthogonality and a major portion of the problems can be solved analytically. This increases the computational efficiency and accuracy. Most importantly, the approach approximates efficiently various type of decaying functions with a wide range of decay rates by appropriate choices of mapping and basis functions. The choice of basis functions also affect the resulting integrals. Hence, using simple basis sets helps in simplifying the integrals involved, and hence increasing computational efficiency.
The basis set used in this paper is Fourier sine series sin(kπx), which is an orthogonal basis set in the domain (0,1). Hence, the UD is mapped to BD (0,1). The mapping is tested on a first order differential equation. It is shown that a wide range of decays that can be approximated efficiently and accurately.

Formulation
A transformation function can be defined as: There are infinite possibilities for such transformations [5,6,1,2]. For any orthogonal basis set {P k (u)} defined in a BD, we seek an orthogonal basis set defined in UD of the form where µ r (g(x)) is an auxiliary function to ensure the vanishing boundary conditions are satisfied.
Since we assume f k (x) is orthogonal, the following relationship must be satisfied: Coordinate transformation, will result in the following form Comparing equations (4) and (5), we get: which shows that µ(u) plays a key role in the transformations. Solving equation 6 for a unique µ r (u) will result in a unique transformation function, which can be used to map the problem from UD to BD. Symbol µ r (u) x(u)

Implementation
The proposed method can be implemented to approximate/solve any function/ODE/PDEs defined in the UDs, provided that the vanishing boundary conditions are satisfied. Here, we present a generalized case of a forced function ODE problem given by: and its approximate function/solution can be written in the form of summation of basis set f l (x) as: The matrix form of equation 7 can be written as: where L and Γ are given by: By proper choice of basis set, L can be calculated analytically, and then a can be computed, which gives the final solution.

Results and Discussions
To test the proposed method, we have chosen our complete basis set to be sine functions. However, any other proper basis set can be used. The complete basis set is given by equation 2, where P k (g(x)) = P k (u) = sin(kπu) and µ r (g(x)) = µ r (u) = sin r 2 (πu), r ∈ N 1 The transformation functions resulting from the chosen auxiliary function are given in Table 1 results are compared with the results obtained using Hermite basis to show the accuracy and efficiency of the developed method. Furthermore, for comparison purposes, the exact solution (for both case studies) is assumed to be known and is given by: where κ = 2 and α ∈ (10 −2 − 10 4 ).
The results of Case I are plotted in Figure 1 where we have computed the required number of basis functions to achieve an accuracy of 10 -10 at different decay rates (α). As can be seen from the results, Hermite is more accurate for lower decay rates whereas the developed method is more accurate for higher decay rates. However, this can be changed through proper scaling. The point of significant importance is the range of decay rates in which a method is efficient, and this is evident from the Figure 1 that the developed method covers a wider range of decay rates and hence can be implemented to physical problems which involves wide range of decaying functions. The Case II is presented to better quantify the range of decay rates that can be covered efficiently using the proposed method. For this case, we have considered a 2 nd order differential equation with constant coefficients given by: where L 2 = L 1 = L 0 = 1. Case II provides a quantitative measure of how much wider range of decay rates can be covered by a particular method. The quantitative measure used is the order of magnitude (OM). The OM is computed for N = (100, 200 and 300), and the results are given in Figure 2. As can be seen from the results, both transformation functions (T 1 and T 2 outperform Hermite for any number of basis functions. An OM of greater than 3.5 can be achieved using the transformation function T 2 . All the results indicate that the developed method is very efficient, accuate and covers a much wider range of decay rates compare to Hermite. This opens a new world of possibilities of basis sets for the unbounded domains.

Conclusion
Problems involving unbounded domains with vanishing boundary conditions are challenging because of the limited basis sets available for these domains. On the contrary, the is an abundance of basis sets for the problems in bounded domains. Therefore, we have proposed a generalized method of coordinate transformation to convert a unbounded domain to a bounded  Figure 2. N needed to achieve ε < 10 -10 for various values of α.
one without losing any information (truncation error). Furthermore, the proposed method also takes into consideration the orthogonality, the need to cover a wide range of decay rates, and solving integrals analytically. Hence, properly designed transformations can potentially unlock the doors to a new world of basis functions for unbounded domains.


extfieldsofstudy:
Computer Science
Engineering

text:
OctopusNet: A Deep Learning Segmentation Network for Multi-modal Medical Images

Deep learning models, such as the fully convolutional network (FCN), have been widely used in 3D biomedical segmentation and achieved state-of-the-art performance. Multiple modalities are often used for disease diagnosis and quantification. Two approaches are widely used in the literature to fuse multiple modalities in the segmentation networks: early-fusion (which stacks multiple modalities as different input channels) and late-fusion (which fuses the segmentation results from different modalities at the very end). These fusion methods easily suffer from the cross-modal interference caused by the input modalities which have wide variations. To address the problem, we propose a novel deep learning architecture, namely OctopusNet, to better leverage and fuse the information contained in multi-modalities. The proposed framework employs a separate encoder for each modality for feature extraction and exploits a hyper-fusion decoder to fuse the extracted features while avoiding feature explosion. We evaluate the proposed OctopusNet on two publicly available datasets, i.e. ISLES-2018 and MRBrainS-2013. The experimental results show that our framework outperforms the commonly-used feature fusion approaches and yields the state-of-the-art segmentation accuracy.

Introduction
Recent years have witnessed the rapid development of deep learning technique. Deep learning models have been widely used for medical image segmentation and achieved impressive performance [1,2,3]. Compared with natural images, medical images, e.g. computed tomography (CT) and magnetic resonance imaging (MRI), often have a lot of scanning protocols in its toolbox and each protocol may reveal a different property (often complementary to other protocols) of the underlying tissue. For examples, to assess ischemic stroke lesion, three modalities using perfusion imaging are commonly captured, i.e. cerebral blood volume (CBV), cerebral blood flow (CBF), and time to peak of the residue function (Tmax). Those modal images may contain different clinical interpretation. To exploit the multi-modal medical data, two fusion approaches are widelyused by current deep learning networks, i.e. stacking multiple modalities as different input channels (early-fusion, Fig. 1(a)) [1,2,4] and fusing the outputs of networks from different modalities (late-fusion, Fig. 1(b)) [3,5]. Neither fusion approach is optimal in using the complementary information from multiple input modalities. Take the perfusion CT for ischemic stroke lesion segmentation as an example. As shown in Fig. 1(c), four modalities, i.e. CBV, CBF and MTT, and Tmax, are captured. It can be observed from the four modalities that the lesion area in CBV and CBF is darker compared to the normal area, while it is lighter in the modalities of MTT and Tmax. Consequently, the information in different modalities may be wrongly fused if we simply adopt the early-fusion approach. On the other hand, although the late fusion approach adopts the separate encoder-decoder for each modality, the whole network is computational expensive and difficult to converge.
In this paper, we propose a novel segmentation network, namely Octopus-Net, which effectively leverages the information contained in multi-modal medical images. Instead of fusing multi-modal images at the input stage, we exploit an individual encoder for each modality and fuse the feature maps generated by middle stages of the network, which specifically extracts features from each modality and explicitly considers the correlations between different modalities. As the modalities are separately encoded, the proposed OctopusNet adopts a novel feature fusion module, namely a hyper-fusion decoder, to merge the feature maps and avoid feature explosion. Extensive comparison experiments are conducted on multi-modal datasets. The results demonstrate the outstanding segmentation performance of the proposed OctopusNet.

OctopusNet
In this section, we introduce the detailed information of our OctopusNet. The framework of our OctopusNet is shown in Fig. 1(c). The colored cubes refer to the feature maps generated at different stages of the framework. Our Octopus-Net addresses the problem of cross-modal interference, occured in early-fusion, by extracting features from the modalities using separate modal encoders, which can be any CNN architecture, e.g. VGG [6], ResNet [7] or DenseNet [8]. As shown in Fig. 1(c), the feature maps generated at different stages of modal encoders are concatenated and fed to the hyper-fusion decoder. The proposed decoder uses hyper-fusion modules to fuse the feature maps from different modalities and avoid the problem of feature explosion. Compared to the late-fusion approach with four separate decoders, the hyper-fusion decoder more effectively fuses cross-modal information and reduces the computational cost. The decoder upsamples the high-level low-resolution feature maps back to the original resolution in the same way as [9], and yields the segmentation result.

Modal encoder
As aforementioned, the modal encoder can be chosen from any widely-used network architectures, e.g. DenseNet. In our experiments, DenseNet-161 usually yields better segmentation accuracy compared to that of VGG and ResNet. Therefore, we take DenseNet-161 as an example to illustrate the pipeline of extracting feature maps from different modalities. The detailed information of network architecture of DenseNet-161 can be found in [8]. The colored cubes in each modal encoder in Fig. 1(c) are the feature maps generated by different stages of DenseNet-161, which correspond to the ones from Dense Block (1) -(4). To better leverage the explicit information contained in different modalities, all the extracted feature maps are concatenated together and fed to the hyper-fusion decoder for feature fusion and distilling.

Hyper-fusion decoder
We propose a novel hyper-fusion decoder to decode and upsample the high-level low-resolution feature maps back to the original resolution of input and yield the segmentation result. As shown in Fig. 1(c), the decoder adopts a hyper-fusion module for feature distilling, deep supervision for better training convergence, and concatenates the fused feature maps (purple cubes) to the upsampled ones (yellow cubes) to produce segmentation result.
Hyper-fusion module. As the network goes deeper, the number of feature maps increases. Consequently, the concatenation of multi-modal feature maps easily causes a problem of feature explosion. For example, assuming there are N modalities as input, the number of concatenated feature maps generated by Dense Block (4) of modal encoders is 2208 × N , which requires high cost of computation and memory consumption. The hyper-fusion module (1) -(4) in Fig. 1(c) is a 1 × 1 convolution, which has the same number of channels to that of feature maps from Dense Block (1) -(4). Therefore, the N concatenated feature maps can be accordingly fused and compacted to single ones using hyper-fusion modules (the purple cubes in Fig. 1(c)).
Deep supervision. The proposed OctopusNet is an end-to-end framework, which means the multiple modal encoders and hyper-fusion decoder are simultaneously trained and updated. However, the networks adopted for modal encoders are usually extremely deep, resulting in a difficulty for training convergence only using a single supervision signal at the very end of a long pipeline. Hence, we added a weak supervision signal to the deepest node of OctopusNet, i.e. the elongated purple cube at the bottom. Assuming DenseNet-161 is adopted as the modal encoder, the size of bottom purple cube is 2208 × 7 × 7. In this situation, a 1 × 1 convolution is used to transform the cube to 1 × 7 × 7 and the original supervision signal is resized from 224 × 224 to 7 × 7 to be as weak supervision.

Experiments
We evaluate the performance of the proposed OctopusNet on publicly available datasets from two challenges, namely ISLES-2018 3 and MRBrainS-2013 4 , and compare with the early-fusion and late-fusion approaches. Though our result is competitive to the top-performance of the ISLES-2018 challenge, the purpose of the experiments is not to win the challenges. Our main purpose is to demonstrate the effectiveness of the proposed fusion approach and compare with other widely used alternative strategies. Our approach is complementary to, and can be easily integrated into, other FCN based multi-modal segmentation approaches.

Datasets
ISLES-2018. Ischemic stroke lesions segmentation (ISLES) is a competition consecutively held since 2015 [10]. In ISLES-2018, the challenge organizer released a new dataset, which is composed of six modalities, including diffusion weighted imaging (DWI) MRI, computed tomography (CT) and four perfusion scans, i.e. mean transit time (MTT), time to peak of the residue function (Tmax), cerebral blood flow (CBF) and cerebral blood volume (CBV). The ISLES-2018 competition provides 94 sets of multi-modal data for training and 62 sets for test. As the ground truth of the test set is not available, participants need to submit their prediction to the online system for performance evaluation.
The dataset has two main challenging issues. First, the appearances of lesion areas among different modalities are widely varied. The lesion area in MTT and Tmax is brighter than the normal area, while it is dark in the modalities of CBF and CBV, as shown in Fig. 1(c). Second, the ISLES-2018 data does not have a uniform size. Though each slice has a fixed size of 256 × 256 pixels, the number of slices contained in a volume varies from 2 to 22. Most ISLES-2018 volumes only have two slices, which presents a difficulty to adapt a 3D segmentation framework to the dataset.
MRBrainS-2013. The MRBrainS-2013 dataset contains five sets of multimodal brain images, in which the brain tissues, i.e. gray matter, white matter and cerebrospinal fluid, are fully annotated. Three registered modalities, i.e. T1-weighted scan (T1), T1-weighted inversion recovery scan (T1 IR) and T2-weighted fluid attenuated inversion recovery scan (T2 FLAIR) are provided. The volumes of the dataset are in an uniform size of 240 × 240 × 48 voxels. Implementation details. The proposed OctopusNet may have different architectures regarding to the input data. As most ISLES-2018 data has a couple of slices, we develop a 2.5D OctopusNet instead of 3D. Three consecutive slices from a volume are extracted and fed to 2D modal encoders as inputs. The first and last slices of the volume are duplicated for padding. In this setting, modal encoders can be pretrained on the ImageNet dataset for better training convergence. Our OctopusNet is implemented using PyTorch. The initial learning rate is set to 0.7 and divided by 10 after every 35 epochs. The network is optimized by stochastic gradient descent (SGD). The used datasets have different number of modalities. Consequently, the proposed OctopusNet involves different numbers of modal encoders for the ISLES-2018 and MRBrainS-2013.
Take the ISLES-2018 as an example. As the ISLES-2018 test set does not provide the DWI modality, five original modalities, i.e. CT, MTT, Tmax, CBF and CBV, are adopted as input for OctopusNet. Furthermore, the lesion area is not clearly visible in the modalities of CT, CBF and CBV. Hence, these three modalities are enhanced by histogram equalization. Finally, five original modalities and three enhanced modalities are fed to the OctopusNet 5 . The input size of each modal encoder is 256 × 256 × 3.

Performance Analysis
We perform a five-fold cross validation on the ISLES-2018 and MRBrainS-2013 training set to evaluate the performance of our OctopusNet. All the experiments are repeated three times to reduce the influence caused by random nature of network training. Hence, the results reported in the paper are the the average results of three repeated experiments. For the convenience of comparison, the frameworks using baseline fusion approaches (early-and late-fusion) in our experiments are in the same setting to that of OctopusNet, e.g. the input format of different modalities. Henceforth, the fusion approach adopted in our Octopus-Net is named as Octopus-fusion. The Dice coefficient, which measures the spatial overlap index between the segmentation results and ground truths, is adopted as the metric to evlauate the segmentation accuracy.
Results on ISLES-2018. As aforementioned, the modal encoder can be chosen from widely used deep learning networks. To evaluate the generalization capability of Octopus-fusion, several network architectures, e.g. VGG-16 [6], ResNet-50 [7] and Dense-Net-161 [8], are adopted as the modal encoder and trained with different fusion approaches on the ISLES-2018 dataset. The results are listed in Table 1. To evaluate the improvement produced by the usage of multi-modal images, we also report segmentation accuracy using a single modality. Due to the space limit, Table 1 only lists the result of the best single modality (i.e., Tmax).
Due to the lack of information contained in extra modalities, the frameworks using single modality only yield Dice coefficients around 44%, which are about 9% lower than that of multi-modal frameworks. For the early-fusion approach, Table 1 shows that the accuracies of all three backbone networks are quite similar with the deep networks (i.e., DenseNet-161 and ResNet-50) slightly outperforming the shallow network of VGG-16. For the late-fusion approach, as it involves multiple encoder-decoder architectures for different modalities, the explicit information contained in multi-modal data can be better extracted. Hence, accuracy of the late-fusion approach surpasses that of early-fusion with the same modal encoders. However, DenseNet-161 only gains marginal improvement, i.e. 0.04%, by switching from early-fusion to late-fusion. The reason for that is the network depth of DenseNet-161 is extremely deep, which makes it difficult to simultaneously well train multiple fully convolutional DenseNet-161 branches in the late-fusion approach. Compared to the results of early-and late-fusion, our Octopus-fusion approach significantly boosts the accuracy of modal encoders. The best lesion segmentation result is achieved by the Octopus-fusion DenseNet-161, i.e. a Dice of 57.72%. By adding the deep supervision signal, the segmentation accuracy is further increased to 57.90%, which is 2.51% higher than that of the best-performance among benchmarking algorithms (late-fusion with ResNet-50). ISLES-2018 Challenge. We participated the ISLES-2018 competition. The proposed OctopusNet using DenseNet-161 achieved an average Dice of 48%, which ranked the third-place of ISLES-2018 challenge 6 . We notice that, for all participating teams, there is a gap between validation and test accuracy. One possible reason is that the test set contains more small lesions, where are difficult to segment accurately for all algorithms. Additionally, the top approaches reported that they used extra modalities, e.g. 4D perfusion CT (ranked 1 st with Dice of 51%) and synthesized DWI (ranked 2 nd with Dice of 49%), which were not adopted by our OctopusNet.
Results on MRBrainS-2013. We also conduct experiments on MRBrainS-2013 to compare the performances of different fusion approaches for the task of brain tissue segmentation. The three original modalities of MRBrainS-2013 are directly employed as input for the proposed OctopusNet. The best-performer on ISLES-2018, i.e. DenseNet-161, is adopted as the backbone of modal encoder. The input size of each modal encoder is 240 × 240 × 3. The Dice coefficients for different tissues, including CSF, gray matter and white matter, produced by different fusion approaches are listed in Table 2. The average Dice (Ave. Dice) is calculated by averaging the Dice coefficients of three tissues.
The framework using single modality is also evaluated for comparison. It is interesting to see that, for the gray matter and white matter, the best single modality (T1) produces even higher segmentation accuracy than early-fusion. The reason for that may be the physicians mark the annotation of the gray and white matter primarily using the T1 scans, while the T1 IR and T2 FLAIR scans usually provide additional information for the annotation of outer border of CSF and white matter lesion, respectively. Therefore, most information contained in the extra modalities, i.e. T1 IR and T2 FLAIR, may be seen as noises for the brain tissue segmentation. The late-fusion approach yields similar average segmentation accuracy to using T1 only (81.76% vs. 81.84%). The rea-son for that may be the post-fusion approach performs information fusion too late; Therefore, it can not fully utilize the complementary information among multiple modalities. Oppositely, by using the proposed Octopus-fusion, the average segmentation accuracy increases to 82.92% and improvement is observed for all tissues, which illustrates that our Octopus-fusion can effectively extract useful information from each modality and prevent the cross-modal interference caused by irrelevant information. An additional observation is that CSF consistently benefits from multi-modality fusion using any fusion strategy, which is concordant to the annotation process of physicians. Again, Octopus-fusion achieves the largest boost in segmentation accuracy of CSF, i.e., +2.13%.

Conclusion
In this paper, we presented a novel deep learning network architecture, namely OctopusNet, for multi-modal medical image segmentation. The proposed Octo-pusNet adopted a separate modal encoder for each modality to explicitly extract features and a hyper-fusion decoder to fuse the features, avoiding the problem of feature explosion. The proposed OctopusNet was evaluated on two publicly available datasets. The experimental results demonstrated that our OctopusNet was a general network architecture, which can provide excellent performance for various segmentation tasks of multi-modal medical data.