A generalization of Possibilistic Fuzzy C-Means Method for Statistical Clustering of Data

— The Fuzzy C-means (FCM) algorithm has been widely used in the field of clustering and classification but has encountered difficulties with noisy data and outliers. Other versions of algorithms related to possibilistic theory have given good results, such as Fuzzy C- Means(FCM), possibilistic C-means (PCM), Fuzzy possibilistic C-means (FPCM) and possibilistic fuzzy C-Means algorithm (PFCM).This last algorithm works effectively in some environments but encountered more shortcomings with noisy databases. To solve this problem, we propose in this manuscript, a new algorithm named Improved Possibilistic Fuzzy C-Means (ImPFCM) by combining the PFCM algorithm with a very powerful statistical method. The properties of this new ImPFCM algorithm show that it is not only applicable on clusters of spherical shapes, but also on clusters of different sizes and densities. The results of the comparative study with very recent algorithms indicate the performance and the superiority of the proposed approach to easily group the datasets in a large-dimensional space and to use not only the Euclidean distance but more sophisticated standards norms, capable to deal with much more complicated problems. On the other hand, we have demonstrated that the ImPFCM algorithm is also capable of detecting the cluster center with high accuracy and performing satisfactorily in multiple environments with noisy data and outliers.

I. INTRODUCTION
o classify the data in different groups according to one or more specific criteria, several methods have been adopted to solve this problem, Clustering is one of them, this method uses in most cases the Euclidean distance. There are 2 types of Clustering algorithms, rigid that is based on one object is in a cluster or not with a degree of membership equal to 0 or 1 [1], [2], this type of algorithm has shown its inefficiency in the case of overlapping of two clusters or, in particular, for points belonging to several clusters at the same time, we cite for example the K-Means algorithm. The other type of clustering is Fuzzy Clustering, which is more efficient method, widely used in several fields such as pattern recognition, image processing, also applicable in the security field for face and fingerprint detection, etc..., the principle of this method is simple: several data can belong to different clusters based on multiple membership values from interval [0,1],[3], [4]. Therefore, over the years, many algorithms appeared; the most used being the Fuzzy C-Means (FCM) algorithm [5]. This algorithm was first developed to deal with relational data, derived from Euclidean and other non-Euclidean distances. [6]. FCM algorithm is effective in the case of spherical clusters [7]. However, given the probabilistic type constraint used in FCM, this algorithm encounters the problem of sensitivity to noise and outliers [8], [9]. To correct this weakness, a n ew algorithm called possibilistic C-Means (PCM) which is based on the possibilistic approach was proposed [10]. Also, the PCM algorithm allows a better treatment of noisy data and outliers. Despite its efficiency, the PCM algorithm has a weak point in its sensitivity of initializations and choices of typing parameters and it can sometimes generate coincident clusters [11], another drawback of this algorithm is that it considers the typicities values but neglects memberships values which is also important as a parameter. Other versions of the PCM algorithm have been improved in order to correct some difficulties encountered in the PCM by modifying the objective function [12], [13], [14], [15]. Although PCM only uses typicalities values, the researchers tried conserving the highlights of FCM and those of PCM while mentioning the importance of membership and typicality values. [16], [17]. Hence, a new complementary model called the Fuzzy possibilistic C-Means (FPCM) clustering algorithm have been proposed, which merges the characteristics of the two algorithms FCM and PCM and allows to optimize them [18].
This algorithm has solved some of the problems encountered in the case of FCM and PCM, but like any algorithm, it has its strengths and also its vulnerabilities. The problem with FPCM is that it i mposes a constraint on the typicality values, and also that the possibilistics values are very small when the size of the data set increases, which will lead to the development of a new and more powerful algorithm, called Possibilistic Fuzzy-C Means (PFCM). This new model was first proposed by [16], [19], [20], [21], [22] to simultaneously produce adhesions and typicities, the PFCM is a hybrid combination of the 2 objective functions of the FCM and PCM algorithms, this algorithm has strong points, overcoming the difficulties encountered in FCM, PCM and FPCM, it h as solved the major problem of noise sensitivity encountered in the use of FCM, and that of overlap and coincidence of Clusters, which is the main problem of PCM. The PFCM algorithm was also found to be less sensitive to outliers. Since noise data influence the estimation of centroid, the PFCM algorithm simultaneously creates adhesions and typicities for each cluster with usual prototypes or cluster centers [14], [24], [25], [26].
Although the FCM, PCM, FPCM and PFCM algorithms are based on the Euclidean norm, the problem of using other more efficient norms occurs, for example the covariance norm, which creates ellipsoidal clusters that give better results with models and data structures, then a further problem is encountered, that of compactness, which can lead to the loss of several important data for clusters, such as efficient processing of noisy data or clustering of data with clusters of different sizes [27], [28], [29].
Another problem occurred with these algorithms when using the covariance norm is that FCM, PCM, FPCM and PFCM are unable to provide accurate prototypes even with their own data, in this work, we proposed a new more improved algorithm called ImPFCM (Improved possibilistic uzzy C-Means) to overcome these problems [30].
The ImPFCM algorithm uses functions from the norm; it is more efficient to handle complicated cases encountered as an obstacle for PFCM, especially in the case of noisy data. Hence, ImPFCM algorithm finds accurate cluster centers, its objective function uses norm functions, and it i s flexible, efficient, and more suitable to different clustering concepts and constraints encountered in the above-mentioned algorithms, it is an algorithm capable of using the covariance norm with convenience.
This new algorithm gives a good solution to many of the most problems encountered with other previously mentioned algorithms.
After the introduction, the manuscript is exposed as follow: we start with the description of many clustering algorithms used frequently in clustering theory in sections II, we will present, our proposed improved ImPFCM algorithm, in section III, Finally, we close this manuscript by section IV with some experiments and results of ImPFCM algorithm and conclusion.

II. LITERATURE
In this section, we review the classical existing algorithms used to clustering data we focus attention of our studies to noisy environment with outliers.

A. Fuzzy C-Means algorithm (FCM)
The FCM algorithm is based on the principle of assigning memberships to that are inversely related to the relative distance from to points prototypes these represent the cluster centers in the FCM model. Before the development of the FCM, many algorithms were used such as k-means and C-means, when running these algorithms, the major problem encountered was the treatment of noise and outlying points which can be explained as follows: For two prototypes having the same distance from the center (equidistant), the value of belonging to each Cluster will be identical, whatever the absolute value of the distance between the two centroids (as well as between the other points of the data). Thus, two distant points which belong to the noises but which are the same distance from the Cluster centers, are selected to have equal membership values in the two Clusters, whereas in reality these two points have a very low membership that cancels each other out in relation to one of the Clusters.
In the case of the C -means algorithm, the membership of each data point to all classes is 1, which makes it suffer from the problem of noise sensitivity.
Thus the theory of fuzzy clustering was able to partially solve these problems. The FCM algorithm is derived from the following optimization problem: Minimize: Where: : is number of data vectors, : is number of clusters, : is center of the th i cluster, : is dimension of the data, : is data vector ( column of the data matrix N r X × ), are also the data centers, is the degree of fuzziness, is a norm matrix. This algorithm is characterized by a matrix called the dimensional fuzzy partition matrix , composed of the elements that represent the degree of belonging to the model to each Cluster. is degree of fuzziness, it measures the efficiency of the FCM algorithm on clustering performance [27].
The FCM algorithm has a very special mode of operation when compared to other partitioning algorithms, it works independently of the number of clusters existing in the data set, the FCM algorithm finds a fuzzy partition in a particular set of data. Furthermore, in the FCM algorithm, the sum of each column in the membership matrix U must be equal to 1, this is a main constraint which is the key element for this algorithm that characterizes it to other Clustering algorithms such as the case of C-means and K-means. The FCM algorithm uses data point belongings that are related to the distance of the data point from the cluster centers. If a data point is at the same distance from the clusters, it will have the same membership value for each cluster. However, to deal with noise and outliers, the FCM algorithm is not the right algorithm, since the existence of at least one outlier can completely affect the result of partitioning in the FCM algorithm [8], which are the weak points of the FCM so in this case, the FCM does not differentiate between noise points or outliers which are also taken into account in the membership values which could cause a big problem later on which influences the final result obtained by the FCM. The second problem is that the FCM algorithm only detects spherical clusters. For non-spherical clusters, the FCM becomes inefficient [9].

B. 6 B Possibilistic C-Means algorithm (PCM)
To overcome and correct this weakness FCM algorithm, a new algorithm called PCM based sur possibilistic approach has been proposed [10] and which improves the column sum constraint is equal to 1.

With constraint
In other words, each element of the th column can be any number between 0 and 1, provided that at least one of them is positive. Therefore, the PCM has relaxed this constraint of the FCM and has succeeded in solving the problem of noise sensitivity. However, PCM tends to generate coincident clusters and is very sensitive to initializations [20], [21], [22]. The PCM algorithm is characterized by the degree of typicality with respect to the cluster which is more accurate when compared to the membership values interpreted by the FCM.
As the values of typicality with respect to one group do not depend on any of the prototypes of the other clusters [23], the degrees of typicality have been defined to solve this problem, by constructing prototypes characterizing the subcategories of data, taking into account the particularities of the points with respect to the other categories and also the similarities of the members of the category, therefore, the degree of typicality is an effective means of distinguishing the moderately atypical member of the group from the very atypical member. When compared to the FCM, the PCM algorithm relaxes the line sum constraint of the FCM algorithm, the PCM algorithm is characterized by a main constraint which is expressed as follows: each membership value in can be between 0 and 1 or equal to one of them [24], i.e., . These values are referred to as the data point types in each group. Thus, the objective function of the PCM algorithm can be formulated as follows: With: : the total number of models in a data set, : is the number of Clusters, : The parameter that defines the fuzziness degree of the partition, : the distance which can be Euclidean or not, = : represents the fuzzy partition of the matrix , : the typicity parameter estimated from the data. It is calculated as follows: With: : is the total number of models in a data set is a parameter that defines the degree of Fuziness of the partition; are the characteristics of the data; represent the Cluster's centroid; ] is a fuzzy matrix partition composed of the degrees of membership of the object of each cluster .
For the PCM algorithm, the membership value will be calculated from the following equation: is the distance, is the parameter of typicity. In the case of the PCM algorithm, the value should be interpreted as the typicity of with respect to the cluster (rather than its cluster membership). For PCM, each line can be interpreted as a possibility of distribution on X. The PCM algorithm helps to identify outliers and noisy data points. Although the PCM is efficient in dealing with noise, it has a drawback with its sensitivity of initializations and typing parameter choices and can generate coincident clusters [11]. Also, in the PCM algorithm, clusters do n ot have too much mobility because each data point is classified as a single set at a time, so the PCM produces inaccurate cluster centers when clusters are not of the same size or when the covariance standard is used.
In this way, as it has been observed, several disadvantage exist at the level of the 2 algorithms FCM and PCM, to overcome their weak points, the strong points of the FCM and PCM have been restored, given the importance of the values of typicity and membership, a new model called the Fuzzy possibilistic C-means Clustering FPCM algorithm has been proposed [17] which merges the characteristics of the two algorithms FCM and PCM and allows to optimize them.

C. 7 B Fuzzy Possibilistic C-Means algorithm (FPCM)
The objective function of the FPCM also contains membership and typicality values; however, they are represented as follows in equation (5): Subject to: Where m and η are the coefficients of Fuzziness and Typicity respectively. According to the constraints 6, 7, 8 and the conditions of optimization of the , considering the initial or extreme conditions of and using Lagrange multipliers we obtain the following equations: After running the FPCM algorithm, it was noted that the main problem with the FPCM algorithm is the constraint that corresponds to the sum of all typicality values of all data in the cluster, especially for a large data set [19]. Compared to the FCM, the FPCM algorithm has the same singularity values as the FCM, on the other hand compared to the PCM, the FPCM does not have the sensitivity problem that is a weak point of the PCM, but when the number of data is large, the typicality values (10) will be too low. Thus, typicality values should be scaled up, as in the case of FCM and PCM [25], [26]. Hence, scaling seems to be a p alliative way to solve the problem of small values (which is caused by the line sum constraint on T, since scaled values do not have any additional information on data points. Thus, scaling is a good way to correct a m athematical flaw in the FPCM. This FPCM algorithm is unreliable and also has some defects such as FCM and PCM, to overcome this problem a new algorithm has been introduced called Possibilistic Fuzzy C-Mean. The problem with the FPCM is that it imposes a constraint on the typicities values from the moment when the sum of the typicities values on all data points of a particular cluster is 1, this constraint is relaxed on the typicity values placed normally on the row while keeping the constraint on the membership values placed on the column [27].

D. 8 B Possibilistic Fuzzy C-Means algorithm (PFCM)
The PFCM algorithm was initially proposed by [19], it is an algorithm that has some strong points that overcome the difficulties encountered by FCM, PCM and FPCM algorithms. Principally, the PFCM solves the problem of cluster overlap, it should also be noted that this algorithm is less sensitive to outliers. PFCM is a hybrid combination of the two objective functions of the PCM and FCM algorithms. The objective function of the PFCM is as follows: Under the constraints The coefficients et are weighting values that determine the importance of the typicity values and the membership values N is the total number of the data set, : is the number of Clusters, The coefficients are constants that respectively define the importance of the membership and typicities values in the objective function.
, and are constants defined according to the problem by the user.
is the matrix of typicity values, is considered as an assignment of the typicity of objects in Cluster.
is the matrix of fuzzy partition. is the value of the fuzzy membership function for sample belonging to the Cluster. is the set of data. S: the space dimension.
is the matrix of cluster centers : is the cluster center.
is the distance between and the center of Cluster .
The membership values have the same meaning as those used in the FCM algorithm. Similarly, the values of typicity have the same interpretations as those defined in the PCM model. The constants must respects the following constraint: , and establish the importance of the membership value and the typicity value , Therefore, if we reduce the importance (weight) of the membership value this necessarily forces us to reduce the importance of typicity to the same extent, it is also restrictive. Moreover, to guarantee optimal typicity it will depend on the importance of the value of b. So, by restricting , the flexibility of the model is compromised. If et for every , then equation (12) is reduced to a FCM optimization problem, on the other hand if , then in this case we will be dealing with a PCM optimization problem. We will see further on that if even if we don't set for all , (12) becomes implicitly equivalent to the FCM model. As the FPCM, placed under the regular conditions of C-means optimization problems, we obtain the first order of the necessary conditions for the extrema of J, if we put for any , and contains at least c distinct data points, then minimize if and only if: The FCM, PCM, FPCM and PFCM algorithms are well adapted models to the Euclidean norm, on the other hand the covariance norm creates ellipsoidal clusters which give better results with the models and data structures. Another problem for the Euclidean norm occurs, when the units of the data rows are different, which shows that the Euclidean norm is not well adapted, whereas the covariance norm gives better results [28], [29], [30].
A second problem occurs in the maximum compactness obtained by minimizing the equations (2) and (12) to obtain a simple and linear update equation for the calculation of prototypes et , if we use instead of this requires the resolution of a non-linear equation for the prototypes, however if we use this can lead to the loss of several important cluster data in the case of processing noisy data or clustering data with clusters of different sizes [28]. Other problems encountered with these algorithms when using the covariance norm is that FCM, PCM, FPCM and PFCM are not able to provide accurate prototypes even with clean data, in this paper, we propose a new more general algorithm called ImPFCM to solve some of this shortcomings

III. METHODOLOGY
To make the PFCM more general and efficient, we proposed a new algorithm called Improved Possibilistic Fuzzy C -Means ImPFCM this algorithm is well performing.
The objective function of the IMPFCM is as follows: under the constraints The objective function of the algorithm ImPFCM is almost the same as the one presented in the case of the PFCM, the only existing difference is the replacement of the term in equation (12) by the two terms ) and ).
As we remark, our proposed ImPFCM objective function algorithm is flexible and more scalable efficaciously to the different concepts and constraints of clustering encountered in the previously mentioned algorithms, and which is also able to easily use another norm, for example, covariance norm.
The optimization of equation (16) is performed using Lagrange multipliers for which the following function is optimized. Using Lagrange multipliers, we optimize equation (16) and obtain the following optimized function: The value of the condition is designed to prevent large clusters from pulling the centers of smaller ones, which is discussed in detail in [28], so we calculate the cluster centers as follows: The ImPFCM algorithm converges when with: : index of the iteration : is a predefined threshold.
The membership values and the typicity values are calculated as follows: is calculated from the following derivation: (22) is calculated from the following derivation: IMPFCM is initialized by , , which are calculated from the PFCM algorithm. In the case of the PFCM, based on the equation (15) if , we will not be able to calculate the membership value neither the typicality value , which in our case will be indeterminate values. On the other hand for the IMPFCM algorithm this problem does not occur, so if we impose the condition , in (22), (23) then the values and will always be well determined. In equation (23) if indeed , then , therefore the term in equation (16) will be equal to 0 and equation (16) will become expressed in terms of the Euclidean norm and will have the following form : under contraints Since the identity norm matrix (Euclidean distance) is inappropriate for data sets where the lines have different physical units and interpretations, then as we have already noticed, this equation will subsequently pose a problem in the case cited. This leads us to use the covariance norm matrix in the ImPFCM. For optimal results, the matrix covariance norm was used which gives a dimensionless distance. The ImPFCM algorithm is based on FCM and PFCM in it's initialization and execution, so the steps of ImPFCM are as follows: 1-The FCM is applied. 2-and are calculated to initialize PFCM. 3-After the V's and 's obtained from the PFCM that will be used to initialize the ImPFCM to obtain the most accurate cluster centers. 4-et are calculated from (22) and (23) considering Once these matrices are computed, we notice that they are less sensitive to noise points than those obtained in the PFCM case from (15) because the noise contributions on the cluster centers are reduced using (17).
In other hand, in the case of PFCM, the minimization of the equation gives us a greater compactness, the clustering will be impacted by the noise which influences the quality of the results obtained. Therefore, the functions et help to overcome this problem, they are effective as long as they allow to dampen the noise on the prototypes. The exponential function where is the characteristic width of the Cluster, very appropriate in the case of high noise or distant points. The choice of the functions and influences the convergence of equation (16), that's right why we have to choose them well and avoid functions with low convergence rate which give a too high computation time and wrong results. However, there are other functions that have also shown their ability to attenuate the impact of noise on cluster centers and that can be used instead of the exponential function, for example: In order to avoid high computational time, it should be noted that the type of functions used affects the convergence of equations (16)- (17) and functions with low convergence rates should be avoided for the reason cited above.

A. Comparative study of FCM, PFCM, ImPFCM and algorithms
In this section, we focus on the experimental analysis and testing ImPFCM performance algorithm, we need to compare it with FCM and PFCM algorithms using simple dataset containing noise and taken from [19] and presented as following in Table I: The six datasets are presented as below in Fig. 1. With various ratios of noise points to the actual data points: Fig. 1 The Six Datasets used for testing proposed clustering algorithms To compare the performance of the three algorithms FCM, PFCM and ImPFCM, in terms of exactness to detect more precisely the clusters centers, we perform a clustering on these 6 datasets, we start with data 1. Table II  and Table III, we can deduce that FCM algorithm is not efficient to find the cluster center exactly, because it detects the approximate location of the prototypes near the potential clusters, while the PFCM algorithm reduces the noise effects and finds the approximate clusters centers, the ImPFCM method localizes the centers precisely compared to FCM and PFCM algorithms, Fig. 2. According to the results provided in Table IV and Table V, we can observe that the ImPFCM algorithm successfully detects the cluster centers accurately, while the PFCM algorithm performs well then FCM algorithm, but the ImPFCM algorithm performs satisfactorily despite the noisy environment. Fig. 3. Results of clustering DATA 2 using FCM, PFCM and ImPFCM, c=4, m=2 In data 2 clustering results, from Table IV, Table V and Fig.3, it's clear that ImPFCM algorithm outperformed in comparison with FCM and PFCM algorithms and the ImPFCM error is 88% smaller than FCM error. In addition, the ImPFCM error is 13% smaller than PFCM error. *For Data 3: Cluster centers and FCM, PFCM, ImPFCM algorithms performances for data 3 are presented Hence, for data 3, with a number of cluster centers equal to 6, and from the obtained tests results in Table.VI, Table VII, Fig.4, we can conclude that the ImPFCM algorithm is more performing to identifying exactly the cluster centers versus the FCM and PFCM algorithm, and, the ImPFCM error is 87% less than that of FCM and 19% less than PFCM error. *For Data5: FCM, PFCM, ImPFCM algorithms Cluster centers and performances are presented in following Table IIX  and Table IX. Fig.5.

PFCM ImPFCM
Once again with six clusters, we observed that DATA 5 is clustered Fig. V. When using the ImPFCM algorithm, we notice that the cluster centres found are identical to the actual centres, Table IIX, Table IX, thus the ImPFCM accurately detected all six cluster centeres. Thereby, the ImPFCM again demonstrates superiority and performance over both the FCM and PFCM algorithms. As we remark, the ImPFCM error is 92% less than that of FCM, and 25% less than PFCM error.  Fig. 4 Result of clustering DATA 3 using FCM, PFCM and ImPFCM, c=6, m=2 for data 3 Table IIX. Matrix of initial and FCM, PFCM, ImPFCM cluster centers for data 5 In order to investigate the ImPFCM algorithm performance in various environments, we scope the execution of the ImPFCM algorithm to other types of databases, and accordingly, we perform clustering on 6 clean and noiseless datasets and 13 real datasets using the FCM, PFCM and ImPFCM algorithms and then we compare them in both number of iterations and execution time, results are presented in following (Fig.6, Table  X): Fig. 6 Six synthetic data sets with no noise and outliers In a further extension of the study to other types of databases, the FCM, PFCM and ImPFCM algorithms were applied to 25 synthetic and real datasets and compared in terms of execution time and number of iterations, as detailed below.
-Data sets 1 to 6 are the six synthetic noisy data sets studied above and shown in Fig. 1, Fig.3, Fig.4, and Fig.5.
-Data sets 7 to 19 are real data taken from the UCI Machine Learning Repository.
-Data sets 20 to 25 are the synthetic noise-free shown in Fig. 6 that contain different numbers of clusters to study the algorithms performance when there are large numbers of clusters in the data.
To examine the reliability of our ImPFCM algorithm, we started by measuring the efficiency of ImPFCM with thirteen real-world datasets with a known number of clusters, and then including previously known datasets such as: Climate Model Simulation Crashes, Connectionist Bench, Glass ID, Energy Efficiency, Fertility, Glass ID, Haberman Survival, Cleveland Heart Disease, Ionosphere, IRIS, Pima Indian Diabetes, Seeds, Wine, and Wisconsin Prognostic Breast Cancer, which are all taken from the UCI Machine Learning Repository. In the case of real datasets, distributed between 7 and 19, and due to outliers, the FCM algorithm remains unable to detect cluster centres characterised by their large size, but in this case the PFCM finds dense clusters, but the ImPFCM algorithm is more efficient on this point compared to the PFCM algorithm, so it can detect large clusters with higher densities and presents remarkable accuracy results compared to the PFCM and FCM algorithms. In another hand, there is no noise or outliers in the 20-25 datasets and their clusters have the same sizes. Therefore, the ImPFCM algorithm easily finds the actual cluster centers of these sets and also iterates only twice because the datasets are clean, and due to the interactions between the clusters Fig.VI, it corrects the insignificant displacements of the cluster centers calculated by the PFCM algorithm. Based on the results in Table X, it deduced that the PFCM and FCM algorithms require relatively less execution time and iterations than the ImPFCM algorithm, this is due to the projection into another space and the non-linear cluster center update equation, two major reasons that influence the execution time of the ImPFCM algorithm, On the other hand, according to what it noticed, sometimes, even if there is no noise or outliers in the data, FCM and PFCM algorithms remain limited in this case and cannot find the exact cluster centers due to the interactions between clusters, ImPFCM algorithm limits or cancels these interactions due to function introduced in the objective function and finds the cluster centers precisely. Hence, we can deduce that ImPFCM algorithm detects the cluster centers more precisely. We can deduce that, when using ImPFCM algorithm, although we lost some time but we gain in terms of accuracy.
B. The impact of Cluster sizes and density to ImPFCM algorithm The quality of the clustering is indicated by the density, so a good clustering is equivalent to a high density which means that the cluster centers calculated by the algorithm are located in dense regions of the data. Therefore, we have a h igher density rho, when there are many data points around each cluster center, which is expressed using the following equation.
In order to evaluate the impact of different cluster sizes on the cluster centers, we notice that the cluster size influences the location of the cluster centers. Thus, when the cluster sizes are varied, the cluster centers are misplaced, and this even in noise-free datasets, because the larger clusters attract the cluster centers to their side. For this reason, if we experiment with the FCM, PFCM, and ImPFCM algorithms on a noisefree dataset composed of two clusters of different sizes as shown in Fig.7, we notice that, the FCM algorithm places the two cluster centers in the larger cluster. On the other hand, the PFCM algorithm identifies the center of the largest cluster accurately and the center of the smallest one as well, the ImPFCM algorithm, is initialized by the cluster centers computed by the PFCM algorithm, this algorithm also shows its efficiency in this point, cancelling the mutual interactions of the clusters and finding the real location of the cluster c.

C. Computational cost of the ImPFCM algorithm
Although the ImPFCM algorithm shows its performance and effectiveness to detect precisely cluster centers, this algorithm in its execution, is greedy in terms of consumption of execution time, and convergence, and finding the precise center of the clusters requiring a higher computational cost compared to the FCM algorithm, and this is due to the nonlinear nature of the equations contained in the objective function and their update, the high execution time of the ImPFCM algorithm compared to FCM, is caused by the complex and excessive computations for convergence, which is not the case for FCM and also because the functions are linear and do not require additional calculations. On the other hand, compared to algorithm PFCM, we notice that for testing FCM , PFCM and ImPFCM algorithms to some data set for example chosen Data 1 to data 6, it is noted that when the program start and the number of iteration was higher in threshold 7, Fig.8. the ImPFCM algorithm has relatively a small error of convergence compared to PFCM algorithm that explain a high precision to find exactly a real clusters centers, finding accurate cluster centers in presence of noise and outliers and finding accurate cluster centers when sizes of the clusters are considerably different, and this is due to using statistical method, so we can deduce that the ImPFCM algorithm improves PFCM and FCM algorithms and works satisfactorily. In summary, the main contribution of the present paper is to improve performance of the well-known FCM and PFCM algorithms by proposing the ImPFCM algorithm. As mentioned in this manuscript and in other researches done previously, the existence of noise in the data, the precise determination of the center of the clusters, the outliers, the density, the different sizes of the clusters, the interactions between the clusters, have always constituted great obstacles for the clustering method, Therefore, several algorithms have been developed to solve these problems, for example as mentioned in this paper among others the algorithms FCM, PCM, FPCM, PFCM these last ones have not been able to determine the real centers of the clusters, and have not given satisfactory results, So in order to overcome these difficulties, a new algorithm named ImPFCM has been proposed which represent an improved version of PFCM, subsequently, it was also shown in this work that the proposed ImPFCM algorithm is more efficient and accurate, when the data is noisy and can accurately calculate the real centers of the clusters when the size of the clusters is significantly different and when there are interactions between these clusters, and, In particular, the ImPFCM algorithm finds the dense regions of the data more precisely when compared to the FCM and PFCM algorithms, which is indicated by the density of the clustering results, and it converges faster than the ImPFCM algorithm. Furthermore, the results of the comparative study algorithms indicates the performance and efficiency of ImPFCM algorithm to easily cluster the data sets in a space with a high dimension and to use not only Euclidean distance but more sophisticated norms able to deal with much more complicated problems. On the other hand, the ImPFCM algorithm performs satisfactorily even when the covariance norm matrix is used, in which case the FCM, PCM, PFCM algorithms fails to find accurate prototypes even in clean data, while the ImPFCM is insensitive to the cluster size and the type of covariance norm matrix, and work effectively in different environments with noisy data and outliers.
1 2 B Acknowledgment The author gratefully acknowledge the all members of scientific team which participated in this research.


The Cross-Lingual Arabic Information REtrieval (CLAIRE) System

Despite advances in neural machine translation, cross-lingual retrieval tasks in which queries and documents live in different natural language spaces remain challenging. Although neural translation models may provide an intuitive approach to tackle the cross-lingual problem, their resource-consuming training and advanced model structures may complicate the overall retrieval pipeline and reduce users engagement. In this paper, we build our end-to-end Cross-Lingual Arabic Information REtrieval (CLAIRE) system based on the cross-lingual word embedding where searchers are assumed to have a passable passive understanding of Arabic and various supporting information in English is provided to aid retrieval experience. The proposed system has three major advantages: (1) The usage of English-Arabic word embedding simplifies the overall pipeline and avoids the potential mistakes caused by machine translation. (2) Our CLAIRE system can incorporate arbitrary word embedding-based neural retrieval models without structural modification. (3) Early empirical results on an Arabic news collection show promising performance.

INTRODUCTION
In the era of big data, information retrieval systems oer a convenient and automatic approach to retrieve target information for users on a large scale. Usually, queries and documents stem from the same language. This single-language assumption stands in direct conict with the ambition of connecting users with as broad as possible a range of information. To bridge the language gap, some works rely on automatic translation of queries or documents or consider altogether multilingual users to begin with [10,15,19,24]. Many multilingual users and new language learners are not equally In this paper, we present late-breaking results of cross-lingual query-by-example retrieval. We assume our users are English speakers with only partial prociency in Arabic who want to explore an Arabic news wire collection. As a consequence, the majority of topic information is phrased in English, while a few limited Arabic documents are given as examples to aid model learning and to provide supporting retrieval information. To cater for this situation, we propose an end-to-end retrieval pipeline, Cross-Lingual Arabic Information REtrieval (CLAIRE) system, which 1) pre-selects potentially relevant documents via an exact-match step, 2) re-ranks them using a neural retrieval model and 3) fuses the result lists for each partial query into an overall ranking.
The remainder of this paper is structured as follows: Section 2 concisely discusses related work on cross-lingual information retrieval, query-by-example schemes, multilingual embeddings, and neural machine translation. Section 3 provides the details of the proposed CLAIRE retrieval system. Section 4 describes the experimental setup and Section 5 its results, before Section 6 closes with an outlook on future directions of inquiry.

BACKGROUND
Query-by-example (QBE) is a popular retrieval setting that oers more information to specify user's interest. It has been applied to retrieve documents or entities from the target corpus [25][26][27], music information retrieval [13] and tuples from relational databases [9]. In many real-world applications, the given query and the target corpus may be written in dierent languages. Cross-lingual information retrieval (CLIR) is essential to bridge the language gap and to ensure reliable ranking performance on this challenging task. There are two major trends: (1) translation-based approach and (2) modelling based on cross-lingual word embedding.
For translation-based approach, Douglas Oard et al. explore methods of translating documents into the query language [19]. And Jianfeng Gao et al. choose to translate queries for eciency [10]. Also, there is a so-called interlingual approach by mapping both queries and documents into a third common language [15,24]. Due to the eective deep representations learned from neural networks, deep learning has revolutionized the domain of machine translation signicantly. Some powerful models are LSTM-Autoencoder-based Google's Neural Machine Translation system (GNMT) [30] and BERT-based translation models [6,32]. However, training translation models can be expensive and time-consuming. Also, how to transfer pre-trained translation models to the new dataset properly and how to incorporate it into the information retrieval system eectively still remain challenging open tasks. On the other hand, the usage of the cross-lingual word embeddings avoids the potential mistakes made by translation models. Specically, cross-lingual word embedding maps words from different languages into the common shared vector space. It enables transferring knowledge from one language to another without the additional translation, which simplies the overall retrieval pipeline signicantly [7]. Also, it opens the retrieval system to the possibilities of exploring semantic matching signals across dierent languages directly and explicitly.
There are four major ways to construct cross-lingual word embeddings [23]: (1) Monolingual mapping, where a special linear mapping among various monolingual word embeddings of dierent languages enable them to transfer words from the source language to the target language [29]. (2) Pseudo-cross-lingual, trains word embedding models on a pseudo-cross-lingual corpus by mixing contexts of dierent languages [8? ]. (3) Joint optimization, jointly optimises a combination of monolingual and cross-lingual losses [5]. (4) Cross-lingual training, relies on the parallel corpus, where it builds pairs of words/sentences with similar meanings from different languages [11,16]. In our retrieval system, we rely on the English-Arabic cross-lingual word embeddings from the work of Lachraf et al. [16], which is trained based on the parallel corpus.

THE CLAIRE SYSTEM
In order to simulate how non-native Arabic speakers retrieve the target Arabic information based on their knowledge in English in the realistic scenarios, we assume each topic to consist of four major components: (1) the topic title in English, which concisely describes what they are looking for, (2) the topic background in English, which gives more general supporting information, (3) related event or knowledge descriptions in English based on the user's experience, and (4) limited examples of known relevant Arabic documents. Four dierent components represent various knowledge aspects users obtained from dierent information sources. An example topic of peace eorts on the Korean peninsula is illustrated in Figure 1.

Retrieval Pipeline
The proposed cross-lingual query by example retrieval system, CLAIRE, comprises three major sub-systems: (1) BM25 pre-selection, (2) neural re-ranking, and (3) fusion system. The overall workow is illustrated in Figure 2.
BM25 Pre-selection: We rely on BM25 [22] to lter out the majority of obviously non-relevant documents, reducing the computational burden of down-stream processing steps. To directly apply BM25 on the Arabic documents we use simple word-level English-Arabic lookup table to translate any English query components to Arabic and then apply BM25 to pre-select potentially relevant candidates.
Neural Re-ranking: Various neural retrieval models such as Con-vKNRM [33], KNRM [3], MatchPyramid [20? ], DRMM [14] and PACRR [12] have been recently proposed. They extract semantic matching signals based on word-level embeddings of queries and documents. Usually, word embeddings are initially trained via skipgram [1] or CBOW models [17]. As a result, all word tokens of the given word embeddings are of the same language. To facilitate direct cross-lingual ranking, we instead utilise joint English-Arabic cross-lingual word embeddings [16], which is trained via the parallel English-Arabic sentence alignment. The employment of this cross-lingual embedding avoids the potential mistakes caused by translation and simplies the overall neural retrieval re-ranking system to make it more robust and time-ecient. We follow the pre-processing and normalization approach described by Lachraf et al. [16] to tokenize Arabic sentences. For each component in our queries, the neural retrieval model is applied to re-rank the corresponding pre-selected Arabic documents.
Fusion System: Dierent components of our retrieval request provide partial relevance information. Rank fusion is the natural and popular technique to integrate ranking information from dierent sources. In the proposed CLAIRE system, Reciprocal Rank Fusion (RRF) [4] is applied to the partial ranked list produced by each topic component (title, background, description, examples). Given the ranking order of the same document 3 in ⇡ dierent sub-rankings, A 1 , A 2 , ..., A ⇡ , the RRF-based ranking score in the system is given by . We employ a two-step fusion due to the dierent ranking score calibrations between BM25 and neural retrieval models. We rst apply RRF to combine ranking results of pre-selected documents for each query component based on the output of BM25 system. Then RRF is also performed on the ranking output of the neural reranking system. The nal ranking scores in the proposed CLAIRE system are based on RRF scores of the BM25 fusion and the neural re-ranking fusion, as shown in Figure 2.

EXPERIMENTS
We conduct our retrieval experiments on a proprietary collection of 730k Arabic news articles covering a broad range of subjects. We construct 35 search topics as input to our retrieval pipeline. On average, each topic identies 2.5 known relevant input example documents.

BM25 Pre-selection
The number of documents pre-selected by BM25 provides a critical foundation for our overall CLAIRE system. Too small thresholds can ignore some reasonably relevant candidates due to the simplicity of BM25. If the threshold is too large, many non-relevant documents may introduce noise or slow down the overall pipeline. Hence, we empirically investigate thresholds of 1,000, 5,000, 10,000 most highly ranked BM25 documents to be included in neural re-ranking.

Neural Retrieval Re-ranking
Our system trains neural retrieval models via the ListNet ranking loss [2] due to its superior ranking performance, fast convergence and scalable reliability. As a listwise approach, ListNet compares a list of candidates simultaneously and extracts the inter-document ranking signals to feed neural retrieval models. It rst applies the classic softmax function to transform both candidate prediction list and relevance label list into two corresponding probability distributions, then utilises cross entropy loss to nalize optimization signals. Its simple and eective formulation often outperforms other listwise ranking schemes, such as ListMLE [31] and Approx-NDCG [21]. We choose the list size of ListNet as 50 concurrently evaluated documents. For each BM25 pre-selected candidate pool, we treat non-example documents as non-relevant and construct all possible candidate lists by randomly selecting non-example documents. Each candidate list contains only a single example document as the relevant candidate.
We consider three neural retrieval models: (1) KNRM [3], a kernel based neural retrieval model that uses Gaussian kernel pooling techniques on the word-level query-document similarity matrix to extract multi-level soft-match features. (2) ConvKNRM [33], the convolutional kernel-based extension of KNRM via Convolutional Neural Networks to compose adjacent words' n-gram embeddings and (3) MatchPyramid [20? ] that regards the constructed wordlevel similarity matrix from query and document embeddings as a special "image" and utilises several stacked CNN layers to extract matching signals for the nal ranking score computation.

Model Implementation
All neural retrieval models are implemented in Tensorow 1 . AraEngVec [16], the cross-lingual word embedding, is used for vocabulary initialization. We use an Adam optimizer to train all models for a xed 21 epochs and save models every three epochs. Then models with the best performance are selected. The learning rate is set to 0.0001. The maximum length of queries and documents are set as 150 and 400, respectively. Regarding model-specic parameters, we apply 11 Gaussian kernels with one harvesting exact matches (mean valuè 0 = 1.0 and standard deviation f 0 = 10 3 ) and ten capturing soft matches (mean values spaced evenly from -0.9 to 0.9, and standard deviation f 1 = 0.1) for both KNRM and ConvKNRM. The number of CNN lters in ConvKNRM is set to 128. For MatchPyramid, we choose 3 convolution layers, each with kernel size 3 ⇥ 3 and 16 convolutional channels.

RESULTS
For each BM25 pre-selection threshold and neural retrieval model, we further translate the English query parts, (1) topic title, (2) topic background and (3) event/knowledge descriptions, to Arabic directly using the simple word-level English-Arabic lookup table for our neural retrieval re-ranking system as the baseline. During model evaluation, each topic query on average has around 50 relevant documents identied by a large pool of in-house expert annotators. We evaluate and compare dierent system settings in terms of Precision@10, Recall@10 and Normalized Discounted Cumulative Gain at the threshold of 10 (nDCG@10). Test performance is shown in Table 1. BM25 pre-selection provides a solid foundation for the overall CLAIRE system. Due to the range of number of relevant documents for each query, a relatively low BM25 threshold of 1,000 preretrieved documents yields most reliable results. When there are more unnecessary documents pre-selected from BM25, the ranking performance of neural retrieval models and the nal fusion results deteriorates gradually. This is especially signicant for MatchPyramid model. Compared to the fully translated systems, the direct retrieval from English query components usually produces better performance in most cases, which shows the eectiveness and robustness of the joint cross-lingual word embeddings. Moreover, KNRM models provide competitive ranking performance, even better than the more sophisticated ConvKNRM in terms of nDCG@10. Since there are only 35 topics and only two or three relevant document examples for model training, simpler models can be a reasonable choice due to their weaker dependency on an abundance of training information. Furthermore, the RRF-based rank fusion improves the overall performance of the system by integrating the BM25 pre-selection system and neural retrieval re-ranking system.

Topic Component Analysis
To further analyse the relative importance of each topic component, we run our system for dierent neural retrieval models on each component individually. We again choose 1,000 as the BM25 threshold. The test performance of the nal fusion is shown in Table 2. Across dierent neural retrieval models, the general relative importance among the four major components is consistent and clear. The Arabic example documents play an essential role and provide reliable ranking performance. Although there are no specic retrieval targets indicated in the example documents, our system can still exploit the main ideas and perform query by example retrieval eectively. On the other hand, the topic title shows relatively weaker contribution due to its limited content. This is especially dicult when there are limited amounts of data for model learning.
Topic background and descriptions yield reasonable performance according to their content size. It is worth noting that even though the full fusion system performs on par with example document components alone in terms of Precision@10 and Recall@10, its nDCG@10 is signicantly better.

Eect of the Fusion System
Besides Reciprocal Rank Fusion (RRF) [4] there are various other unsupervised rank fusion algorithms, such as CombSUM [28], CombM-NZ [28] and Inverse Square Rank (ISR) [18]. Table 3 explores the eectiveness of such alternative fusion schemes. RRF-based systems outperform other fusion methods signicantly across three neural retrieval models and dierent sub-systems. Although the dierence between RRF and CombSUM is relatively small for BM25 pre-selection, CombSUM yields much worse results on neural reranking systems and the nal fusion pipeline. In general, simple methods such as RRF and CombSUM perform better than CombMNZ and the more involved ISR. Due to the unsupervised nature of these rank fusion methods, simpler methods with fewer a priori assumptions are more robust and generalizable to diverse scenarios.

CONCLUSION
This chapter describes our late-breaking investigation results of cross-lingual query by example retrieval models. Our end-to-end Cross-Lingual Arabic Information REtrieval (CLAIRE) system yields promising performance without incorporating the complicated neural translation models into our neural retrieval system. To simulate how non-Arabic speakers conduct Arabic retrieval tasks in realistic scenarios, we design 35 structural topic queries to reect various knowledge information users obtained from their experience. We use three fundamental retrieval steps: (1) BM25 pre-selection, (2) neural re-ranking, and (3) retrieval fusion. It is interesting to observe that the simpler KNRM model provides very competitive performance under the circumstances of the limited training information, compared to ConvKNRM and MatchPyramid with higher complexities. Moreover, we present the eect of dierent BM25 thresholds and various fusion methods. As a classic retrieval approach, BM25 provides a solid foundation of our CLAIRE system. Compared to other rank fusion methods, Reciprocal Rank Fusion (RRF) is robust and generalizable to dierent retrieval systems. Its reliable performance further boosts our CLAIRE system. For future work, we would like to explore more general approaches to build multi-lingual retrieval systems to aid users in exploring various languages simultaneously.

ACKNOWLEDGMENTS
Acknowledgements blinded for anonymous review.


s2fieldsofstudy:
Computer Science
Business

text:
Multi-Antenna Systems by Transmissive Reconfigurable Meta-Surface

Reconfigurable meta-surface (RMS) is proposed as a very promising and novel technology, which is composed of a large number of low-cost passive elements, and can achieve passive beamforming by controlling the amplitude and phase of incident electromagnetic (EM) waves. Therefore, in order to solve the challenges of high power consumption and high cost of existing base stations (BSs), we propose a low-cost and low-power consumption transmissive RMS multi-antenna system in this paper. Specifically, we first provide an overview of the transmissive RMS multi-antenna system, including its advantages, network architecture, transmission mechanism, modulation principle, channel model and channel estimation technique. Then, we address transceiver design and optimization for downlink (DL) and uplink (UL), and some numerical results are also given to verify the effectiveness of the proposed algorithm. Finally, several potential research directions of the transmissive RMS multi-antenna system are given to inspire further investigation in future work.

Abstract-Reconfigurable meta-surface (RMS) is proposed as a very promising and novel technology, which is composed of a large number of low-cost passive elements, and can achieve passive beamforming by controlling the amplitude and phase of incident electromagnetic (EM) waves. Therefore, in order to solve the challenges of high power consumption and high cost of existing base stations (BSs), we propose a low-cost and low-power consumption transmissive RMS multi-antenna system in this paper. Specifically, we first provide an overview of the transmissive RMS multi-antenna system, including its advantages, network architecture, transmission mechanism, modulation principle, channel model and channel estimation technique. Then, we address transceiver design and optimization for downlink (DL) and uplink (UL), and some numerical results are also given to verify the effectiveness of the proposed algorithm. Finally, several potential research directions of the transmissive RMS multi-antenna system are given to inspire further investigation in future work.

I. INTRODUCTION
W ITH the evolution and development of mobile communication technology, the power consumption and cost of base stations (BSs) are increasing. Specifically, the power consumption of a single 5G macro BS is about 3500W, while the power consumption of a single 4G macro BS is about 1300W. Therefore, the power consumption of the former is about three times that of the latter. In addition, compared to 4G networks, 5G networks are operated at higher frequency bands, and the attenuation is more serious. To achieve the same coverage area as 4G networks, the number of 5G BSs is about three to four times that of 4G BSs, so the deployment costs are relatively high. Moreover, a large number of radio frequency (RF) chains and complex signal processing units are required adapting the massive multi-input multioutput (MIMO) architecture adopted in the 5G networks [1]. Accordingly, compared with 4G BS, the design cost of 5G BS is also greatly increased. Therefore, it is imperative to seek a novel transceiver architecture that can reduce power consumption and cost for future beyond 5G and 6G networks.
Recently, reconfigurable meta-surface (RMS) has been proposed as a promising technology [2]. In detail, RMS is a plane, which is composed of a large number of low-cost passive elements. The amplitude and phase shift of these elements can be adjusted independently by intelligent controller equipped with RMS, thereby generating directional beams. Due to the almost passive full-duplex operation mode, the RMS only passively reflects or transmits the incident signal. Therefore, compared with traditional relay technology, RMS does not encounter the problems of self-interference and additional noise. Compared with the existing multi-antenna technology, it does not need complicated RF chains and signal processing modules, so the required hardware cost and power consumption are much reduced. These advantages greatly show the potential of RMS in the next generation communication networks.
RMS is mainly divided into reflective mode and transmissive mode. At present, more work focuses on the two modes RMS-assisted communication [3]- [6]. The RMS of the reflective mode or the transmissive mode realizes the reconstruction of the wireless environment by adjusting the amplitude and phase shift of the incident signal, realizing the enhancement of the desired signal strength and the reduction of the interference signal strength, thereby improving the spectral efficiency, energy efficiency and coverage area of the networks. In addition to being used for auxiliary communication, RMS can also be designed as a transmitter [7], [8], which are two completely different research perspectives. The latter is also in its infancy, and it is very promising. It can greatly meet the requirements of future networks for low power consumption and low cost transceivers. However, compared to the reflective RMS multi-antenna system, the transmissive RMS can be designed to have a higher efficiency [9], [10]. This is mainly due to the following reasons. First, the transmissive RMS multi-antenna system has no feed blockage (occlusion). Second, the transmissive RMS multi-antenna system will not cause self-interference problems. The aperture efficiency of the transmissive RMS multi-antenna system is higher. Finally, the operating bandwidth of the transmissive RMS multi-antenna system is typically larger. The details will be explained in the next section. These advantages make the transmissive RMS multi-antenna system very promising for the future networks.
Although the transmissive RMS multi-antenna system has many advantages, it is composed of passive transmissive RMS and a single antenna feed, which is still very different from the traditional multi-antenna system composed of active components. So far, the transmissive RMS multi-antenna system is in its initial stage, it is crucial to depict its framework and mechanism. Therefore, this promotes this paper to give an overview of the transmissive RMS multi-antenna system, including its advantages, network architecture, transmission mechanism, modulation principle, channel model, channel estimation. Moreover, we present the design methodology of the downlink (DL) transceiver and the uplink (UL) transceiver. In addition, numerical results are provided to verify the effectiveness of the proposed algorithm in transmissive RMS multiantenna systems. Finally, some potential research directions for the future work are given. F r e q u e n c y ( G H z ) S 1 1 ᒬ π -s t a t e S 1 2 ᒬ π -s t a t e S 2 2 ᒬ π -s t a t e

II. THE ADVANTAGES OF TRANSMISSIVE RMS MULTI-ANTENNA SYSTEM
In this section, we first provide the advantages of our proposed transmissive RMS multi-antenna system compared to the traditional multi-antenna system and the reflective RMS multi-antenna system.
Compared with the traditional multi-antenna system, since the transmissive RMS multi-antenna system is only composed of a single feed (receiving) antenna and passive transmissive RMS without complicated RF chains and signal processing units, its power consumption and cost can be greatly reduced. Then, we mainly elaborate on the advantages of the transmissive RMS multi-antenna system over the reflective RMS multi-antenna system from four aspects.

A. No feed blockage (occlusion)
The user and feed antenna of the reflective RMS multiantenna system are located on the same side of the RMS, while the user and receiving antenna of the transmissive RMS multiantenna system are located on the opposite sides of the RMS. Therefore, there is no feed blockage (occlusion) problem in the transmissive RMS multi-antenna system, which will make the aperture efficiency of the transmissive RMS multi-antenna system higher.

B. No self-interference problem
For the reflective RMS multi-antenna system, the incident signals and the reflective signals are on the same side of the RMS, which will bring self-interference problems. As for the transmissive RMS multi-antenna system, the incident signals and the transmissive signals are on the opposite sides of the RMS, which can avoid this problem well. Therefore, the transmissive RMS multi-antenna system is more promising for the more complicated communication environments in the future.

C. Higher aperture efficiency
According to the literature [9] and [10], the aperture efficiency of the transmissive RMS multi-antenna system can be designed to be higher than that of the reflective RMS multi-antenna system. For instance, the literature [9] gives the simulated scattering coefficient of the 1-bit transmissive unit as shown in the Fig. 1, where S 11 and S 12 represent reflective coefficient and transmissive coefficient, respectively. According to Fig. 1 (a) and (b), it can be seen that from 7.12 to 7.85 GHz, in both 0-state and π-state, the transmissive coefficient is above -2dB, and the reflective coefficient is below -10dB. Therefore, the transmissive RMS multi-antenna system can be designed to achieve higher aperture efficiency while avoiding the feed blockage (occlusion).

D. Higher operating bandwidth
It can be seen from Fig. 1 (c) that the phase difference in transmissive coefficients of the π/0 states remains stable at around π with very small deviation over a wide frequency band. However, the reflective phase difference of the reflective RMS is unstable enough [11]. The more stable phase difference of the transmissive RMS multi-antenna system makes it have a larger operating bandwidth.

III. ARCHITECTURE AND TRANSMISSION MECHANISM OF
TRANSMISSIVE RMS MULTI-ANTENNA SYSTEM As shown in Fig. 2, the architecture consists of a feed (receiving) antenna, RMS with M transmissive elements, intelligent controller and K users. In DL, the feed antenna can radiate single-frequency electromagnetic (EM) waves, while in UL, the receiving antenna can receive signals. Let f m = β m e jθm denote the RMS transmissive coefficient, where β m ∈ [0, 1] and θ m ∈ [0, 2π) respectively represent the transmissive amplitude and phase shift of the m-th element of RMS. Moreover, the controller equipped with RMS can adjust the amplitude and phase shift of the transmissive element to achieve the modulation and beamforming of the DL, as well as the enhancement of the UL incident signal. It is worth noting that due to the unique structure of the transmissive RMS multi-antenna system, DL can adopt multi-user space division multiple access (SDMA), and UL can adopt multiuser orthogonal frequency division multiple access (OFDMA) to achieve multi-user diversity. The specific DL and UL transmission mechanism is as follows:

A. Downlink transmission
We have proposed the downlink transmission process of the transmissive RMS multi-antenna system in [8]. Specifically, the information from the signal source is modulated by the controller (high-order modulation can now be achieved by time sequence modulation [7]) as part of the RMS transmissive coefficient, and the beamforming design proposed by [8] is used as the other part of the RMS transmissive coefficient. The two are multiplied as the RMS transmissive coefficient. In other words, the RMS controller can realize information modulation and beamforming design through the corresponding control signal, and the coefficient can be mapped to each element of the transmissive RMS. Then, the feed antenna sends a single tone signal to carry and send the information of each element to multiple users, and the multi-user diversity can be achieved by using SDMA with beamforming.

B. Uplink transmission
This process has been studied in [12], which is devoted to the performance improvement of the uplink transmission via RMS transmissive coefficient design for the transmissive RMS multi-antenna system. Since the receiver is equipped with a single antenna, we consider multi-user access by using OFDMA. The transmissive RMS strengthens the uplink signal and forwards it to the receiving antenna, and then the receiving antenna sends the received signal to the controller for demodulation and decoding.

IV. DL MODULATION PRINCIPLE
In this section, we mainly provide the DL modulation principle of the transmissive RMS multi-antenna system. Note that the RMS in the transmissive RMS multi-antenna system does not have the complicated RF chains and baseband processing capabilities, so the conventional modulation method cannot be directly applied for this system. Several modulation schemes that can be used in the proposed system are described in detail below.
The modulation for this system is mainly divided into constant envelope modulation and non-constant envelope modulation. In the case of using constant envelope modulation, taking binary phase shift keying (BPSK) and quadrature phase shift keying (QPSK) as instances, we can control the conduction state of the diode in each element of the RMS by a control signal, i.e., BPSK and QPSK can be realized by 1bit and 2-bit metasurfaces, respectively. Similarly, higher-order phase modulation can also be achieved in this way, but it is challenging. This is mainly due to space constraints, which is hard to accommodate a larger number of diodes in each RMS element. Moreover, the control signal needs to be multiplexed, which is also challenging to design. However, the degree of control freedom for constant envelope modulation remains at one. In this way, only one modulation symbol with a constant envelope can be generated on the carrier frequency, and highorder phase and amplitude joint modulation (i.e., non-constant envelope modulation) such as 16-quadrature amplitude modulation (QAM) cannot be realized, which will greatly limit the transmission rate.
To solve this problem, a nonlinear modulation technique (i.e., time sequence modulation) can be used to achieve highorder phase and amplitude joint modulation [7]. As shown as in Fig. 3, it is worth noting that after adopting time modulation, the waveforms of control signal has two degrees of control freedom, i.e., phase start time t on,n and conduction duration τ n . By expanding the Fourier series of the control symbol, we can get its amplitude and phase shift on the l-order harmonic. The amplitude and phase shift of the l-order harmonics can be independently adjusted by setting the two degrees of freedom. Furthermore, the mapping relationship between the corresponding modulation scheme and the amplitude and phase of the l-order harmonic can be realized, i.e., the joint modulation of the phase and the amplitude can be realized. Therefore, 16-QAM or higher order QAM can be achieved by applying the time modulation scheme, which can greatly improve the system performance. In addition, the abovementioned BPSK, QPSK and higher order phase modulation can also be implemented. Moreover, for high-order modulation, which can be implemented on 1-bit or 2-bit RMS, the structure of the required RMS is simpler, and the control signal is also simpler. In addition, the scheme is reconfigurable, and the control signal waveform can be changed according to different modulation requirements. Therefore, the modulation scheme has great potential for the design of the proposed system. V. FAR-NEAR FIELD CHANNEL MODEL As depicted in Fig. 4, the EM wave radiation field in the wireless communication network can be divided into far field and near field [13], which have different channel models. The far-near field is determined by the Rayleigh distance 2D 2 λ , where D and λ respectively denote array aperture and wavelength. The judgment method by applying Rayleigh distance is that, if the distance between the transmitter and the receiver is larger than Rayleigh distance, the channel can be regarded as a far-field channel, and the wavefront can be approximated as a planar wave. Otherwise, when the distance between the the transmitter and the receiver is less than Rayleigh distance, the channel can be modeled as near-field channel, and the wavefront is spherical wave. Hence, the channel model of the proposed RMS multi-antenna system can be elaborated as follows.

A. Far-field RMS-user channel model
The channel from the user to the RMS can be named RMSuser channel. Since the distance from the user to the RMS is larger than the Rayleigh distance, the RMS-user channel can be regarded as a far-field channel, which is modeled under the assumption of planar waves. Considering that there are both line-of-sight (LoS) components and non-line-of-sight (NLoS) components between the user and the RMS, we model the RMS-user channel as a Rician fading channel. When the Rice factor is zero, the RMS-user channel can be transformed into a Rayleigh channel. Furthermore, each element of the NLoS component is an independent and identically distributed circularly symmetric complex Gaussian (CSCG) random vector with zero mean and unit variance. For the LoS component, it can be modeled in two ways: uniform linear array (ULA) and uniform planar array (UPA). It is obvious that the UPA is more practical due to the RMS is a plane.

B. Near-field RMS-feed (receiving) channel model
The channel from the RMS to the feed (receiving) antenna can be named RMS-feed (receiving) channel. Since the distance from the feed (receiving) antenna to the RMS is less than the Rayleigh distance, the RMS-feed (receiving) channel can be regarded as a near-field channel, which is modeled under the assumption of spherical waves. Considering that there is no blockage (occlusion) between the RMS and the feed (receiving) antenna, we can model it as a LoS channel with UPA. For the specific modeling and expression of the far-near field channel, please refer to [12].

VI. CHANNEL ESTIMATION
Since both the uplink and downlink transmissive coefficient designs require channel state information (CSI), the channel estimation for the proposed transmissive RMS multi-antenna system is vital. Hence, we interpret the channel estimation for the proposed system in this section. It is worth noting that the transmissive RMS can only transmit the signal passively, so the channel estimation of the system is not easy. To estimate the DL channel for RMS beamforming design, we consider the time-division duplexing (TDD) protocol, and estimate the UL channel. According to the channel reciprocity, the CSI of DL can be obtained. For UL transmission, because the receiving antenna has the receiving ability, and the RMS transmissive element has no receiving ability, we can only obtain the CSI of the UL cascaded channel by direct cascaded channel estimation, and cannot directly obtain the CSI of the separable channel. Nevertheless, only when we obtain the CSI of the UL RMS-user channel, the CSI of DL RMS-user channel can be obtained according to the channel reciprocity, and then the DL RMS transmissive coefficient can be designed. Moreover, the near-field RMS-receiving channel can be obtained by calculation or measurement, and it is usually fixed. Therefore, in the case of known cascaded CSI and near-field CSI, we can learn from some RMS separable cascaded channel estimation methods to obtain far-field RMS-user CSI. Below, we provide the two channel estimation methods in detail, i.e., direct cascaded channel estimation and separable cascaded channel estimation.

A. Direct cascaded channel estimation
For the direct cascaded channel, i.e., the multiplication of two parts of the channel, the existing work usually proposes some channel estimation protocol, and then uses different algorithms to estimate it [14]. Direct cascaded channel estimation algorithms usually include message passing (MP) algorithm, channel correlation method, optimization-based channel estimation algorithm, decomposition, interpolation recovery, etc.. It is worth noting that the MP algorithm usually requires a sparse representation of the channel, and the method of considering the channel correlation can reduce the overhead of the pilot. We can consider the above-mentioned channel estimation method to obtain the UL cascaded CSI.

B. Separable cascaded channel estimation
For the separable cascaded channel estimation, some studies have proposed the semi-passive RMS channel estimation method and a fully passive RMS channel estimation method [15]. For semi-passive RMS, it is equipped with a part of active elements to receive the transmitted signal from the transceiver or user, and then estimate the angle and path loss of the transceiver-RMS channel and RMS-user channel. However, this method needs to feed back to the transceiver and the user for beamforming design again, which will reduce efficiency. In addition, it will also increase the cost of hardware. Full-passive RMS can solve this limitation well. For the full-passive RMS, the user transmits different pilot sequences to the transceiver through the RMS, and through the analysis of the received signal, a two-stage method can be used to estimate the cascade channel. Note that the near-field channel of the transmissive RMS multi-antenna system can be calculated or measured, and the cascaded channel can be obtained by the method in the previous subsection. Therefore, we can use the method mentioned above to estimate the RMS-user channel.

VII. DL TRANSCEIVER DESIGN AND OPTIMIZATION
With the UL transmission mechanism and CSI based on the above channel estimation, the DL transceiver can be designed and optimized. In [8], we have proposed a transmissive RMS transmitter architecture, and conducted a preliminary study on the downlink multi-user beamforming design (i.e., RMS transmissive coefficient design) and power allocation under the transmissive RMS transmitter architecture. Specifically, a transmitter design based on a transmissive RMS equipped with a feed antenna is first proposed, where the RMS array is distributed in UPA and we use a geometric channel model with L propagation paths to model the channel between the transmitter and the user. Also, the modulation method is introduced in detail. Then, the achievable system sum-rate are maximized by jointly optimizing the RMS beamforming design and power allocation. Considering that the optimization variables are coupled, this formulated optimization problem is non-convex. Accordingly, we propose an alternating optimization (AO) technique based on difference-of-convex (DC) programming and successive convex approximation (SCA) to solve this problem. Via applying this algorithm, the nonconvex optimization problem is transformed into a convex optimization problem, which can be effectively solved by using the CVX toolbox.
For DL transceiver design and optimization, we consider a system setup of K = 4 users and M = 25 RMS elements. We assume that the parameters of all users are the same. We compare the performance of the proposed algorithm with other baseline algorithms in Fig. 5.
• Equal allocation (EA): The beamforming design is the same as our proposed algorithm, and the power allocation is equal allocation. • Zero-forcing (ZF) beamforming: The beamforming design applies ZF algorithm, and the power allocation is equal allocation. • Random allocation (RA): The beamforming design and power allocation adopt random allocation. Fig. 5 depicts the change of system sum-rate with the number of RMS elements. It can be seen that as the number of RMS elements increases, the system sum-rate increases. This is because more RMS elements can bring more spatial diversity gains. Meanwhile, more RMS elements can get more signal scattering paths, so that the user can get a stronger signal. In addition, when the number of RMS elements is the same, 10 15 our proposed algorithm has a better gain than other baseline algorithms.

VIII. UL TRANSCEIVER DESIGN AND OPTIMIZATION
After obtaining the uplink cascaded CSI through the above channel estimation algorithm, we have already studied uplink transceiver design and optimization for transmissive RMS multi-antenna systems in [12]. In detail, we propose a transmissive RMS-based receiver equipped with a single receiving antenna, and a far-near field channel model based on planar waves and spherical waves is given. Also, we discuss the advantages of the proposed architecture. In this architecture, the multi users access the networks by OFDMA due to the receiver is equipped with single antenna. Then, in order to maximize the system sum-rate of uplink communications, we formulate a joint optimization problem over subcarrier allocation, power allocation and RMS transmissive coefficient design. Due to the coupling of optimization variables, the optimization problem is non-convex. We apply the AO algorithm baesd on Lagrangian dual decomposition method, DC programming, SCA and penalty function methods to tackle this problem. Moreover, the superiority of the proposed scheme is verified by numerical simulation.
For UL transceiver design and optimization, we consider a 3D coordinate system setup with K = 5 users and M = 25 transmissive elements. In order to evaluate the performance of the proposed algorithm, we consider the following benchmark algorithm. System sum-rate (bps/Hz) power allocation, and RMS transmissive coefficient design. Fig. 6 shows the change of system sum-rate under different algorithms when the number of RMS transmissive elements increases. We can see that the performance of the proposed algorithm outperforms other benchmark algorithms. Moreover, the system sum-rate increases as the number of RMS transmissive elements increases. This is because the RMS transmissive element will strengthen the user's uplink transmission signal and bring a certain gain, which is positively correlated with the number of RMS transmissive elements. Therefore, the more RMS transmissive elements, the more gain it brings, and the greater the system sum-rate. It provides a good guide for multi-antenna systems based on transmissive RMS, which can improve system performance by increasing the number of low-cost transmissive elements, which has great potential for the future networks that needs to reduce cost and power consumption.

IX. POTENTIAL DIRECTIONS
With the advantages of the transmissive RMS multi-antenna system, in this section, we provide a potential research direction in the future under this new type of multi-antenna system.

A. Non-orthogonal multiple access (NOMA) with transmissive RMS multi-antenna system
For the UL of the transmissive RMS multi-antenna system, the receiver is a single antenna. Therefore, in addition to considering the orthogonal multiple access (OMA) scheme, NOMA is also a very important access scheme. Moreover, for DL, NOMA can well meet the large-scale connections of 6G networks and the higher service requirements of users. Unlike traditional OMA, NOMA can support multiple users to share the same resources, such as time, frequency, coding, etc.. Specifically, taking NOMA in the power domain as an example, the transmitter can use the same resources to serve multiple users, which can greatly improve the spectrum efficiency to meet the user's service requirements. For NOMA transmission in the DL and UL power domain, superposition coding and serial interference cancellation (SIC) technologies are applied at the transceiver and the user, respectively. By applying SIC technology, users with stronger channel gains can remove co-channel interference caused by users with weaker channel gains before decoding. It is a very novel work to consider the design of SIC decoding order and RMS transmissive coefficient at the same time.

B. Robust beamforming design with transmissive RMS multiantenna system
At present, the design and optimization of DL and UL transceivers have been completed under the assumption that perfect CSI can be obtained. However, it is more practical to consider the design and optimization of DL and UL transceivers in the case of imperfect CSI. Specifically, we can consider the optimization of system performance when the channel estimation has a certain error. The modeling of channel estimation errors can be modeling based on bounded and based on statistical outage probability. When we consider that the channel estimation has a certain error, the solution of the optimization problem will become more complicated and challenging, but it is more practical, which can be used as a very meaningful work in the future.

C. Wireless power transmission (WPT) with transmissive RMS multi-antenna system
Considering that there will be more large-scale, low-power, energy-constrained Internet-of-things (IoT) devices in 6G networks, to provide continuous information transmission and energy transmission requirements, simultaneous wireless information and power transfer (SWIPT) and wireless powered communication networks (WPCN) can serve them well. We can propose WPT with transmissive RMS multi-antenna system to realize a new network paradigm for 6G WPT with lower power consumption and cost. Meanwhile, considering the limitation of system performance degradation due to distance limitation, we can also consider using reflective RMS to further strengthen, greatly improving the transmission efficiency and coverage, so as to realize the transmissive-reflective RMS-empowered WPT network architecture in 6G networks. However, the joint design for WPT network parameters and RMS coefficients is still challenging.

X. CONCLUSIONS
In this paper, we provide an overview of the efficient transmissive RMS multi-antenna system for achieving low power consumption and low cost transceiver in the beyond 5G and 6G networks. RMS can adjust its transmissive coefficient to achieve DL beamforming and UL signal enhancement. Since the design and optimization of transmissive RMS transceivers are new and have not been explored to a large extent, it is hoped that this paper can provide some useful guidance for future research. Meanwhile, we can foresee that the application of low-cost and low-power RMS in future wireless communication networks will cover the overall closed-loop wireless transmission link composed of transmitters, auxiliary wireless communications, and receivers. The new RMS-based wireless communication system is expected to achieve the best overall performance through joint optimization design.