extfieldsofstudy:
Computer Science
Engineering

text:
A Single-Layer Asymmetric RNN: Potential Low Hardware Complexity Linear Equation Solver

A single layer neural network for the solution of linear equations is presented. The proposed circuit is based on the standard Hopfield model albeit with the added flexibility that the interconnection weight matrix need not be symmetric. This results in an asymmetric Hopfield neural network capable of solving linear equations. PSPICE simulation results are given which verify the theoretical predictions. Experimental results for circuits set up to solve small problems further confirm the operation of the proposed circuit.

I. INTRODUCTION
Efficient and fast solution of a system of linear equations (LE) has been in focus of research for quite some time now. With the advent of digital computers, algorithms took centerstage for such tasks. However, even modern day computers running highly optimized algorithms do not match up to the speeds offered by analog computation for providing the solution for systems of simultaneous linear equations (SLE) [1]- [4].
The Hopfield neural network (also popular by the more simplified 'Hopfield Network') is widely known to be amongst the most prominent and influential models in the research on bio-inspired neural architectures and artificial neural networks (ANNs) [5]. Several other competing architectures that have appeared in the technical literature following the rise and popularity of Hopfield networks, for instance Boltzmann machine (also called stochastic Hopfield network) [6], bidirectional associative memory (BAM) [7], attractor neural network with Q-state neurons [8], etc., have their origins in the theory of the Hopfield Network, and therefore are either improved generalizations or direct minor/major variants of the standard Hopfield network [5]. In fact, a quick survey of the technical literature leads to the observation that thehe Hopfield neural network (HNN) has indeed been found to be quite useful in an assortment of applications, such as face recognition, solution of the Travelling Salesman Problem (which finds wide applicability in a range of different scenarios), Content Addressable Memories (CAM), analog-todigital (A/D) conversion, solution of linear programming problems (LPP), grayscale image recognition, and signal decision. To embed and solve problems related to signal processing, Author is with the Aligarh Muslim University, India. (Email: samar.ansari@zhect.ac.in.) grayscale image recognition, neural control and constrained optimization, the HNN needs to be appropriately designed to possess one (unique, singular) and globally stable point of equilibrium, in order to mitigate the risk of spurious (and therefore incorrect and unwanted) responses that are bound to creep in due to the widespread problem of the presence of local minima in such circuits [9]. In order to accomplish these objectives, it has become customary to enforce certain restrictions (i.e. constraint conditions) on the weight matrix (or the interconnection matrix) of the network [10]. One pertinent example of such restrictions being imposed is as follows. The majority of study and applications of the Hopfield (and Hopfield-type) ANNs has been undertaken with the intrinsic presupposition that the interconnection (weight) matrix for the network under consideration is symmetric [5].
This paper looks at the prospect of obtaining the solution of a system of n simultaneous linear equations in n variables using HNN-based approaches. Section II contains an recapitulation of Hopfield's original proposal together with a deliberation on the reasons which lead to the standard HNN being not amenable for the purpose of solving sets of linear equations. The next section, i.e. Section III contains the circuit and explanation of a modified HNN being applied for obtaining the solution of linear equations for the particular case wherein the coefficient matrix for the system of simultaneous linear equations is symmetric. Computer simulation results using PSPICE software for different sets of linear equations (having symmetric weight matrices) are also included. Section IV highlights the application of a modified HNN for solving linear equations, in the case where the coefficient matrix A, corresponding to the set of equations, is asymmetric. Issues pertaining to the convergence and stability of the asymmetric HNN are also pondered upon. Section V presents the results of computer simulations for the proposed aysmmetric neural circuit employed for the solution of different sets of simultaneous linear equations of assorted sizes. Section VII contains remarks related to the limitations, hardware complexity, applicability and practicability of the proposed network. Section VIII presents some concluding remarks.
II. THE HOPFIELD NETWORK It is well know that the behaviour of a typical neuron (a genearlized i-th neuron in HNN is shown in Fig. 1) is modeled as a differential equation: where C i > 0, R i > 0, and i i are the capacity (modeled electrically as capacitance), resistance value, and bias voltage applied to the neuron, respectively, and u i and v i are the input state and the output state, respectively, of the i-th neuron and g i (.) is the characteristic function defining the i-th neuron [11]- [13]. W ij are the individual elements, each having a different {i, j} combination, of the weight matrix W (which in turn decides the interconnections amongst the multiple neurons in the overall system). These weights are realized using appropriately valued resistors, R ij (= 1/W ij ), in the equivalent electronic (hardware) realization of the HNN, as has been illustrated in Fig, 1. It is to be noted that Hopfield performed his seminal study on such HNN networks with the following assumptions: and The assumptions mentioned above have the following implications on the hardware realization of the HNN.
• Equation (2) signifies that self-interactions amongst the neurons (self-feedbacks in the hardware realization) are not allowed in the original version of the HNN. • Equation (3) places a more rigorous condition on the allowable synergy between the various units in the HNN in the sense that only symmetric connections (feedbacks) are allowed. This leads to the restriction that the HNN weight matrix W (=(W ij ) n×n ) has to be symmetric in nature. It has indeed been proven that, for such type of cases, where self-interactions are disallowed (i.e. no self-feedbacks, because W ii = 0) and a symmetric W exists (i.e. W ij = W ji ), the HNN is totally stable [14]. Under these restrictive conditions, the Lyapunov energy function defined for the HNN can be written as The last term in (4) is usually neglected because of the following reason: it is only consequential near the saturating values of the operational amplifier used to emulate the neurons [15]. A 3D plot depicting the shape of a typical energy function, is shown in Figure 2, for an HNN with the following Fig. 2. Representative Lyapunov energy function plot for the standard HNN when applied with 2 neurons for the special scenario of a symmetric weight matrix W with all diagonal elements being equal to zero specifications: 2-neuron system, symmetric weight matrix, all zeroes as diagonal elements in W (fulfilling the requirements laid out in (2) and (3)), and no external bias i i . It can readily be seen from Figure 2 that the stable states will be at the corner points of the virtual hypercube [-V m ,V m ] where ±V m are the DC biasing supply voltages (±12-15V for the µA-741 operational amplifier) of the opamps that are employed to realize the activation functions of the neurons.
Therefore, from the above discussion it is sufficiently clear that the standard HNN is unsuitable for the goal of solving linear equations, due to the points reproduced below: • Equation (2) requires that W must always have all zero diagonal elements • Equation (3) dictates that W must always be symmetric in nature • The stable point(s) i.e. minima of the Lyapunov energy function E, obtained for the standard HNN, have been shown to occur at the corners of the hypercube, and it is not possible to shift these minima to lie at a particular point of interest, for instance, as is required if we want to use the HNN for solving linear equations -we would want the minima to coincide with the mathematical solution point of the set of equations under consideration. From the points noted above, it is quite evident that in order to make the standard HNN amenable for linear equation solving, some appropriate modifications would be required. One straight-forward modification, for instance, is the interchanging of the non-inverting and inverting inputs of the amplifiers in a standard HNN. The effect of this simple change is quite drastic on the energy function (4), and will cause it to become: The significance of the sign inversion in the energy function will become more evident when the above modification is coupled with the allowing of self-feedback (self-interactions) amongst the neurons i.e. relaxation in the restrictive condition of (2). These two modifications cause the minimum point of the Lyapunov energy function to now appear at the centre of the hypercube, and the same is presented in Figure 3 for a pictorial comparison with the conventional HNN energy function shown in Fig. 2. Although an appropriately altered HNN, as has been discussed above, can be tailored to acquire a single (unique) minimum in the Lyapunov energy function, that stable minimum equilibrium point can only exist at the centre of the hypercube i.e. at v i = 0, for all i. This leads to the following observation. The modified HNN is still not useful for being employed for the purpose of solving linear equations. This is because, for an ANN circuit to be able to do that, the minimum point in the Lyapunov energy function must coincide with the algebraic solution point of the set of linear equations. Such a solution point may be anywhere on the hyperplane, (and not compulsorily at the origin). The subsequent section considers some additional modifications/conditions to make the HNN applicable for the task under consideration in this paper.

III. SOLVING LINEAR EQUATIONS USING A MODIFIED HNN
The present section deals with details of the modifications that are required in order to make the HNN (elaborated in the previous section) suitable for the purpose of providing the solution of different SLE. Let the system of n SLE in n variables to be solved be represented as where V 1 , V 2 , V 3 , . . . , V n are the unknowns and a ij and b i are coefficients and constants respectively, from the set of linear equations under consideration. Since a voltage-mode (VM) linear equation solver is presented in this work (meaning that the outputs of the circuit will be voltages equal in magnitude to the algebraic solution), the decision variables are labelled as voltages V 1 , V 2 , V 3 , . . . , V n in order to match the output states of neurons in the proposed circuit. The proposed network is based on the assumption that the coefficient matrix A is invertible (nondegenerate). This means that the system of LE (6) is not under-determined and is consistent. This also means that the linear system (6) has a uniquely determined solution. Moreover, there is another restriction that needs to be imposed considering the electronic hardware implementation of the HNN. The solution must not fall beyond the operational region (in terms of voltages) of the circuit i.e. inside the hypercube defined by |V i | ≤ V m (i = 1, 2, 3, . . . , n). This restriction is due to the fact that the output obtained from an operational amplifier (which is used to emulate the neuron) cannot go beyond its biasing voltage. The generalized i-th neuron of the VM modified HNN based circuit for solving (6) is depicted in Figure 4. It needs to be mentioned that the output voltage of the generalized neuronal amplifier is now labelled as V i (in place of v i in Figure 1) to comply with the notation used for the unknowns i.e. the decision variables in (9). C pi and R pi are the parasitic capacitance and parasitic resistance respectively, of the voltage amplifier emulating the i-th neuron. These circuit parasitics have been considered for modelling the time-varying (dynamic) nature of the output characteristics of the voltage amplifier. As is evident from Figure 4, individual equations from the SLE (6) have been scaled (by a factor s i ) prior to their application as inputs to the HNN neurons. This scaling is essential for ensuring that all [a ij /s i ] coefficients are less than unity, which in turn means that these could be realized in hardware with passive voltage dividers implemented with discrete resistors. As will be elaborated subsequently , the magnitudes of the individual scaling factors (s i ) could be selected independently for each equation. If that is the case, the scaling factor for the i-th LE, s i , should adhere to: All equations in the set to be solved could also be scaled by the same magnitude s, which should be the highest amongst all s i selected for individual LE separately i.e.
Considering the circuit of Figure 4, and writing the node equation for the node marked 'A' (i.e. the inverting input of the operational amplifier) yields the equation of motion for i-th neuron in the following form: where u i represents the internal state in the i-th neuron, and For obtaining a valid Lyapunov energy function for the modified HNN depicted in Figure 4, we should proceed in the following manner. The association between V i and u i for the opamp (emulating the neuron) in Figure 4 is given as which can be rearranged to the following form Further, for dynamical systems such as the one depicted in Figure 4, the gradient of the Lyapunov energy function E is concomitant to the dynamical progression-in-time of the circuit, as represented by (16) [16].
Using (12,16), along with (3), the energy function E corresponding to the modified HNN circuit of Figure 4 can be written as The last term in (17) is insignificant when compared with the first two terms, and is therefore usually neglected in further analyses [15], [16]. Thus, the Lyapunov energy function may further be written in a reduced and simplified form as: From (18), it is clearly evident that the minimum (and therefore the most stable point, to which the circuit would converge) of E will also be dictated by the individual constants contained in the vector B in (8). This goes on to show that the minimum point will now not (always) appear at the centre of the n-dimensional hypercube, which is in contrast to the energy function derived in (5) which necessitated the minima to lie at the origin.
The stationary point (minima) of E (18) may be obtained by setting from where, we get The set of linear equations of (6), after scaling, can be written in a generalized form as For the equilibrium point (minima) of E given in (18), which was derived in (20), to overlap with the solution of the SLE (21), the magnitudes of the resistors in the circuit may be obtained as follows: The value of the individual resistors R i , may be computed using (20,21) by equating their respective last terms, and resulting in 1 which can be expanded to Also, since the value of R pi is much greater than all other resistances appearing in (24), R pi can readily be neglected when the equivalent value of all the resistors (which appear in parallel configuration) connected to the i-th input node (inverting terminal of the i-th opamp) is computed. Therefore, we have Putting the values of resistances R ij obtained from (22) It is evident that, the constraint being enforced on the individual scaling factors (in (10)) can also be derived from (26), because since selecting a vlaue of scaling factor which violates (10) is bound to result in a negative resistor value from (26).
Referring to (22), the magnitudes of the weight resistors in the modified HNN when used to solve SLE can be computed as . . .
The full generalized version of the symmetric HNN employed for solving n equations in n variables obtained by using the neurons rendered in Figure 4 is illustrated in Figure 5.
For verifying that the modified HNN-based equation solving circuit of Figure 5 is indeed capable of converging to solutions of SLE, the network was tested using PSPICE simulations.   Fig. 7. Computer simulation results using PSPICE, for the circuit of Figure 6 employed to obtain solution for (28) and The circuit to solve the 2 SLE in 2 variables (28), drawn using the generalized network of Figure 5, is illustrated in Figure 6. It can be readily found that the mathematical solution for (28) is V 1 = −2 and V 2 = 3. Results of computer simulation (using PSPICE software) using the resistor values provided in (29, 30) are included in Figure 7. It can be seen that the resultant output node voltages (measured in Volts) converge to V(1) = −2.00 and V(2) = 3.00, which correspond perfectly with the mathematical solution.

IV. SOLVING SLE WITH ASYMMETRIC HNN (AHNN)
The preceding section dealt with a discussion on how the standard HNN may be appropriately modified for the solution of systems of n SLE in n variables, although with the strict restriction that the coefficient matrix A is symmetric. This assumption, leading to W being symmetric, formed the bedrock of the development of the theory of stability of the HNN, as well as the presence of a valid/legitimate Lyapunov energy function. Customarily, weights are realized using resistances in the conventional electronic implementations of the HNN, and therefore the strict condition of symmetry of W dictates that meticulously controlled resistor values be used in the circuit realizations for preserving and maintaining the symmetry of W. However, when the HNN is employed for real-world (practical) scenarios using appropriate hardware implementations, it is impractical to expect, and impossible to guarantee, that all the neuronal interactions are symmetric in nature, since this will require ensuring that two electrical quantities (i.e. values of resistances, voltage gains of the opamps used as neuronal amplifiers) are of the exact same value. Therefore, a physically implemented HNN circuit, would in all probability, actually turn out to be one with an asymmetric weight matrix. Furthermore, as Vidyasagar has aptly pointed out in his work [14]: "The consequences of even slight asymmetries in the interactions are disastrous to the theory of stability as put forward by Hopfield" a substantial quantum of research endeavor has already been targeted towards the study and understanding of the qualitative properties of oscillation, convergence, and stability for AHNN. [5], [9], [10], [14], [17].
In the existing technical literature, the sufficient conditions which can guarantee that an AHNN has a unique (exponentially) stable equilibrium state, have been presented in several different manifestations: • diagonally row or column dominance property in W [10] • bounded-ness of the activation functions of the neurons [ [22] In addition to the above, the presence or absence of a unique equilibrium and the consequent stability of an AHNN can also be ascertained with a new collection of simplified sufficient conditions [9]. The condition for which a normalized AHNN is stable, as has been shown mathematically in [9]. The same has been reproduced here for ready reference of the readers. As a precondition, first the M-matrix is defined as follows.
Let T (=(t ij ) n×n ) be a square matrix which has nonpositive off-diagonal elements. Now, if each of the leading principal minors of T is positive, the matrix T is referred to as an M-matrix [9].
The following steps are therefore required to determine whether an AHNN has a unique equilibrium and is stable: 1) A square matrix T (=(t ij ) n×n ) is first obtained from W using (31).

2) If T is a valid M-matrix, then the AHNN pertaining to
W is declared to be globally asymptotically stable [9]. , It is therefore clear that (31) puts some 'restrictions' on W. This can be understood as follows. An AHNN will not necessarily possess a unique equilibrium for all W. Instead, only those interconnection matrices W, from which the counterpart T matrix obtained using (31), turns out to be a valid M-matrix, are admissible. This restriction on W leads to a limitation in the application of AHNN to real-world problems. This is also true for the linear equation solver circuit presented here, where it is observed that the proposed network is capable of solving only those SLE for which W satisfies the conditions discussed above.
In the succeeding section, the application of an AHNN for the solution of sets of SLE of varying sizes is presented. It should be mentioned at this juncture that although the prospect of the application of AHNN for the solution of SLE has always been in existance since the advent of the standard HNN itself, a thorough search of the technical literature does not reveal any such attempt. To the best of the author's knowledge, the work contained in this paper is the first pursuit to exploit AHNNs for the purpose of obtaining solutions to systems of SLE.

V. CIRCUIT SIMULATION RESULTS
The correct operation of the AHNN used for solving linear equations was verified with extensive PSPICE simulations. Different sets of linear equations, starting with 2 and going up to 20 variables, were generated, and solved by the proposed AHNN circuit. Results obtained from the computer simulations are presented in this section, and were found to be in close agreement with the mathematically obtained solutions, for all the equation sets tested.
The circuit of Figure 6 was used to obtain the solution of the following simultaneous linear equations in 2 variables: The values of the resistors (in KΩ) as calculated from (22) and (26) for the example set of 2 linear equations (32) are presented in (33, 34). The magnitudes of the scaling factors for the two equations were chosen to be s 1 = 6 and s 2 = 21 respectively. It should be noted that the scaling factors are to be chosen such that the resulting values of the resistor(s) are realistic and reasonable. For instance, choosing a very low value of the scaling factor (< 1) would result in a very low value of the resistances when the coefficients in the linear equations are positive integers higher than unity. Similarly, selection of a very high value of the scaling factor(s) would result in high values of the resistors, which is inadvisable from an integrated circuit fabrication viewpoint. and Results of the PSPICE simulation test for the circuit of Figure 6, which is used to solve (32) using the resistor values shown in (33) and (34) are depicted in Figure 8 from where it can be observed that the obtained (stable, settled) voltages at the output nodes are V(1,2) = (1.01, −2.01). These match  Table I. It can be readily observed from Table I that the AHNN can provide correct solutions for all the chosen sets of SLE for which the criterion of stability (discussed in Section IV) is fulfilled. It is important to note that each of the matrix T shown in Table I (for the different cases) is calculated by utilizing −A (instead of A). This is required since the criterion of stability presented in Section IV is mathematically obtained for the case when the amplifier emulating the neuron in the AHNN has a monotonically increasing activation function, whereas for the case of the circuit of the AHNN that is actually applied to solve the different sets of SLE, the opamp which is used in the inverting configuration, exhibits monotonically decreasing transfer characteristics.
A system of 3 linear equations in 3 variables (35) was next solved using the proposed AHNN. Values of the different resistors required in the AHNN, as calculated using (22) and (26) for the chosen set of 3 SLE in 3 variables (35) are presented below. All the three LE were scaled by s 1 = s 2 = s 3 = 10.
The proposed AHNN circuit was employed to solve (35) using the resistor values provided in (36) and (37). Results of PSPICE simulations are depicted in Figure 9 from where it is evident that the output node voltages (in Volts) obtained are V(1,2,3) = (1.00,−1.99,1.99) which are in close agreement with the exact algebraic solution of (35) which is V 1 = 1, V 2 = −2 and V 3 = 2.
Next, different sets of 3 SLE in 3 unknowns were solved with the proposed AHNN. The outcomes for each of the Fig. 10.
Results of computer simulations for the AHNN employed for obtaining the correct answers for the first set of equations in Table III Fig. 11. Results as obtained after simulations, for the AHNN being used to solve the 10-variables set of equations listed in Table IV Fig. 12. Results that were obtained after simulation, for the AHNN put to solve the 20-variables linear equation system listed in Table IV  TABLE I  RESULTS OF THE AHNN USED TO CONVERGE TO SOLUTIONS OF VARIOUS SETS OF LINEAR EQUATIONS IN 2 VARIABLES * the stability criteria is as elucidated in Section IV different set of SLE, using circuit simulations, are shown in Table II. It is seen that the circuit is indeed capable of providing correct solutions for all the cases (for which the stability criterion elaborated in Section IV is satisfied).
To test the AHNN further, different sets of 5 SLE in 5 variables were then solved with the proposed network. The results of PSPICE simulations for each of the chose set of equations are tabulated in Table III from where it is clearly evident that the proposed AHNN converges to correct solutions for all the chosen sets of equations (provided that the stability criteria mentioned earlier is fulfilled).
Next, to test the scalability of the proposed AHNN for higher number of variables, the proposed circuit was then tested for the solution of SLE in 10 and 20 variables. The results as obtained from the circuit, using simulations with PSPICE, are provided in Table IV from where it can be seen that the AHNN does indeed outputs accurate solutions for both the large sets of SLE (10 variables and 20 variables). The voltage output plots (as obtained from PSPICE) for the AHNN configured for the 10 and 20 variable SLE are depicted in Figure 11 and Figure 12 respectively. From these plots, it is readily evident that all the output voltages initially start from zero, and eventually converge to their respective 'solution' values.

VI. HARDWARE TEST RESULTS
Although the results of the extensive circuit simulations presented in the previous section validate the proposed AHNN, further confirmation of the network functionality is demonstrated by performing hardware realizations of the AHNN for small-sized problems, using commercially available electronic components. In addition to the verification of expected functionality of the proposed AHNN, such actual breadboard implementations using discrete electronic components also provide an opportunity to test the convergence of the AHNN to the desired (mathematical) solution, commencing the convergence from diverse initial conditions (i.e. different/random internal node voltages in the circuit -as would be the case when the circuit is operated in a real-world scenario). The noise present in any electronic circuit (e.g. Johnson noise, Shot noise) causes the circuit to 'start-up' from a random initial condition every  time it is powered on. Stock electronic components viz. the µA741 operational amplifier and axial-lead resistors have been used for the breadboard implementations.
The proposed AHNN was first tested by setting up the hardware for the solution of the two equations (38). A photograph of the experimental set-up, and the obtained results, both on a digital Cathode Ray Oscilloscope (CRO) and digital multimeters, is included in Figure 13, from where it is readily seen that the obtained (stable, converged) values (in Volts) of the outputs of the 2 neurons are (0.99, −2.96). These values when compared with the actual solution (1, −3) highlight the correct functionality of the breadboard realization.
Different examples of linear equation sets (with variables ranging from 2 to 4) were then chosen and the AHNN hardware for each was then set up in the laboratory using commercially available discrete components. Each of the circuits was run 10 times so as to see whether the circuit converges to the same solution every time after being started up with varying (distinct) initial conditions (due to the inherent noise which is always present in the system and cannot be eliminated). Outcomes of the hardware implementation tests are tabulated in Table V Table V, and it can be seen that the hardware implementations also provide solutions very near to the mathematically exact solutions (as was the case with the PSPICE simulation results). The maximum error in the output value is 2%.

VII. DISCUSSION
This section contains a concise discussion on the applicability, hardware complexity, the strict restrictions on the values of resistors, and the scalability of the circuit.

A. Applicability
As has been discussed in previous sections, the proposed network is applicable only for those systems of equations which strictly satisfy the criterion of stability laid out in this work. Although the stability definition in this paper is presented in the form of T and M matrices, some other versions of the stability conditions are easier to comprehend. For instance, the presence of diagonally row (or column) dominance in the interconnection matrix W is the simplest form amongst the several stability criteria listed in Section IV. As is well known, a given square matrix A can be referred to as a diagonally dominant matrix if, for each row of A, the magnitude (absolute value) of the diagonal entry in that particular row is greater than, or equal to, the total of the absolute values (magnitudes) of all the other entries (i.e. the non-diagonal elements) in that row. Algebraically this can be written in the following form. A given matrix A is diagonally where a ij denotes the elements at the location specified by the i-th row and j-th column.
The above phenomenon of diagonal dominance in the coefficient matrices can be readily verified by observation of Table III and Table IV. It is to be noted that as the size of the coefficient matrix increases, the diagonal elements need to be more and more large in magnitude to satisfy (39).
It is well known that several matrices that appear in finite element methods (FEMs) analyses are intrinsically having the diagonally dominance property. Further, it has also been established that sets of linear partial differential equations generated from the Boundary Element Method (BEM) can be solved iteratively, with assured convergence to the accurate solution, if the BEM equations can be first recast into an analogous, diagonally dominant set of equations [23]. In the light of these applications, the proposed network assumes significance where a fast and efficient solution technique is required.
In view of the limited applicability scenarios for the circuit presented in this work, alternative analog hardware techniques (which may or may not be neural networks based) are required for designing a generalized network which is capable of generating valid & correct solutions for all sets of linear equations (irrespective of the satisfaction of the stability criteria as mentioned in this paper), provided that a unique (single) solution to the systems of equations under consideration exists. Some such neural circuits which are capable of this functionality, albeit at the cost of increased hardware complexity, are discussed in [3], [4], [24].

B. Hardware Complexity
AS can be deduced from Figure 5, the following set of hardware is required for the proposed circuit. For solving an n variables set of equations, n operational amplifiers, n 2 + n resistors, and n + 2 DC voltage sources are required. Amongst the n + 2 DC voltages mentioned, n sources are required for implementing the constant inputs b i /s i for connecting to the non-inverting input terminals of the opamps, and 2 voltage sources are required for biasing the opamp (±V m ). In an actual integrated circuit implementation, the n voltages may be generated using a potential divider type arrangement, or a multi-output reference voltage supply may be employed.
since the stability and convergence of the circuit is dependent upon the values of the resistances, it is important to have precise valued resistors (with negligible thermal drift) in a real-world implementation of the proposed circuit. This can be achieved by (i) laser trimming of the integrated resistors [25], or (ii) by using voltage-controlled electronic resistors [26]. While the former will lead to a circuit hard-wired to solve a certain set of equations, the latter may be a better choice for imparting operational flexibility to the proposed circuit.

C. Scalability
While the proposed network is theoretically able to solve n linear equations in n variables, provided the condition of stability is satisfied, there are going to be limits on n in an actual implementation of the circuit. First, the requirement of the n 2 + n precise resistors puts a restriction on the extent of integration possible for a chip implementation. Secondly, the requirement of n voltage supplies to provide the b i /s i inputs to the various neurons also marks a limit that would be defined by the hardware availability. Lastly, the noise floor will also play a role in dictating the maximum number of outputs (and therefore the number of variables n) allowed.

D. Speed
Most of the discussion contained in the Section V was centered around the accuracy of the obtained solutions and not on the time taken by the circuit to converge to the solution. As can be seen from the simulation results included in Figure 9 through Figure 12, the network converges to the correct solution(s) in time of the order of microseconds. This convergence time is obtained when the simulations were performed using a IC-741 PSIPCE macromodel 1 . This PSPICE model was used to demonstrate the workability of the proposal, and is in no way a reflection on the actual convergence speed that will be available when modern day fast opamps (such as Linear Technology's LTC6253-7 2GHz operational amplifier 2 ) are employed.

VIII. CONCLUSION
An HNN-based circuit (referred to as the Asymmetric Hopfield Neural Network) was presented which was capable of providing the solution of a system of n linear equations in n variables. Since the weight matrix (W) of the AHNN is dependent on (and derived from) the coefficient matrix (A) of the set of equations, it therefore cannot be said to be axiomatically symmetric. The AHNN circuit therefore falls in the class of asymmetric HNNs [5], [17]. Since the asymmetry of W imposes certain limitations on the stability of the AHNN,


Integrated Model to Develop Grammar Checker for Afaan Oromo using Morphological Analysis:

This study has implemented a rule-based approach on grammar checkers by integrating a spell-checker with a morphological analyzer to improve the Afaan Oromo grammar checker. A corpus containing about 300,000 words has been prepared to be used for spell-checker. About 300 grammar rules are constructed to detect the grammar error within the Afaan Oromo text and to suggest the possible grammar correction. The developed frameworks have experimented on the document having pairs of 100 correct and incorrect sentences. The experimental result for checking the spelling errors has scored 73% of recall, 76% precision, and 75% of F-measure. The score for suggesting the correct spelling is 78% of recall, 62% precision, and 70% precision F-measure while the evaluation result for detecting the grammar errors has 47% recall, 90% precision and 68% f-measure score. For suggesting the possible correct grammar on the detected error, the system has scored 61% recall, 71% precision and 66% f-measure. The overall performance of the developed system has a good performance. However, there is still a need to conduct further research to improve the Afaan Oromo grammar checker. Keywords—Grammar checker; spell checker; part-of-speech tag; error detection; syntactic analysis; semantic analysis; morphological analyzer; NLP

I. INTRODUCTION
For communication, a natural language is a language used by humans. Natural language processing (NLP) is a field of study mainly concerned with the communications between human languages and the computer. During a written communication, people may make an error which can be a spelling error and/or a grammar error. A spelling error is an error if the given the word does not exist in the language's vocabulary, whereas a grammatical error occurs when the written sentence is not as per the grammar rule of a given language [1].
With globalization, many individuals work on National and International languages for everyday communications. But it is difficult for many to write various contents using correct spelling and grammar, whether professional communication or a regular discussion, which makes grammar checking one important tool in the word processing software. Grammar checking has significance for having a good flow of ideas, exactness and quality of the expressed written content [1]. Grammar checking is one of the activities in the natural language's written communication and NLP application.
Grammar checking checks the grammatical errors in text and suggests possible corrections in many cases. This is one of the most frequently used tools in language engineering.
One of the main challenges in the application of grammar checking for natural language is it is language-specific. Due to this, the grammar checker that works for one language may not work for another language. Therefore, it's necessary to design a grammar checker for each language as per the grammatical rule of a particular language.
Afaan Oromo is one of the most widely spoken languages in east Africa, which accounts for about 40 million speakers in Ethiopia; it is also spoken in neighboring Ethiopian countries like Kenya, Somalia, Djibouti and Egypt [2].
Today, Afaan Oromo is an official working language of Oromia national, regional, and educational languages for primary school (1-8) students in the Oromia region. As a result, many works for official and/or personal purposes use Afaan Oromo for interpersonal communication. Most of the time, while preparing various reports, peoples are vulnerable to make a grammatically wrong statement unintentionally or unknowingly. Some languages, such as English, French, etc., have a tool to catch and make corrections on these problems. However, Afaan Oromo is one of the under-resourced languages. Due to this, there are no tools that can assist the users in catching and making corrections on the grammatically wrong statements. Therefore, it is required to develop and implement the Afaan Oromo grammar checker to eliminate these problems. Developing a grammar checker for Afaan Oromo is significant for non-writers and nontechnical peoples to have accurate and quality written content. This study is intended to design and develop an integrated model to develop a grammar checker for Afaan Oromo using morphological analysis -A Rule-Based Approach.
II. GRAMMAR CHECKER APPROACHES There are three widely used approaches by various researchers like Syntax-based approach, statistical-based approach and rule-based checking [3].
• Syntax-based approach: In this approach, text parsing is used by assigning a tree structure to the sentence. If the text parsing mismatches, the statement is labeled as a grammatically wrong sentence; otherwise, it is considered correct.
*Corresponding Author 209 | P a g e www.ijacsa.thesai.org • Statistical-based approach: In this approach, a corpus annotated with its part of the speech tag is prepared and used to build a sequence of parts of the speech tag list. If the frequency of the tagged text is less than with the already trained model, this text is considered incorrect otherwise it's it is considered correct.
• A rule-based approach: In this approach, a set of grammar rules has been manually prepared to match against a text. If the text matches the crafted rules, it is considered a correct otherwise wrong sentence.

III. RELATED WORKS
Various studies have been conducted on natural language processing for the Afaan Oromo language. For instance, [4] have designed a spell checker for non-word Afaan Oromo spell checker. This work is focused on checking misspelled words and use the proposed algorithm to observe if the right word is generated under suggestion. But authors are not considering the context of the statement in their work.
The author in [5] attempted to design a grammar checker for Afaan Oromo using a rule-based approach. The author constructed about 123 rules for the detection of the Afaan Oromo grammar error. The rules are constructed depending on the language rule of affixation. However, false alarms in the developed framework lead the system to become challenged to catch the grammar error correctly. The author in [5] has mentioned some of the reasons for the problem that occurred. These are some words root and affix are identified wrongly by the stammer, a wrongly assigned part-of-speech tag and incompleteness of the rules. However, there are many reasons for the problems with [5] framework in addition to the reasons mentioned. Rules ranging from 81-86 were constructed to catch the error of past perfect tense. But this system may falsely alarm the correct grammar as wrong due to incorrect rules. Let's have a look at the following example, which tests his rule. One of the rules which are crafted by [5] says, "If the subject of the sentence is third-person singular masculine, so the verb must end with the suffix -ee, and the sentence must end with ture." Example: Callisee Bira dabruu yaalee ture. (He tried to pass in silence).
But the above rule may flag other correct statements as a grammatically wrong sentence. Because, in Afaan Oromo, if the subject of the sentence is third-person singular masculine, but the verb may end with the suffix other than -ee and also it's not necessary for the sentence to end with 'ture'." Example: Callisee Bira dabruu yaalus hin milkoofne. (He tried to pass in silence, but he did not).
The author in [6] has developed the Afaan Oromo grammar checker by using a statistical approach. The author has implemented two statistical approaches: the token n-gram frequency whose probability of sequence is calculated and the tag n-gram frequency whose POS tag is assigned to all tokens, and the probability of the tag sequence is calculated for training and testing the checker. However, [6] didn't incorporate the checking of spelling errors in his work which is fundamental to the work of grammar checkers. The corpus used for training contains only 10000 words due to this POS tagger, and the Morphological analyzer has become inaccurate. Additionally, the article didn't highlight constructing a rule that will be used for training the model and improving the model performance. As a result of the factors mentioned, the work proposed in the paper has to be improved by incorporating spell checking along with a good morphological analyzer.
Even if few works have been done on grammar checkers for the Afaan Oromo language, there is still a gap in developing a good grammar corrector that can help to fix common spelling errors, instances of the passive voice, clunky and hard to understand language.

A. Material Used
Various development tools have been used to design the Afaan Oromo grammar checker. Python is one of the tools that were used to develop the prototype. Python is used because it's easy to work with, learn and adaptable scripting language, making it attractive for development [7]. HORN MORPHO 2.5 is used for POS tagging activity. It is a program that analyzes Amharic, AO, and Tigrinya words into their constituent morphemes (meaningful parts) and generates words, given a root or stem and a representation of the word's grammatical structure [8]. The proposed research has also used the Hunspell dictionary to store the required grammar correction rules and integrate the developed framework into Apache OpenOffice.

B. Design Approach
There are quite a lot of ways in which diverse approaches to grammar checking can be illustrated. This study intended to develop a grammar checker for Afaan Oromo using a rulebased approach by integrating with spell-checker and morphological analyzer. The dictionary containing about 300,000 unique words is constructed that can help to detect spelling errors. For grammar checking, a set of grammar rules has been constructed, which accounts for about 250 rules that check the errors in the grammar.
To perform the grammar checking, the developed system will find a word from the sentence. Then the tag of the word is checked against the developed grammar rules to detect various errors such as style problems, agreement in gender, number, word order, etc. If the sentence contains an error, the suggestion to the error will be displayed. The designed framework has two components, namely the spell-checker component and grammar checker component.

1) Spell-checker components:
The spell-checker components have a set of words and affix rules of Afaan Oromo languages. This component helps to catch the spelling errors from the texts and suggests the possible correction to the wrong words depending on the morphological rules of the Afaan Oromo languages.
2) Grammar checker components: This module consists of a set of grammatical rules of the Afaan Oromo languages. This component works by checking the arrangements of words in a given sentence. The crafted rules check the syntactic agreement features in the statement, i.e., the agreement in the 210 | P a g e www.ijacsa.thesai.org gender (Masculine or Feminine), number (Singular or Plural), a person (1 st /2 nd /3 rd person singular, 1 st /2 nd /3 rd person plural) and proofreading (punctuation). In Fig. 1, some of the sample rules are constructed.

1) Implementation:
The developed framework has been implemented using a Libreoffice text/document writer. LibreOffice is one of the document writer software supporting the integration of the Afaan Oromo spelling and grammar checker. Fig. 2 shows a list of extensions already in the Libreoffice software before installing the Afaan Oromo spelling and grammar checker.
As shown in Fig. 2, installing the developed framework as an extension of writing aids, there is no indicator of the Afaan Oromo language. However, after installing the Afaan Oromo spelling and grammar checker, the writer started to indicate an icon showing the language extension for grammar and spellchecking is installed. As shown in Fig. 3, the layout for the language extension is highlighted with a red color.  The developed framework has been implemented to perform spelling checking, correction, grammar error detection and suggestion.
a) Spell-Checker Implementation: At this phase, the system is implemented to detect the wrongly spelled words and suggest possible corrections. Fig. 4 shows the implementation of the spell-checker. To indicate the wrongly spelled word in a sentence, all wrongly spelled words are underlined with a red flag. But if the word has no spelling error, no sign is displayed. As shown in Fig. 4, the wrongly written words are flagged with a red color underscore. When these wrongly written words are rightclicked, the possible suggestion is shown. b) Grammar-Checker: This implementation was done to determine the wrongly written statements and extract the grammatically wrong statements within the text by the system. The prepared corpus (doc1 and doc2) is used alternatively for error detection and false alarm identification in this implementation. Fig. 5 is the screenshot of the implementation of detection and correction of the grammar of the proposed framework. Fig. 5 shows how the system detects the grammar error and suggests the possible correction.
The statement that has a grammar error will be flagged with a blue underscore to make the user alerted and take proper action. The grammar suggestion will be shown when rightclicked on the flagged text. However, those statements written with a correct grammar rule will not be flagged. 2) Evaluation: The developed system is evaluated to check whether the objectives of the research are achieved or not. To evaluate the performance of the systems, two documents named doc1 and doc2 are prepared. These documents are used for testing and evaluation purposes, where each document contains about 500 sentences. All of the words and sentences within the first document (doc1) are 211 | P a g e www.ijacsa.thesai.org grammatically correct, whereas the second document (doc2) contains the possible wrong spelling and grammar of doc1. The evaluation task is done in two phases, and the first phase evaluation is applied to test the detection and suggestion capability of the system for the spelling error within a sentence. The second phase is evaluated to check the system's performance for grammar error detection and correction suggestions made to the identified error. a) Evaluation of Phase 1: The performance of the system for spelling checking has been evaluated at this phase. The document with 50 words has been prepared, and of these, 25 words were wrongly spelled while the rest 25 correctly spelled. The system is expected to suggest at least three correctly spelled words related to the detected spelling error.
b) Evaluation of Phase 2 • Grammar Error Detection: The framework is tested against doc1 to identify the grammatically wrong statements within the text, whereas doc2 is used to handle false alarms.
• Correction Suggestions: The correction suggestions evaluation has been done to evaluate the system's performance for suggesting the possible correct grammar of the detected errors. The system is tested by using the document containing the wrong grammar statements (doc1). Depending on the error detected, the number of correction suggestions may vary. However, the system is evaluated to suggest a minimum of two correct suggestions per error.

3) Confusion matrix:
To demonstrate the performance of the proposed system, a confusion matrix has been used. The confusion matrix is a commonly used technique to describe the classifier's performance based on test data [9]. Table I shown below, describes the confusion matrix result for the system performance.
The performance measure shows the detailed description for the activities of the systems such as; checking the spelling error, suggesting the correction for the spelling error, grammar checking and correction are shown in the Fig. 6, 7, 8 and 9.
Evaluation matrix performance can also be explained using f-measure, precision and recall values. Following are the formulas used for these calculations.   212 | P a g e www.ijacsa.thesai.org

VI. DISCUSSION
The developed system has been evaluated on four basic tasks: checking spelling errors within the text, suggesting the correct spelling for the detected error, checking grammar errors, and suggesting the correct grammar for the identified grammar errors. Accordingly, the confusion matrix result for checking the spelling error has scored 51.35% TP, 13.51% FP, 18.92% FN, and 16.22% TN, whereas spell-checker correction suggestion 45.90% TP, 12.30% FP, 13.11% FN and 28.69% TN score. But, for checking the grammar error and suggesting the possible correct grammar, the system has scored 43.88% TP, 2.04% FP, 48.98% FN and 5.10% TN result of confusion matrix measurement.
The performance evaluation result of the system, which is evaluated by the confusion matrix measurement, is described by using the f-measure, precision and recall values. So, for checking the spelling errors, the score is 73% recall, 76% precision and 75% f-measure. To suggest the correct spelling for the detected error, it has scored 78% recall, 62% precision and 70% f-measure. For checking grammar errors, the system has scored recall 47%, precision 90% and f-measure 68%, whereas, for the task of suggesting the correct grammar for identified error, it has 61% recall, 71% precision and 66% fmeasure.
According to the experimental result presented, the framework has performed well for handling false alarm issues than error detection. The main reason that false alarm handling tasks scored higher than error detection tasks is that the written rules are primarily focused only on the common errors that mislead the meaning or make the statement meaningless. The evaluation performance for the task of grammar error detection within a statement scored lower performance. The problem with the task of error detection happened as a result of uncovered cases within the rule. Within a given statement, if the grammar problem is a semantic error, it will not be detected and/or corrected by checking only the sentence structures. To find and/or correct the semantic errors of the statement, it is must to analyze the semantic structure of the sentence. Therefore, it's possible to solve a challenge to detecting grammar errors due to the dynamic occurrences of the error.

VII. CONCLUSION
This study has developed an integrated model to develop a grammar checker for Afaan Oromo using a rule-based. Approach integrated a spell-checker along with a morphological analysis on Afaan Oromo grammar checker to improve its performance. Data has been collected from various sources such as Oromia Broadcasting Network, BBC Afaan Oromo and other sources used for designing and implementation purposes.
The experimentation was done in two phases: spellchecking & correction phase and grammar-error detection & correction phase. The system is evaluated on detecting wrong spelling and grammar from the text and the possible suggestion made to the error. As a result, the evaluation result for detecting grammar errors in a statement has a low score than other activities. One of the main reasons is due to the uncovered cases within the rule for detecting the error.

VIII. RECOMMENDATION
The designed prototype for improving the Afaan Oromo grammar checker has a promising result. Even if the model performance showed a good result, there is still work that needs to be done to improve the performance. The following are recommended as a future research direction.
• The grammar error can be syntactic, related to the structure of a sentence and/or semantic, which is related to the meaning of the sentences. But this work attempted to solve the grammars having only syntactic errors. Therefore, further research work has to be done on the Afaan Oromo Grammar checker that can check the semantic analysis of the sentences.
• Since it is difficult to catch all of the grammar errors using only a set of rules, it must implement another method. Therefore, machine learning methods combined with a rule-based approach may improve the performance of the Afaan Oromo grammar checker.


Classification of Hepatocellular Carcinoma Using Machine Learning

Hepatocellular Carcinoma (HCC) proves to be challenging for detection and classification of its stages mainly due to the lack of disparity between cancerous and non cancerous cells. This work focuses on detecting hepatic cancer stages from histopathology data using machine learning techniques. It aims to develop a prototype which helps the pathologists to deliver a report in a quick manner and detect the stage of the cancer cell. Hence we propose a system to identify and classify HCC based on the features obtained by deep learning using pre-trained models such as VGG-16, ResNet-50, DenseNet-121, InceptionV3, InceptionResNet50 and Xception followed by machine learning using support vector machine (SVM) to learn from these features. The accuracy obtained using the system comprised of DenseNet-121 for feature extraction and SVM for classification gives 82% accuracy.

Introduction
The existing work on Hepatic tumor is concerned with clinical data acquired through blood samples, urine samples and serum test, and non-invasive images like CT, MRI, PET and SPECT. The manual identification of cancer from microscopic biopsy images is subjective in nature and may vary from expert to expert depending on their expertise and other factors which include lack of specific and accurate quantitative measures to classify the biopsy images as normal or cancerous one. Stains such as Hematoxylin and Eosin (H and E stain) are used for better emphasis of the nuclei of liver cells. Based on the amount of stain absorbed by the nuclei, it can be classified into various types since nuclei size increases with the stages of cancer. The stain can also be accumulated on the tissues causing ambiguity to the pathologist. Such ambiguity in the images can be overlooked by an individual. Color normalization is done to highlight the nuclei for visually better features.
Normalization techniques discussed in the study [1] where the images are classified by their colors using K Means Clustering and JSEG segmentation In this method, the nuclei get segmented as a separate segment. Then it is passed onto the SVM classifier. This technique enables effective segmentation of colored images. Similarly JSEG segmentation technique has two phases: color quantization and spatial segmentation [2]. Color quantization is based on peer group filtering(PGF) and vector quantization to reduce the number of colors in the images. For addressing the drawbacks of JSEG method, contrast map and improved contrast map were obtained.This technique saw a significant improvement in detecting more homogeneous regions than that of JSEG method. Due to the inherent difficulty involved in obtaining liver cell images from the biopsies, Liangqun et al. proposed to use neural networks for feature extraction and SVM for classification [3]. This method aims at providing better efficiency from less number of images.
The findings of the study [4] demonstrated the capability of Convolutional Neural Network (CNN) to recognize distinct features that can detect tumor masses in a histopathological liver tissue image. The author proposed to implement the CNN model for segmentation and classification of different stages of HCC. However, the major drawback of using CNNs for the feature extraction process is that these models need large amounts of data to process. This is a huge challenge for the biomedical field as it is pragmatically difficult to have access to massive data. Moreover, feature learning is pertinent on the size, shape and degree of annotation of images which are not uniform across datasets.
Chen et al. developed a deep convolutional neural network to classify the lung tumor stage and predict the most commonly mutated genes in lung cancer tissue cells [5]. Ehteshami et al. also produced a promising result for the classification of breast tumors using deep learning techniques [6]. The author developed an algorithm to differentiate stroma invasive cancer and stroma from benign biopsies However, the deep learning models were applied to non solid tumors. Thus, it remains uncertain if they can produce the same accuracy when applied to solid tumors.

Proposed methodology
The workflow contains 4 modules as follows:

Data collection
The first phase involved collection of data from Dataset collected from Global Hospital, Perumbakkam, Chennai. In a span of 3 weeks, images were collected from the biopsies of 3 patients. The three types of cancerous images obtained during the data collection phase are well-differentiated, moderately differentiated and poorly differentiated. The total number of images collected is 687 whose split up is given in Table 1.
Below are some images from the dataset collected, Figures 1-4.

Color normalization
The features of the nuclei include the texture, size and roundness. Applying a stain on these biopsies cause the nuclei to be highlighted due to absorption of the stain. The color difference between the nuclei and the tissues may be visually comparable or less different. Hence, color normalization is done to highlight the nuclei. Highlighting the nuclei makes it easier to extract the features from them.   The normalization method [3] is exclusive to H and E stain. Normalized images are shown below (Figures 5 and 6).

Creation of a classification system
Using convolution neural networks (CNN) can be less efficient in creating a classifier system mainly due to its requirement of a large dataset to learn from. Using CNN is not a very practical approach as it may not be feasible to collect a dataset containing large numbers of images. Thus an alternative method is proposed  where features are extracted from the images using unsupervised deep learning and then a supervised machine learning classifier is used to learn from those features for classification. The advantage of this method is the elimination of overfitting of the class with majority data and the system can work fairly well with less number of images. Using a support vector machine (SVM) the classifier is built and pretrained models such as VGG-16, ResNet50, DenseNet −121, DenseNet −169, DenseNet-201, InceptionV3, InceptionResNet50 and Xception.

Performance analysis
To select the best feature extractor from all the pretrained models, metrics such as F1-score and accuracy are considered. Higher accuracies may not be the most efficient and reliable metric always. Hence, F1-score is also considered as it shows individual class performance and is useful when the dataset is highly imbalanced. Table 2 shows the overall accuracies obtained when all the pretrained models are used.
From Table 2, it is found that performance of DenseNet is better than the other deep learning architectures. The performance of the variants of DenseNet is given in Table 3. Here it is observed that with the increase in the number of layers of DenseNet from 121 to 201, there is a degradation in the accuracy. Hence, the F1 score is also affected.

S. no
Model Accuracy (%)    The final pretrained architecture selected for feature extraction is DenseNet −121 to be combined with the machine learning classifiers. Supervised algorithms such as decision tree, SVM, Naive bayes were taken into consideration to find the optimal classifier. The results of the feature extractor and classifier are given in Table 4. From Table 4, SVM is chosen to be the optimal classifier that works best with DenseNet −121 feature extractor. DenseNet-121 is chosen due to high f1-score in spite of having less accuracy than DenseNet-169. Performance analysis of DenseNet-121 is given in Table 5.

Conclusions and future work
From the results obtained, it is observed that this method can provide better accuracy although the dataset is highly imbalanced and when there is a deficit in the dataset. Using convolution neural networks (CNN) can underperform when the dataset is imbalanced and it requires an extensive dataset to learn from. Improvements can be made by obtaining more data. Procuring more images from biopsies and medical data will help improve the system's efficiency and this can be extended as a separate component for the microscope. © 2021 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Real-time Human-Robot Collaborative Manipulations of Cylindrical and Cubic Objects via Geometric Primitives and Depth Information

Many objects commonly found in household and industrial environments are represented by cylindrical and cubic shapes. Thus, it is available for robots to manipulate them through the real-time detection of elliptic and rectangle shape primitives formed by the circular and rectangle tops of these objects. We devise a robust grasping system that enables a robot to manipulate cylindrical and cubic objects in collaboration scenarios by the proposed perception strategy including the detection of elliptic and rectangle shape primitives and depth information. The proposed method of detecting ellipses and rectangles incorporates a one-stage detection backbone and then, accommodates the proposed adaptive multi-branch multi-scale net with a designed iterative feature pyramid network, local inception net, and multi-receptive-filed feature fusion net to generate object detection recommendations. In terms of manipulating objects with different shapes, we propose the grasp synthetic to align the grasp pose of the gripper with an object's pose based on the proposed detector and registered depth information. The proposed robotic perception algorithm has been integrated on a robot to demonstrate the ability to carry out human-robot collaborative manipulations of cylindrical and cubic objects in real-time. We show that the robotic manipulator, empowered by the proposed detector, performs well in practical manipulation scenarios.(An experiment video is available in YouTube, https://www.youtube.com/watch?v=Amcs8lwvNK8.)

I. INTRODUCTION
Robots manipulating objects and transfer of objects between humans and robots are critical capabilities for collaborative robots across a multitude of application areas from physical assistance in household environments to collaborative manufacturing in industrial scenarios [1][2][3]. Examples of robot-human collaborative manipulations include a household robot that assists its user to grasp daily objects, a collaborative industrial robot that passes parts to a human worker for assembly, and so on. One difficulty of such physical activities is to enable robots to perform successful grasps in real-time, which is the main feature of human-robot collaboration manipulation.
A great number of objects in households and industries, such as cans, cups, cellphones, toothpaste boxes, circle or rectangle mechanical parts, have surfaces with elliptic and rectangle geometric primitives so that cylindrical and cubic objects in images are represented through ellipses and rectangles depicting the circular tops of the cylindrical objects at most angles of observation and the rectangle tops of cubic objects under the view of vertical angles, respectively. Therefore, a robot can grasp cylindrical and cubic objects via detecting ellipse and rectangle geometric primitives. In this work, we focus on real-time human-robot collaborative manipulations of cylindrical and cubic objects by the proposed detector of ellipses and rectangles and depth information in practical scenarios; see Fig.1 for an example.
Indeed, ellipse detection has been deemed useful in the computer vision applications, such as industrial inspection, medical diagnosis, security, autonomous navigation, and tracking targets [4]. Moreover, rectangle detection arises in several practical applications, such as text box detection, tracking vehicles, and recognition of buildings [5]. The prior detection algorithms of ellipses or rectangles always cause failure detections, especially in cluttered scenes where target objects are often densely placed. In different observation views, a variety of scales for objects results in the shape variations of ellipses and rectangles. Moreover, the low detection speed of an ellipse or a rectangle poses a considerable challenge for ellipse and rectangle detection. Almost all the detection approaches of ellipse or rectangle are not available for robotic static and dynamic manipulation because of low shape detection accuracy or execution speed, sometimes both. We note here that Dong et al. [1] successfully realized robotic static and dynamic manipulation of cylindrical objects via the proposed real-time accurate ellipse detection and also, have a robust capability of detecting small, occluded, and overlapped ellipses. However, this method has an unsatisfactory performance of detecting small ellipses in practical scenarios. In addition, this cannot detect rectangles.
To address or relieve the above issues for achieving humanrobot collaborative manipulations of cylindrical and cubic  Figure. 1. The robot grasping cylindrical and cubic objects. The pink ellipses represent the cylindrical objects and yellow rectangles indicate the cubic objects; the green text is the object class confidence. objects, we propose a novel strategy via a constructed end-toend network framework of fast and accurate detection of ellipses and rectangles (Note that here ellipses includes circles and rectangles includes squares.) in practical scenarios. In particular, we first extract original features from the input image with basic geometric primitives through the one-stage backbone [6], then accommodate the proposed adaptive multibranch multiscale net with a designed iterative feature pyramid network, local inception net, and multi-receptivefiled feature fusion net to generate object detection recommendations and determine object position. We design a new loss function, considering the position of the rotated bounding box and object class. Finally, we utilize a highly fast modified rotation R-NMS method to determine the final detection.
We highlight the novelties of this work. Foremost, our core contribution of this work is addressing the problem of perceiving cylindrical and cubic objects for real-time robotic manipulations via elliptic and rectangle geometric primitives in practical scenarios. The work provides the first solution, to the best of our knowledge, for robotic manipulation of cylindrical and cubic objects in physical scenarios through elliptic and rectangle shape primitives only. The second novelty locates in the detection of ellipses and rectangles. We are the first to propose the incorporation of detecting four shapes with different geometric primitives-circle, ellipse, square, rectangle and achieve segmented-fitting detection simultaneously rather than just use a bounding box to indicate a zone covering the detected objects, which provides many probabilities of robotic practical applications. Also, the proposed model takes the combination advantage of the same data structure representatives of ellipse and rectangle in the training data to simplify the model design such as to detect ellipses and rectangles. The third novelty incorporates several new aspects. First, an end-to-end one-stage detection model for multi-scale ellipses and rectangles is constructed. The newly proposed model archives an effective detection performance on ellipses and rectangles at a sufficiently fast speed in cluttered environments. An adaptive multi-branch multi-scale module(AMM) is proposed to fuse multilevel feature reuse for enhancing feature propagation. Second, a wide variety of rotated anchor boxes and a new loss function are designed to match targets of a large number of scales of ellipses and rectangles. There is a small contribution that we propose an approach of building synthetic datasets via compositing the real background images and 3D models of grasped objects and automatically generating transferred label points to bridge the domain gap of the insufficient quantity of the manually labeled data, thereby increasing the robustness of the network. The last contribution is that we implement our perception strategy based on the ellipse and rectangle detection on an industrial robot for achieving plenty of human-robot collaborative manipulations in cluttered scenarios, obtaining highly successful rates of grasp experiments.
The work is the extension of the robotic grasp strategy proposed in [1]. Although the work [1] also found the importance of considering the real-time ellipse detector as a perception strategy for robotic manipulations in complex scenarios, the underlying motivation, methodology as well as purpose in this paper are quite different. The rest of this paper is organized as follows. The following section II introduces the related work. Section III presents the proposed algorithm for the detection of ellipse and rectangle. Section IV provides demonstrations of robot-human collaborative manipulation scenarios. The paper is concluded in Section V.

II. RELATED WORK
A vast amount of prior works on the topics of geometric shape and robotic grasp detections were developed. Here we just introduce some representative works with respect to our work.

A. Geometric Shape Detection 1) Ellipse detection
Until now, almost all ellipse detection methods are based on parametric analysis, algebraic analysis, as well as geometric analysis of the properties of ellipses [1,7]. Parametric analysis approaches [8,9] based on Hough Transform were developed to detect ellipses through combing with geometric or algebraic analysis [7,10,11]. The algebraic approaches, such as those utilizing least-squares fitting approaches [12,13], random sample consensus [14], etc., extract edges that are likely to consist of detected ellipses. Moreover, there exist some geometry-based ellipse detection methods [1,7,11] that utilize short straight lines to approximate arc segments and then, these arc segments are grouped and merged into ellipses. Nevertheless, they are still limiting for rapidly detecting ellipses in severely complex environments.

2) Rectangle detection
For rectangle detection, some of the traditional analytic techniques explore geometric characteristics of a rectangle in Hough Transform space [15], ignoring the extraction and/or grouping of linear segments [5]. However, most traditional rectangle detection methods illustrated in the literature capture the information from edge [16], line [17,18], and corner primitives [19,20]. There are a lot of learning-based approaches regressing the bounding box of the detection box. However, the shapes of many objects are not rectangle strictly and the predicted bounding box just can cover the profile of the object. It does not indicate that the covered object is with rectangle geometric primitives. That is, regressing a rectangle is just considered as the detection of a bounding box. CNNbased algorithms of the regression bounding box are applied to detecting the ships in remote sensing images and text boxes [21][22][23][24]. Some methods were proposed to detect the horizontal box bound of a ship in remote sensing images [25]. Compared with the ship detection methods, works on text detection [23,24] generally pay more attention to object orientation. However, these approaches are generally invalid in handling images in complex environments where ships and boats in remote sensing images and scene texts are small, occluded by each other, or some detectors cannot reach real-time requirements.

B. Robotic Grasp Detection of Objects
Robotic grasp detection of objects is a widely studied field in robot applications. These detection methods include traditional analytical methods and learning-based approaches [26]. Early traditional grasp detection makes use of a set of 2-dimensional(2D) key points and the edge features to obtain the information of the class and position of the targeted object [27,28]. For the robotic grasp detection, the 3dimensional(3D) descriptor captured by a RGB-D sensor is available and more feasible than ever. These works based on the 3D descriptor further combines local template descriptor, shape, depth, and scene semantic as features to detect the objects [29]. Recently, learning-based methods (e.g., [30] [31], [32], [33], [26]) have proven to produce an effective performance for robot grasping detection, mapping from the features-set to grasp-quality via extracting rich features from the training data. In this context, most of these methods regressed oriented rectangles (defined by their positions, orientations, widths, and heights) in scenarios to do grasp detection representatives. The above detection methods have difficulty in coping with these more challenging robotic application scenarios where a robot utilizes the above detection methods to rapidly manipulate objects in dynamic scenarios. In addition, such grasp detection ignores the object itself shape and does not fit the profile of the grasped object, which results in invalid robotic manipulation such as assembling tasks.

III. THE PROPOSED METHODOLOGY
The approach for the detection of elliptic and rectangle shapes in images consists of three main blocks, namely, data preparation, network architecture construction, and detail implementation.

A. Data Preparation
We establish a synthetic dataset with cylindrical and cubic objects for robotic grasp(S-RG) for evaluating the proposed method for robotic grasp. The key reason is that generating synthetic datasets can relieve human labor. We elaborately choose several daily objects as a representative set, including the cylindrical and cubic objects, and employ a 3D scanner-Artec Eva to scan 3D models of objects, as shown in Fig.2(A). A virtual camera in OpenGL is used in rendering and capturing the target object images from various viewpoints uniformly (see Fig.2-B). Sequentially, the scene images are collected as the backgrounds. Then, the background image and the object images are composited to generate images of the virtual scenes with the target objects expected to be encountered in the real physical world. In terms of S-RG dataset, we capture images from practical grasping scenarios as backgrounds.

B. Learning Model Construction 1) Network Architecture
The end-to-end framework of the proposed method is summarized in Fig.3, which mainly consists of two parts. Specifically, the proposed network first employs the YOLOv3-Darknet53 architecture [6] as a backbone network to extract original features from the input image for the detection of objects with basic geometric primitives, then accommodate the proposed adaptive multibranch multiscale net (AMM-Net) with designed iterative feature pyramid fusion network (IFPF), local inception net and multireceptive-filed feature fusion net to generate object detection recommendations and determine object position by predicting hull orientation and relative offset.

a) Design of iterative feature pyramid fusion network
It is well known that feature context information in low levels is are more spatially-sensitive for the features of edges, texture, and corners to be significantly helpful for locating objects but conveys less semantic information. On the contrary, high-level features with low resolution have a large receptive field and are rich context information, being insensitive to geometric distortion and small shift, which is beneficial for dealing with the classification task [34]. Considering that the background in a cluttered environment has the wide coverage of the input image and the little area in a view is occupied by objects, object detection can be viewed as small object detection tasks in cluttered environments. Moreover, object detections suffer from the impact of different sizes and arbitrary orientations. The challenges, as stated above, motivates us to design an iterative feature pyramid network inspired by [35], which integrates three output layers via a down-top and lateral connection because of the smooth feature propagation and multi-time feature reuse. Thus, the proposed IFPF is designed to effectively enhance feature propagation, and fuse multilevel feature reuse is robust against the impact of different sizes (see Fig.4).
The fundamental architecture YOLOv3-Darknet53 [6], which considers object detection as a classification and regression problem, can run in real-time and thus, is chosen to be applied to detecting objects rapidly. We employ feature maps { 1 , 2 , 3 } generated by the Darknet-53 feedforward network. Note that these three feature maps have strides of 8, 16, 32 pixels with a stride ratio 2. In other words, the stride is equal to the reduction factor of the feature map relative to the original image. In terms of the designed down-top network, we achieve higher resolution features by lateral connections and iterative connections, { 1 , 2 , 3 }. Since feature maps in the IFPF net are not of the same resolution, we need to normalize the feature maps before integrating them together. For example, to obtain 1 , we first utilize nearest neighbor Figure.2. The 3D models of some cylindrical and cubic objects (A). A virtual camera captures images from the 3D models and the images extracted from 3D models; the synthetic images extracted from 3D models of objects and practical background; the cylindrical and cubic objects are described by five parameters (B). , and represent the height, width and orientation of the object, respectively.
up-sampling to process all the preceding feature maps for achieving the same size and then, merged it with the next feature maps by concatenation. Also, the merged map and 1 both undergo a 1 × 1 convolutional layer to reduce channel dimensions before element-wise addition. Note that a1 × 1 convolutional filter can be used to change the dimensionality in the filter space. If the number of 1 × 1 convolutional channels is bigger than the number of the operated feature map channels then we are increasing dimensionality while if the number of 1 × 1 convolutional channels is smaller than the number of the operated feature map channels we are decreasing dimensionality in the filter dimension. Finally, to eliminate the aliasing effects of up-sampling, we append a 3 × 3 convolutional layer to the merged map, while reducing the number of channels. After the iteration above, the feature maps are { 1 , 2 , 3 }. The feature maps 2 , 3 are reused two times. During this stage, we still output three feature maps rather than one feature map sharing classification and regression to generate more complementary semantic information.

b) Design of the local inception net
For the purpose of reducing network parameters, we only integrate the local inception model with the middle feature map of the output of the IFPF-Net for the fusion to balance the semantic information and location information while ignoring other less relevant features. The outputs 2 and 3 of IFPF-Net are first taken up-sampling to have the same size by the corresponding anchor strides. Then, we pass through the designed inception structure to expand its receptive field and increase semantic information, as shown in Fig.5.
Indeed, the inception structure does not increase the net width but also provides the adaption of the net to the scale. At the same layer, the outputs of different channels are highly correlated. In the inception structure, a 1 × 1 convolutional kernel cannot only naturally incorporate features from different channels, but also reduce the number of channels of   the input features, thereby reducing the number of parameters included in the module. While other kernels can ensure that the features extracted by them have the scale diversity of the target objects. 3 × 3 convolutional kernel, 5 × 5 convolutional kernel, and 3 × 3 max-pooling kernel are adopted for different convolution branches to achieve feature information extraction for three different scales respectively. Finally, a new feature map 2 is obtained by element-wise addition of the two channels for the middle feature map 2 . c) Design of the multi-receptive-filed feature fusion net The multibranch convolution structure above allows object features at different scales to be activated on different branches. Here we integrate feature maps from three branches by adopting reorganization and route layers, which together bring features from the earlier 1 feature map, the earlier 2 feature map, and the earlier 3 feature map. Fig. 3 and Fig.5 briefly demonstrate the implementation detail.
We set the size of the integrated feature maps as the same size. In our implementation, we achieve this by the reshape operation which converts larger/smaller feature maps to a set of smaller/larger ones. We set two convolutional layers after the integrated feature map goes through ResN block and CBL block, as shown in Fig.3. Convolutional kernel sizes of these two layers are 3 × 3 and 1 × 1 , respectively. These two convolution layers are the batch normalization layer and rectified linear unit layer sequentially. The final output works as a tensor to predict the location parameters, confidence score, the class score of each rotated bounding box.

2) Design of prior anchors
In terms of object detection methods such as YOLO [6], SSD [36], and Faster R-CNN [37], anchors are initial shapes of regions of interest that provide the coarse initial bounding box of the object at each cell in the feature map. Similarly, an ellipse or a rotated rectangle can be estimated coarsely by a prior rotatable anchor. In this case, we add an orientation variable for getting the prior rotated anchor. However, predicting all the five parameters becomes more complex compared with using typical horizontal bounding boxes, not only due to the increased number of unknown variables, but also the fact that the ellipse or rectangle orientation can be arbitrary. But actually, it is fortunate that the shape of an ellipse or rectangle with arbitrary can be assumed to be rotationally invariant, which indicates that it is possible to predict the center, width/major axis, height/minor axis, and orientation independently. To enable a feasible regression, we predict the orientation angle offset relative to prior orientations. We also utilize the k-means algorithm to regress the dimension of ellipses and rectangles in a dataset for obtaining the widths and heights of anchors, as shown in Fig.6. Thus, a series of anchors are calculated by combining different widths, heights, and orientation angles(see Fig.7).

3) Geometric description of ellipses and rectangles
To determine an ellipse or rectangle, we need to define the ellipse and rectangle with an arbitrary orientation by five parameters, the geometric center ( , ), the width/major axis, and height/minor axis ( , ℎ) and the orientation angle .
Directly predicting parameters results in the model instability [38]. Most of the instability comes from predicting the locations for the box. Similar but different from [38], we do not only predict the 5 parameters ( , , , ℎ , ) of horizontal bounding box and the object class but also regress the orientation angle for obtaining the locations of ellipses and rectangles at each cell in the feature map for the proposed network. As shown in Fig.8, the coordinate ( , ) is parameterized as an offset, from the top left corner of the feature map by ( , ), of a particular grid cell and it is normalized between 0 and 1 by the logistic activation. Note that we use an exponential function to constrain the network's predictions of the bounding box width and height to be positive and the anchor box has width and height ( , ℎ ), then the predictions of the bounding box are as follows: = ( ) + = ( ) + = ℎ = ℎ ℎ (1) The orientation angle , which is defined as the angle between the semi-major axis direction or the width direction and the x-axis positive direction, determines the orientation of an ellipse or a rectangle. We set the anticlockwise direction to be positive.
The orientation angle needs to be normalized to a limited range so that the prediction for the angle is convergent. Due to the axis symmetry, we need to constrain the angle between − 2 and 2 to cover all the rotation cases. Although theoretically it may be possible to predict the orientation angle through direct regression, we found that it is impossible to achieve the expected performance. To address this issue, we implement an arc-tangent function for constraining the orientation angle to be between − 2 and 2 and thus, the potential angle is provided as = tan −1 .
(2) The confidence score of each rotated bounding box indicates the probability that the predicted bounding box is an ellipse or rectangle. A logistic activation is applied to normalizing the network's predictions of the confidence score such as to fall in the range between 0 and 1. The confidence score prediction is given as where Pr ( ) denotes the class probabilities per grid cell, represents the IoU between the predicted box and the ground truth.

4) Multi-task loss construction
Since the proposed network is an end-to-end learning model, we just need to summarize a global loss function for ellipse or rectangle detection. The overall objective loss function is a weighted sum of the rotated bounding box parameters loss and the class confidence loss.
Unlike YOLO [6] and Faster R-CNN [37], we optimize the model global loss while training as follows, , regarding the rotated bounding box, orientation angle, the cases with and without target objects, respectively. We use these control parameters to adjust the corresponding loss weight; , , , ℎ , are the location parameters of the prediction bounding box and the corresponding label information is represented by ̂,̂,̂, ĥ ,̂; and ̂ represents the prediction parameters and ground truth label information, respectively. For the loss function, we introduce the cross-entropy loss to calculate the losses of the center coordinates, the orientation angle, and the class confidence probabilities.

C. Implementation Details
PyTorch is implemented to train the proposed learning model. We use a step strategy with mini-batch stochastic gradient descent (SGD) to train the networks with the batch size 8 on a GeForce GTX 1070 GPU. A dynamic learning rate is applied to the training process with an initial learning rate of 10 −3 for

IV. ROBOTIC MANIPULATION EXPERIMENTS
To evaluate the performance of our framework, we run an extensive series of human-robot collaborative manipulation experiments of cylindrical and cubic objects using a stereo camera-ZED, an industrial robot arm-UR5 with the repeatability accuracy of ±0.1 mm, the supported payload of 5kg, the maximum reach of 850mm, and a 3-finger Robotiq gripper without tactile sensor mounted on the robot arm(see Fig.1).
We choose 12 household objects (including cylindrical and cubic objects), covering some appearances. The S-RG dataset, as mentioned above, is used in training the proposed learning model; however, this dataset also includes labeled images. In real scenarios, the tops of cubic objects are not strictly rectangles due to the two cases. One is that the cubic objects are placed at random positions so that the camera observes the cubic objects in arbitrary view angles. The other is that the top surface of objects undertakes distortion to a certain extent due to the squeezing. Contrastively, in synthetic images, all the top  shapes of cylindrical and cubic objects are standard ellipses and rectangles. To bridge this gap, we also label 760 images with real scenarios. This synthetic dataset fed to the learning model for robot grasp consists of labeled 760 images and 240 images generated by the proposed solution of generating a synthetic image.

A. Robotic Grasp Synthesis
Given an image including target objects, our objective is to identify the grasp configuration for robotic manipulation. Since the space information is not involved in the proposed model, we need to introduce a 3D pose representation for the grasp configuration. From an RGB image, the proposed method can detect target objects and obtain ellipse and rectangle centres considered as the 2D pixel coordinates. Some detection cases are indicated in Fig.9. Through registered the depth image, the 2D pixel coordinates of the centres of detected objects are converted to 3D world coordinates. We collect 10-time detection results for each cylindrical or cubic object from the live video stream to get the median value of the centres of 10 ellipses or rectangles, avoiding the cases of partially missing data captured by the camera.

B. Robotic Manipulations 1)Robot grasping a single cylindrical or cubic object
The first human-robot collaboration task is considered as follows. The vision system first perceives the arrival of new objects via detecting the tops of the cylindrical and cubic objects as ellipses and rectangles on the table and then, the robot subsequently grasps and delivers them to the human operator. While manipulating, the robot is not only a receiver but also a giver. The workflow of collaborative grasp is illustrated in Fig.10. In particular, the 3D centre position of the top of the cylindrical or cubic object is calculated, as introduced above. Inferring the planning paths, the robot performs the actions of grasping the targeted object and passes Figure. 9. Some snapshots of a video stream capturing practical scenarios. Each image includes multiple cylindrical and cubic objects with elliptic and rectangle primitives. When the objects are being moved, the proposed detector is still able to detect them, which verifies our detector can be implemented to practical applications. Figure.11. The robotic grasping 6 cylindrical objects and 6 cubic objects highlighted by black ellipse frames captured by the front camera(the first four rows) and by the top ZED camera(the last four rows).

3D sphere model
it to the human operator.
We perform at least 120 times of picking experiments for 12 different objects, 6 cylindrical objects including the sphere object, Shincup, chocolate bucket, potato chip bucket, pepper cup, small potato chip cup, and 6 cubic objects such as the coffee box, biscuit box, milk box, pie box, OREO box, swiss roll box, as shown in Fig.11. Each object is put on the table at least ten rounds with random initial orientations and positions. That is, the robot runs at least 10 attempted grasps for per object. When the object is placed close to the left and right top corners of the table, the robot cannot plan a reachable path to achieve a successful grasp due to the large distance between the robot base and the targeted object. If the human operator puts objects within the robotic graspable area, the robot can grasp objects perfectly, which illustrates that the proposed detection strategy can realize accurate ellipse and rectangle detections. Moreover, the appearances of some real objects in robotic grasp experiments are a little or totally different from the scanned 3D models due to product updates. But, fortunately, our self-supervised learning system can track this challenge and successfully adapts to the new scenarios.

2) Grasps of multiple objects in cluttered environments
To illustrate that the robot is able to manipulate multiple objects in cluttered environments, we conduct the experiments of the robot grasping cylindrical and cubic objects from a densely packed pile of objects in cluttered environments. The multiple objects are put randomly on a table. The human operator generally feeds objects to the robot by putting them on the table during the period of the robot grasping other objects. Grasping from densely arranged objects presents much more challenging than one single object isolated. Perceiving cluttered visual scenarios and avoiding collisions between the gripper and objects attribute to the main challenges. Our detection approach, however, overcomes the first issue and detects multiple objects at the same time accurately. Providing an accurate perception feedback to the robot is the prerequisite of avoiding collision to realize a successful grasp. We construct a 'closed-loop' grasp system.
In particular, the robot continuously attempts multiple grasps until all objects are grasped, as shown in Fig.12 and Fig.13.
We perform 20 groups of experiments, totally around 100 trials on cylindrical and cubic objects, including 6, 6, 8 groups of grasping the cylindrical objects, cubic objects, mixed cylindrical and cubic objects, respectively. As a result, a total of 18 out of the 20 experimental sets are successfully accomplished without a single failure. Of the two failures, they are due to the estimation of inaccurate positions in the 3D space. Given an accurate detection in the 2D space, the failed grasps are caused by the inaccurate depth information, which allows the gripper to reach an ungraspable position and generates collisions with the object during the grasp attempt. The reflective property of the object's surface results in inaccurate depth. Indeed, the grasp failures cannot attribute to our detector. Under the camera view, the proposed detector almost realizes 100% successful detections. That is, the proposed detector stills manifest a higher average success rate of detecting ellipses and rectangles in cluttered environments, which indicates that the proposed perception strategy can better understand the spatial geometry of the objects.

V. CONCLUSION
We consider the problem of real-time human-robot collaborative manipulations of cylindrical and cubic objects through the perception strategy in the 2D space. Thus, we present a jointing network that detects ellipses and rectangles with sufficient accuracy and speed to realize robotic manipulation of objects in the presence of complicated scenes. In human-robot collaborative manipulation experiments, the robot successfully grasps and tracks cylindrical and cubic objects in real-time in cluttered environments, which indicates the proposed detector is potentially applied to industrial or household environments. In the future, we will focus on detecting objects with complex shape profiles.