Analysing and Measuring the Performance ofMemristive Integrating Amplifiers

Recording reliably extracellular neural activities isan essential prerequisite for the development of bioelectronicsand neuroprosthetic applications. Recently, a fully differential,2-stage, integrating pre-amplifier was proposed for amplifyingand then digitising neural signals. The amplifier featured a finelytuneable offset that was used as a variable threshold detector.Given that the amplifier is integrating, the DC operating pointkeeps changing during integration, rendering traditional analysis(AC/DC) unsuitable. In this work, we analyse the operation ofthis circuit and propose alternative definitions for validating thenecessary key performance metrics, including: gain, bandwidth,offset tuning range and offset sensitivity with respect to thememory states of the employed memristors. The amplificationprocess is analysed largely through investigating the transientbehaviour during the integration phase. This benchmarkingapproach is finally leveraged for providing useful insights anddesign trade-offs of the memristor-based integrating amplifier.

Analysing and Measuring the Performance of Memristive Integrating Amplifiers
Jiaqi Wang, Alexander Serb, Christos Papavassiliou, Sachin Maheshwari, Themistoklis Prodromakis Abstract-Recording reliably extracellular neural activities is an essential prerequisite for the development of bioelectronics and neuroprosthetic applications. Recently, a fully differential, 2-stage, integrating pre-amplifier was proposed for amplifying and then digitising neural signals. The amplifier featured a finely tuneable offset that was used as a variable threshold detector. Given that the amplifier is integrating, the DC operating point keeps changing during integration, rendering traditional analysis (AC/DC) unsuitable. In this work, we analyse the operation of this circuit and propose alternative definitions for validating the necessary key performance metrics, including: gain, bandwidth, offset tuning range and offset sensitivity with respect to the memory states of the employed memristors. The amplification process is analysed largely through investigating the transient behaviour during the integration phase. This benchmarking approach is finally leveraged for providing useful insights and design trade-offs of the memristor-based integrating amplifier.
Index Terms-neural spike detection, threshold detection, hybrid CMOS/memristor circuit, integrating amplifier, high sensitivity

I. INTRODUCTION
R ECORDING neural signals using implantable microsystems is essential to the development of diagnostic and therapeutic solutions [1], Brain Machine Interfaces (BMIs) [2] and neuroscience research [3]. The implantable device typically contains electrodes as well as front-end and back-end module, where raw neural signals collected from electrodes will be fed into the other two modules for further processing [4]. After processing, analogue neuronal trains [5] or digital format [6] will be transmitted to external devices wirelessly. With digital output, a neural spike (Action Potential, AP) detection algorithm which comprises threshold detection and digitisation can be applied in back-end stage typically [7]. For an implantable device, this is required to have low power/heat dissipation (< 80 mW/cm 2 ) in order to avoid damaging surrounding tissue [8]. The low power dissipation contributes to high integration density. Furthermore, both dc offset [9] and minute extracellular neural activity signals (in the order of 10s-100s of µV ) picked up by electrodes will be fed into frontend devices for amplification and filtering [10]. In summary, the implantable front-end module needs to have low-power dissipation, low-noise and also to reject dc offset and other noise interference.
To achieve low power consumption, a number of multichannel neural recording architectures has been proposed [11] [12] [13]. It is clear that the energy consumed in the analogue multiplexer before ADC can be reduced to improve power efficiency. From the system level point of view, Serb et al. [14] propose to perform spike detection and digitisation directly on the neural signal from electrodes in order to save power from processing local field potentials (LFP) which will be discarded.
Preamplifiers are critical for boosting the extremely weak input signals to levels where they can be further processed and so they act as the first stage in any neural recording processing (a result of the Friis formula) [15]. Alternatively, the operational transconductance amplifier-capacitor (OTA-C) structure is suitable for bio-electronic devices as the lowpass filter for neural signals [16]. The objective of combining an OTA with load capacitor is to integrate signals instead of simply amplifying them in continuous mode in order to boost effective gain. A different technique has been proposed to compensate the DC offset of the electrode-tissue interface [17] [18] [19]. The Harrison topology is capable of rejecting large dc offset, operates in continuous mode and is the current standard in the field [20]. It is possible to conduct threshold detection directly on the signals in Harrison amplifier. [14].
With the characteristic of analogue modulation of their resistive state, memristive devices can be utilised in CMOS circuit as trimming component [21]. Such an integrating preamplifier enhanced with offset tuning for ultra-fine threshold detection was proposed [14]. In this work, memristive devices were utilised as non-volatile resistive loads [22] to trim the offset voltage with high precision.
The architecture and preliminary analysis presented in [14] demonstrated the general operating principle of what we may describe as a "memristive integrating amplifier". In this work we add detail on the operation of this type of amplifier as well as investigate how important parameters such as clocking and differential/common input voltage affect performance. One of the challenges identified in doing this is defining important performance (e.g. gain, bandwidth, CMRR and etc.) in a manner suitable to the operation of integrating amplifiers. We provide such metrics that suit the particular implementation of the memristive integrating pre-amplifier. The mathematical descriptions of the resulting metrics and insight obtained from examining the behaviour of transistors during the key integration phase of the amplifier illuminate various trade-offs that characterise the design. This work has been done using commercially available 0.18µm CMOS technology with 1.8V supply voltage across all the experiments.
The paper is organised as follows: A brief overview of the pre-amplifier and its operation, followed by the re-definition of its key performance metrics is presented in section II.

arXiv:2009.09909v1 [cs.ET] 21 Sep 2020
Simulation set-up, analysis and results are shown in section III. A discussion of design trade-offs and other points of interest pertaining to the amplifier design is in section IV. Finally, section V summarises and concludes the paper.

A. Amplifier design overview
The architecture studied is a modified/simplified version of the original design proposed in [14]; it is shown in Fig.  1. It consists of three main sections: I) a fully differential core amplifier (effectively a single-stage analogue amplifier acting as the 1st stage of the design), II) a dynamic latched comparator (DLC) amplifying and quantising the output state of the core amp and III) a current bias unit powering the system's core. The overall system operates as a threshold detection circuit which compares two minute input signals and ultimately outputs a binary flag, as shown in Fig.2.
Each threshold detection operation is carried out in four phases that we label as: (i) reset, (ii) integrating, (iii) digitisation and (iv) off. These are illustrated in Fig. 3. They remain unchanged from the original work and act as follows: In the reset phase (i) the core amplifier is on (clk ana, clk rst: high, clk, clk anabar: low) and the load capacitors are discharged (V mida/b = 0), so that voltage/current in core amplifier is initialised and cleared before integration commences in the next phase.
In the integrating phase (ii) (clk ana: high, clk anabar, clk rst, clk: low) the reset transistors (M8&M9) are switched off and the currents flowing through the branches of the core amplifier drain into the load capacitors. From a 'large signal' perspective, V mida and V midb continuously increase during integration. In terms of 'small signal', ∆V mida−midb increases with time and normal operation is maintained so long as the cascode transistors M6&M7 remain in the saturation region. The voltage difference between nodes mida and midb is impacted by the charging speed/current and integration time. Memristors R1&R2 work as trimming devices and tune the offset of the core with very high sensitivity (1µV /kΩ shown in the original paper). At the end of this phase, V mida/b should be high enough to successfully trigger the DLC and ∆V mida−midb should be as large as possible for maximising gain.
In the digitisation phase (iii) (clk ana, clk: high, clk anabar, clk rst: low) clk goes high, triggering the DLC to perform the conversion of V mida/b into the final digital outputs. By convention we take the output from the branch where output '1' represents a spike while '0' represents the absence of a spike. Shortly after the decision is committed by the DLC, the core amplifier is turned off as the system re-enters the off phase.
Finally, in the off phase (iv) (clk: high, clk ana, clk anabar, clk rst: low), the tail current is cut off by setting clk ana to zero. The pre-amplfier is turned off and stops recording neural signals. clk anabar is also deactivated (goes to high) thus preventing the accumulated charge across the large gate capacitances of M4&M6 from draining away. The circuit can be divided into three parts: (I) core integrating amplifier, (II) dynamic latch comparator, (III) current bias control unit. In this paper, the clocking signals are all assumed to be generated by a shared source and be strictly periodic.

B. Differences vs. the original design
The design under study has been simplified vis-a-vis the original from [14] as follows: First a fixed clocking scheme was adopted. The previous design featured an asynchronous clock generation circuit embedded in each channel. When it determined that the result would be available on nodes mida/midb it triggered, on-demand, the clocks. However, it has been observed that the integration results for the very small differential input signals of interest always become available at fixed intervals; therefore, an on-demand triggering system is not required here. Second, the sizes of the input transistor pair were decreased. In [14], huge transistors were introduced as input pair to reduce noise. However, the associated large parasitic capacitances together with the µA tail currents result in a low transit frequency (f T ). Therefore, in consideration of speed and power consumption the sizes of input transistors have been decreased.

C. Key performance metrics
The main performance indicators for the core amplifier include: gain, bandwidth, offset tuning range and sensitivity on memristor resistance, noise performance, input range, common-mode rejection ratio (CMRR) and power consumption. All these metrics (with the exception of power) mostly depend on the integrating phase, when amplification is conducted. In this stage the cascode transistors are in saturation  1V − 100µV , 1.1V + 100µV ] over 2ms while the pre-amplifier is carrying out a conversion every 10µs to detect the relationship between inputs A and B. Input B (inb) remains stable at 1.1 V throughout. In this test the amplifier was balanced (R1 = R2). When V ina < V inb , the left branch current is larger than the right branch current, inducing V mida − V midb > 0. The DLC captures this relation and generates binary signals: Vouta = 1 and V outb = 0, which appears in the bottom trace as a predominantly orange output trace. Conversely when V ina > V inb , Vouta = 0 and V outb = 1, which appears as a combined orange/blue output trace. Note: this type of simulation can also be used to test the offset tuning range and tuning sensitivity on the resistive state of memristive devices. When R1 > R2, V ina must be lower than V inb to ensure a balanced output, creating an offset. This is read in the output trace as an encroachment of the blue region into the orange (and vice versa for R1 < R2).
mode. As V mida/b keep increasing throughout the integration phase, there is no set DC operating point. nonetheless, because this is an extremely small signal amplifier, the current flowing through each branch is under normal circumstances approximately the same and constant. This allows for analysis similar in spirit to regular small-signal analysis by using transient simulations for obtaining the relevant data. Standard DC operating point and AC analysis cannot be applied here directly. It is perhaps more appropriate to think of V mida/b as 'large signal' in the mV -range and ∆V mid = V mida − V midb as 'small signal' in the µV -range.

1) Gain:
The gain is defined, as usual, as the ratio of the output signal amplitude over the input signal amplitude, δV out /δV in . For the core amplifier this translates into δV mid /δV in , where δV mid is taken at the end of the integration phase and δV in is considered constant for the purposes of this analysis. Time (ns) A more explicit formula can be obtained for the gain: The input differential voltage induces through the input differential pair and its associated current branches a difference in currents: where g m is the transconductance of the input differential pair in normal operating conditions. This induces a difference of charge on the load capacitors: where τ is the integration phase duration. Finally, this gets transformed into the voltage difference we observe at δV mid through the load capacitances C: Combining the above yields the gain (G): Given that we know that the currents filling each load capacitor are approximately constant and equal we can express integration time τ as a function of the difference in V mid levels at the start and end of integration ∆V mid = V mid | t=te − V mid | t=t0 , where t 0 and t e denote the start and end of the integration phase. In our case V mid | t=t0 = GND = 0 and therefore ∆V mid = V mid | t=te . This is a voltage level that we can adjust by choosing appropriate values for the tail current of the amplifier and the integration time. Given this interdependence we now seek to find an expression for τ that depends only on engineering parameters. We begin by observing that: where ∆Q is the total charge accumulated on each node (mida/b) as a result of the tail current. This can, however, be easily expressed as: where i tail/2 is the half-tail current of the amplifier core. This allows us to express τ as follows: where we replaced the approximation symbol with an equality for clarity, since the deviation is expected to be sufficiently small under normal operation. Now we can substitute Eq.7 into Eq. 4 and obtain gain as: which further simplifies to: where T E is the transconductor efficiency factor of the input diff pair transistors. In other words the differential gain of the pre-amplifier core only depends on the T E and the voltage range over which we are integrating. Integration time and tail current can be freely traded off, in principle (but consider noise, etc.). Note that ∆V mid represents voltage difference during the integration phase, while δV mid is the output that captured at the end of integration.
2) Bandwidth: In an integrating amplifier, such as the one studied here, the notion of bandwidth is somewhat different than in continuous mode systems because the output is not a continuous waveform whose Fourier component at some frequency can be compared in magnitude to an input stimulus of the same frequency. Instead, our amplifier output is a single value that is influenced (in magnitude) by the input in proportion to the input's absolute integral. For a unit magnitude pure tone signal of angular frequency ω = 2πf , the maximum absolute integral within a time window 2a is given by: where I ef f stands for 'effective integral'. There is no need for introducing a phase shift φ into cos(ωt); using the trignomoetric identity for cosine of sum of angles we can easily prove that I ef f maximises for φ = 0. At DC the integral is simply 2a and subsequently it decreases within the envelope of 1/f as frequency increases. If we divide I ef f,max by the length of the window we obtain what can be interpreted as an attenuation factor:  Fig. 4 illustrates the evolution of λ with f ; the frequency of the sinusoid in units of π a . We many now define the effective bandwidth of the amplifier as the frequency above which λ(f ) is always below a certain value p. An indicative measure of bandwidth may be given by p = 20%. For an integration period of 1 µs this yields around 1.5 M Hz bandwidth. Naturally, p can be set to another suitably chosen value to yield different appropriate bandwidth figures. This metric holds only so long as the resulting frequency is much lower than all other RCs in the amplifier core, and thus we have no additional attenuation. It is worth noting that the most typical neural signals of interest, action potentials (spikes), last in the order of ms. This implies that the even spike features of the order of 100s of µs will be integrated without any significant attenuation.
Throughout our analysis we make the following approximation: the amplifier is integrating linearly throughout its integration voltage range ∆V mid . The input differential signals of interest are so small that linear approximations can be assumed to hold throughout the whole system (eqs. (1) and (6)). In practice, there will be some additional distortion due to the changing V ds experienced by the cascode transistors, but we currently ignore this effect in our analysis.
3) Tuning Sensitivity and range: The memristive devices applied in the current branches regulate the charging speed to load capacitors by modulating the effective output resistance of the core amplifier as seen by the capacitive load. To see the mechanics of this action we refer to the schematic in Fig.  1 and the standard equation for the impedance of a draindegenerated MOSFET, looking into the source. When this is applied to the source of M6 we obtain: where Z s6 is the impedance looking into the source of M6, g m6 is the differential transconductance of M6, and R o6 the output resistance of M6.
Extending this principle to calculate the impedance of M4, as drain-degenerated by the M6-R1 cascade we obtain: which eventually unfolds to: A similar expression also applies for the right current branch.
Setting A = 1 gm4 + 1 gm6gm4Ro4 and B = 1 gm6R06gm4Ro4 we can express the impedances seen by M3 looking into each current branch as: where Z l = Z s4 is the left current branch impedance and Z r is the right branch impedance. Next, examining the distribution of tail current across the branches we obtain an expression for the left branch current i l as follows: where i T = i 3 is the tail current. Given that B 1 (as it is the product of two maximum FET amplifier gains), i l can be further approximated as follows: Similarly for the right branch current i r : This yields a total current imbalance of: which if divided by the common transconductance of the input differential pair transistors yields the required voltage offset to rebalance the branches as a function of the difference in RRAM resistive states: which when fully unfolded yields: where we have renamed our variables to explicitly stress the common values of output impedances and differential transconductances of the input differential pair and cascode transistors (R o,cas = output impedance of cascode transistor, g m,in = diff. transconductance of the input diff pair). This result relies on the standard small-signal assumptions that the various g m s and R o s remain constant, all transistors involved remain saturated (either over or below threshold) and crucially, it makes no other assumptions on the voltage present at the load capacitors. So long as: a) the g m s of all transistors remain mostly unchanged and b) the change in load capacitor voltage does not affect the absolute difference in RRAM resistive states seen by the system, the capacitors charge uniformly under balanced conditions (V in = V os ). Whilst condition (a) can be reasonably approximated as true in saturation, condition (b) is not generally true because of the non-linearity in the I −V of the RRAM devices [23]. Analysis of this phenomenon is outside the scope of the paper as it is RRAM technology-specific, but in general if the absolute resistive state difference changes as the integration process progresses, we obtain offset voltage drift that may potentially affect operation when a fixed, non-zero offset is specifically required (e.g. for threshold detection with the offset acting as threshold).
Overall, Eq. 22 shows that in small-signal conditions the offset voltage of the core amplifier is proportional to the difference in RRAM resistive states divided by the maximum transistor gains of the input diff pair and cascode transistors. This division explains the extreme fineness of tuning achievable.
The tuning range can in principle be extended under the rule of Eq. 22 for as long as the underlying assumptions hold. We note two important limiting conditions: 1) If the imbalance in currents becomes large, eventually the assumption of equal g m s on both current branches collapses. Exactly when this occurs depends on the tightness of the specifications. 2) If the voltage dropped across the larger of the pair R 1,2 becomes comparable to the capacitor voltage range through which the amplifier can integrate while maintaining transistor saturation (normal operation), eventually the amplifier will run out of integration voltage headroom. Thus, introducing a headroom vs. maximum tuning range headroom (so long as condition (2) remains the dominant limit).
4) Input-Referred Noise: The amplifier's core noise is dominated by the input differential pair. The reasons are the same as in continuous mode amplifiers such as the Harrison [20]: the input pair provides substantial gain through its g m and thus mitigates the input-referred contributions from downstream elements (primarily the cascode transistors and the RRAM devices).
The standard MOSFET input referred-noise model containing both thermal and flicker noise is given by the following expression for spectral density [24]: where k is Boltzmann's constant, T is the absolute temperature, γ = 2 3 for long-channel transistors and higher for shorter channel devices, K a typically empirically determined factor scaling 1/f noise, C ox the gate capacitance, W, L the transistor sizes and f denotes (linear) frequency.
In our amplifier the noise from each transistor in the input differential pair from Eq. 23 propagates to the output via the gain G from Eq. 8 and is then moderated by the attenuation factor λ from Eq. 11. Moreover, bearing in mind that the amplifier's output is the difference V mid and that it is operating in a 'nearly balanced' regime, the total noise spectral density equation at the outputs becomes: where we substitute all ωs with f for simplicity and have assumed that both branches contribute equally to noise. We note the following: First, the application of λ(f ) turns white noise into 1/f (more accurately 1/af ) and 1/f noise into 1/f 2 , as is typical of single-pole low-pass filters. Second, if we desire short integration periods, noise moderation effect by λ(f ) may become too weak to make any practical difference because of the a 1 factor. Finally, input-referring Eq. 23 and the contributions of λ(f ) (which can be ignored in this case), we obtain the following noise profile: where the g m , W and L factors are the same (at least approximately) for both transistors in the input diff pair. 5) Input Range: Under normal operation, the input differential pair transistors M4,5 must be in subthreshold saturation. This implies two operating conditions: (1) A minimum drainsource voltage |V ds,min | = m · V T , where V T is the thermal voltage and good rule of thumb for ensuring subthreshold saturation is 3 ≤ m ≤ 4 (here we will use m = 4) [25]. (2) We need an appropriate gate-source voltage |V gs,4 |(< |V th |) that allows the transistor to pass ≈ i tail /2 in subthreshold saturation. This is treated as approximately constant in this analysis.
Therefore the common mode voltage V CM is bounded: The top boundary is simply: where V ds,sat,x is the drain-source saturation voltage of transistor x. Exceeding the boundary causes M3 to triode and simultaneously encroaches on V gs,4 , progressively shutting the amplifier down. The bottom boundary hinges on maintaining the input diferential pair in subthreshold saturation (|V ds,4 | ≥ |V ds,min,4 |): 6 is the gate-source voltage allowing the cascode transistor to pass ≈ i tail /2. This is also treated as approximately constant in this analysis. The 2nd term is recognised as V drain a under normal operation and node voltage V drain a can be seen in the schematic of Fig. 1. This unfolds to: Here, the cascode transistor M6 enforces a specific and relatively fixed value of V drain a under the control of V anabar,low (similarly for M7 and V drain b ). Combining Eqs. 26 and 28 we can find the approximate value of V anabar,low above which the input differential pair runs out of common mode range: From here we can see the trade-off between common mode and integration voltage ranges (directly connected to gain). If the input stage of the amplifier is AC-coupled, the required V CM range may become very small. 6) CMRR and CMGD: In continuous mode amplifiers CMRR (common more rejection ratio) is defined as the ratio of the differential gain vs. the common mode gain. In our case this is given by: where A dm , A cm are the differential and common mode gains respectively. In a perfectly balanced amplifier (nominal design) this will be zero at first order, so it would be perhaps more informative to measure this directly in silico.
There is a slightly different effect which will impact our integrating amp and can be analysed easily: Gain distortion vs. common mode voltage V CM : We define this 'common mode gain distortion' as: Taking Eq. 9 and substituting g m = i tail/2 Vgs,4 we obtain: We can now unfold the derivative dG dV CM as follows: where dVgs dVg ≈ 1 due to the high impedance of M3. We note that this value could easily be as low as 1 (e.g. consider the case of ∆V mid = 0.5V and V gs = 0.7V ). This means that for every Volt of change in V CM the gain deviates by a unit (e.g. G = 25 at V CM = x V means G = 26 at V CM = (x − 1) V ). Nevertheless, for indicative values of G = 25 and V CM fluctuations in the low 100s of mV we obtain gain deviations/errors in the order of 1%.

III. PERFORMANCE MEASUREMENTS AND RESULTS
In this section, the suitably defined performance parameters from the previous section will be assessed for an example design in simulation. We split the results into two groups for convenience: differential mode and common mode effects. Under differential mode-related effects we examine the differential gain, bandwidth, and tuneable range/sensitivity of offset vs. RRAM device resistive state. Under 'Common moderelated effects' we include input range (largely determine by the common mode by assumption) and CMRR/CMGD. Finally, power consumption is discussed on its own at the end. For these simulations we used a commercially available 0.18µm CMOS technology with VDD = 1.8V .
A. Differential Mode Effects 1) Gain: For the purposes of amplifier gain analysis, we have run multiple, single data-point amplification transients sweeping a range of input differential voltages centred around zero. These simulations are under nominal conditions for this study: no added noise, mismatch or process variation was included.
There are two main experiments: First we set an integration phase run where δV in = 0 and the clk signal does not interrupt the integration process, but rather lets it run its course until both V mida/b saturate. Thus, the important features of the resulting waveform (e.g. position of peaks) are revealed. A key question we seek to answer here is whether there is an optimum time to stop the amplification in order to reliably obtain maximum gain, and if so when that occurs. The second experiment uses a fixed clock allowing us to explore the gain linearity for fixed integration period: we run multiple simulation runs with δV in swept from −100µV to 100µV with integration period τ = 150ns. The key question here is whether the amplifier has a usable linear range centred around the 0V differential input and if so, how wide it is.
The first experiment is illustrated in Fig. 5(a). We observe that for all test inputs δV in ∈ {−100, −50, −5, 5, 50, 100}µV ∆V mid increases linearly to a global peak at ≈ 170ns into the integration phase and then gradually decreases to zero, at which point both V mida/b have saturated and any potential difference they had is erased. The peak occurs because as we keep integrating, the voltage at mida/b nodes eventually increases to the point where the cascode transistors enter the triode mode. This causes the rate of voltage accumulation on whichever V mid node is highest to slow first, allowing the other node to catch up (and leading to the post-peak drop in ∆V mid ). At this point we are past maximum gain and continuing the integration eventually equalises the V mid s.
Next, we note that the peak gain time is nearly perfectly aligned for all input samples; the maximum peak time difference is only 1ps. The high quality of alignment arises because the time at which the V mid voltages start trioding the cascode transistors is determined primarily by the tail current and not the differential currents. The small discrepancy is explained by the fact that the peak gain time is technically determined by the time at which the first of V mida/b reaches the point where it triodes its cascode transistor. This has two key engineering implications: 1) It allows us to set a universally optimal DLC triggering time. 2) It states that the optimal trigger time is bounded by the trioding time obtained for V mida = min and V midb = max (or vice versa), in which case we have the fastest trioding corner.
The results from the 2nd experiment are shown in Fig.  5(b). The differential output voltages δV mid for τ = 150 ns are plotted versus input differential voltage δV in . We notice excellent gain linearity arising again from the extremely small effect that the differential voltages have on the behaviour of the voltages at V mida,b . For this experiment the differential input voltage was swept on the basis of a fixed input V midb = 1.  2) Bandwidth: We operated our amplifier with an integration period of 150ns as shown in Fig.3 and ran a collection of transient analyses for fixed amplitude pure tone signal inputs. The tone frequencies ranged from 1Hz to 27M Hz (covers around four cycles of window) and for each frequency the phases where stepped in increments of 10 o . Additionally we also carried out a DC run (δV in = 100µV ). For each simulation run we looked at the amplifier output δV mid after 150ns of integration. The outcome was a plot of maximum |δV mid | as a function of frequency, as illustrated in Fig.6 (normalised to |δV mid | at DC). The resulting curve is closely bounded by the envelope calculated by Eq. 11 indicating no surprises. To keep λ > 20%, the bandwidth achieves four fifths cycles/window period in Fig.6 that yields 5.4M Hz bandwidth.
We note that if we assume that the highest frequency spectral component of interest in a neural spike lies at 10kHz, the maximum attenuation of this particular design is around 0.08%. Therefore we can reliably sample spiking waveforms with this design.
3) Tuneable Range and Tuning Sensitivity: To obtain the tuneable range and sensitivity of implanted memristive devices, multiple transient simulations such as those seen in Fig.2 can be repeated while sweeping both RRAM device resistive states (R1 and R2). By tracking at what difference ∆V in the outputs flip value we can obtain an estimate for the offset. The quality of the estimate is calculated as follows: if at cycle n we had V outa = 0 and at cycle n + 1 we obtained V outa = 1, it means that somewhere between δV in |n and δV in |(n + 1) we crossed the amplifier's offset voltage. The tracking will be applied in both ascending and descending phase, after which offset voltage will be averaged. Assuming that the amplifier always makes a decision at approximately the same relative time in each cycle (in our case always at 150ns into the integration phase), the duration of this interval is fixed and given by the total swept range over the number of sampling cycles. In our case, we run 200 cycles (10µs/cycle for a total duration of 2ms) and sweep the input across a range of 400µV (200µV ascending and 200µV descending) Table II shows the offset voltage as a function of R1, R2. From there we observe: 1) The overall trimming range for this particular design is ≈ 235µV . 2) The maximum induced offset occurs, as expected, at the maximum R1, R2 imbalance corners.
3) The offset sensitivity is close to 1µV /kΩ for any combination of R1, R2. 4) The table is almost symmetric (as expected). The slight asymmetry indicates that the common mode voltage influences the offset voltage. This effect will be the subject of a dedicated study. Finally, the quoted offsets were checked and are the same both on the upward and the downward slopes, indicating no history-dependence.

4) Input-Referred Noise:
To estimate the noise behaviour we employ the following trick: we take the core of the basic circuit shown in Fig. 1, balance the inputs and add a pair of ideal, noiseless resistors that sink the baseline value of i tail/2 for some suitably chosen equilibrium voltage V mida/b = V equil within the amplifier's integrating range. This is shown in Fig.  7(a) (note that we have removed M17&M18 for simplicitythey only increment node capacitance by a small fraction). Then, we need to run our noise analysis and apply the sinc moderation (Eq. 11) in order to obtain our final results.
Before we begin, we need to make some key observations/assumptions: 1) At DC equilibrium, what is left on V mid after removing the baseline tail currents is fluctuations due 2) Any distortions introduced by the finite impedance of the compensation resistors is negligible due to the minute input signals at play. 3) Input-referred output noise levels are expected to be comparable throughout the entire integration range given that most of the noise is generated by the input differential pair. Additionally, running the noise test at half-gain is compensation against underestimating the noise generated by other sources (most notably the cascode pair). Now we can run our noise analysis.
For baseline compensation resistances R comp = 330kΩ, we get V equil ≈ 0.5V and a noise spectrum (with and without sinc moderation) as shown in Fig. 7(b). Across a [0.05Hz − 50M Hz] bandwidth we obtain a root-mean square (RMS) voltage noise level of ≈ 350µV unmoderated, dropping to 34µV moderated. This represents a saving of ≈ 90%. We also observe a 1/f corner frequency around 250Hz. We have tested that expanding the included noise bandwidth both to the left and to the right does not change the above figures significantly. The present analysis excludes noise contributions from the RRAM devices.
The overall result suggests that for neural probing, the noise levels obtained for this design may still be slightly too high, especially if we include additional noise from the RRAM devices. In this case switching to longer integration periods would help.  Fig. 8. Input range results of pre-amplifier. In this simulation, the common mode voltage was swept from zero to 1.8V with 50µV differential input. For V CM ∈ [0.5 − 1.4]V we notice that V midb (a) reaches sufficiently high voltage to prompt a stable output from the DLC (c) within 50ns of triggering, and for our chosen differential input the output is always correct. However, the analogue gain in (b) of the core is maximised in the narrower range [0.9, 1.3]V .

B. Common Mode Effects
1) Input and Range: In order to experimentally demonstrate the input range of the amplifier we performed a series of experiments querying different potential range limitation factors in practice. First, we checked the behaviour of the system at different stages as a function of common mode voltage by running a series of integration cycles whilst sweeping V CM from 0V to VDD in steps of 50mV . At each run the differential input was 50µV and the outputs were registered after integrating for 150ns. Results were registered at: i) V midb , ii) δV mid and iii) the overall system output after the DLC. Results are shown in Fig. 8. Note: in order to check for possible input signal history-dependence during these tests, each test integration cycle was preceded by three integration cycles ran with V CM = 1.8V . We have sample-tested a few runs with initial V CM between 0.1V and 1.8V and confirm that the history-dependence effect is negligible.
From the results in Fig. 8 we can draw three key conclusions: 1) The DLC successfully triggers for V CM between approx. 0.5V and 1.4V . This means that V midb is sufficiently high for the DLC to settle to an output within 50ns of it triggering (which occurs when clk goes high). 2) In this case the DLC provides the correct answer so long as it triggers, but this might change towards the edges of the range once we take noise into account. 3) The actual analogue gain of the amplifier remains close to maximum (≈ 28dB) within a narrower region: approx[0.9, 1.3]V . We would recommend that maximum gain area is taken as the effective V CM range in order to maximise the chances of correctly capturing small differential inputs under noisy conditions. Nevertheless, this shows that by de-rating the specification of the amplifier to higher δV mid we can extend its effective input range.
In order to visualise the effects leading to loss of gain outside the region V CM ∈ [0.9, 1.3]V we ran some unrestricted integration tests as shown in fig. 5 for different values of V CM . The results are shown in Fig. 9 where we observe that for V CM between 1.0V and 1.3V the integration traces follow each other very closely, with traces at 0.9V and 1.4V beginning to show more substantial deviations. We note how excessively low V CM s shorten the peak without shifting (a result of desaturating the input differential pair but not changing the integration range) whilst excessively high V CM s shift the peak without changing its magnitude.
2) CMRR and CMGD: For evaluating the CMRR we set the differential input to 0V and swept V CM between [0.9, 1.4]V. Since we deliberately don't account for process variations and mismatch in this work, we obtain the expected common mode gain of 0.
For CMGD, we run a series of integration runs with fixed differential input voltage (50µV ) and sweep V CM in steps 10mV and plot the gain as illustrated in Fig. 10(a). The highlighted region where the gain maximises is then resampled at 5mV step and for each consecutive pair of data points we calculate the derivative. As per Eq.33 this yields our CMGD. Converting appropriately we obtain CMGD≥ 20dB for V CM ∈ [0.99V, 1.14V ]. To exemplify this effect, a 0.15V change in common mode voltage ∆V CM causes less than 1.5% change in the output of the amplifier core ( dG dV CM ∆V CM ).

C. Power Consumption
The power consumption has to be assessed for all operating phases of the pre-amplifier. The most power-hungry phase is the reset phase since it is the only one where a DC path exists between the power supplies. For this reason the reset phase should be kept as short as possible. However, it is also during the reset phase that the core amplifier reaches steady state at all nodes so that the integrating phase can then commence without any history-dependence, i.e. influence from or 'memory of' its previous inputs. Finding the optimal reset phase duration is a key optimisation task for this design. Next, the cost associated with the integration and digitisation phases can be split into two main components: First, the integration cost is equal to charging the core amplifier's capacitors from GND to their equilibrium level, where the integration selfterminates (≈ 1.26V in our case -note how this integration cost currently spans both integration and digitisation phases because we do not stop the integration once we trigger the DLCs). Second, the comparison cost is equal to the energy needed to operate the DLC. Finally, during the 'off' phase power dissipation is mainly down to leakages. Through one detection cycle (350ns), the average energy consumption is 1.927pJ, of which 663f J during the reset phase, 814f J during the integration phase and 450f J during digitalisation. This yields a power rating of 5.5µW for continuous operation (no off phase), of which the core amplifier accounts for 5µW . If we operate the amplifier at typical biointerface sampling rates of ≈ 20kHz, power dissipation becomes 38.5nW (assuming practically zero 'off' mode dissipation).
For a more complete, multiple channel pre-amplifier, additional power will be dissipated by 1) the current reference generation unit (III in Fig.1), 2) the control system, including clk ana, clk anabar, clk rst and clk generators. Both of the above would be shared across multiple channels, yielding a certain degree of amortisation.

IV. DISCUSSION
From the analysis and simulation of the integrating amplifier we highlight some key conclusions: First, the performance improvement of the integrating amplifier over more traditional e.g. Harrison designs relies on the integration process, which enhances the gain and decreases the effective bandwidth (helping reduce noise in the process). To visualise this let us consider an integrating amplifier using the same tail current as a standard OTA first stage. During integration the power dissipation is effectively the same, but the gain and bandwidth are different. In this sense the design represents a trade-off between gain and bandwidth without changing power dissipation or using feedback.
Next, we note that there is a natural trade-off between tail current and integration time while keeping the overall energy dissipation approximately constant. This is the result of the fixed duration of the reset phase (just enough to clear any residual charge at the V mid nodes) and the fact that energy consumption during the integration phase only depends on the size of the load caps and the voltage change across them during that phase. Thus, in principle we can design for a wide range of required sampling rates or bandwidths for the same energy budget.
The trade is not completely free: Changing the tail current affects gain, bandwidth and noise performance, by altering the g m s of all transistors involved and the integration period. Furthermore, if using real RRAM devices with non-linear IV curves, changing the tail current also changes the static resistance of the RRAM devices. Together with changes in transistor g m s this means that the tuneability range is also affected since it depends on the impedance balance between RRAM and transistors. Thus, whilst the integrating amplifier clearly offers a lot of design flexibility, the precise design trade-off space is also not trivial, much like as it is for OpAmps. This is an important subject meriting its own dedicated study.
The last design decision to highlight concerns the size of the load capacitors C. The gain analysis in section II shows that C doesn't affect the gain, but it does affect the integration period and therefore can be used to adjust the bandwidth, if for some reason that cannot be achieved by tweaking the tail current. Effectively it is a design parameter that trades away energy for design flexibility.
In terms of operation, we note the importance of ensuring that the integrating amplifier is cleared properly in preparation for each integrating phase in order to avoid history-dependence of the output. This means that all node voltages should be equalised across the left and right branches prior to the commencement of the sensitive integration phase. In the current design this is achieved by forcefully flushing the system during the reset phase, but more energy-efficient approaches are under development as the rest phase represents a substantial fraction of the energy budget.
Finally, we compare our amplifier's performance with a few standard designs as shown in Table III. We observe a slightly reduced gain and increased noise levels traded against power dissipation as a result of our design decisions so far. Importantly, for relatively low precision operations such as threshold detection of neuronal spikes a 10-fold increase in noise may be an acceptable price for a 100-fold reduction in power dissipation. We also note that the present design is not completely optimised, with an increase in integration time as a very promising avenue of investigation for decreasing noise levels within the same power envelope.

V. CONCLUSION
In this work we have performed a theoretical analysis of the core functionality of memristive integrating amplifiers and used industrial CAD-level simulations to provide a specific example for an integrating amplifier design targeting electrophysiological applications. Throughout our analysis we have concluded that the performance enhancement over traditional, continuous mode amplifiers can be most intuitively understood as a gain boosting effect arising from the integration process and showed how this process erodes the amplifier's effective bandwidth (which is desirable for electrophysiology applications). Moreover, we have explained how standard metrics of amplifier performance such as gain and input common mode range, but also new metrics such as offset voltage tuneability range can be described by governing equations for use by designers. Finally, we implemented an exemplar design in commercially available 180nm CMOS and demonstrated typical values for all studied performance parameters that can be expected from a 0.18µm node technology. These included gain of 25V /V , offset tuning range of 235µV , input-referred noise of 34µV rms and power dissipation of 38.5nW at 20kHz sampling rate. These are competitive vs current literature for an not fully optimised design.
This work is a stepping stone towards de-risking and documenting the RRAM-based integrating amplifier. We believe that the trade-off induced by the integration process in combination with the offset trimming enabled by RRAM has the potential to add a powerful circuit topology to the arsenal of the analogue designer.


Analysis and Construction of EEMD Smart Model and Fuzzy Forecasting through Improved Bayesian Estimation

Different portfolio models will have a great impact on the model rate of return. In order to meet the data needs of portfolio investment forecasting, a big data generation method based on incremental Bayesian network model is proposed. Firstly, the fuzzy interval number is constructed by using the global GM (1pare 1) prediction model, then the fuzzy Mmurv model is established based on the interval fuzzy number, and the model is optimized based on the midpoint, radius and acceptability of the interval number, thus a single objective programming model with parameters is obtained. The future data is partially generated by using the time series generation algorithm, and the Bayesian network model trained by historical data is updated with the newly generated data. So that the updated Bayesian network can reflect the relationship between the variables and the laws contained in the new and old financial data in this period of time. And use the model to predict the amount of foreign direct investment absorbed by China in 2020, test the robustness and analyze the impact of major random events to show the reliability of the prediction. The analysis is carried out on multiple scales. The results show that the integration of IMF1 and IMF2 components through the white noise test can be regarded as the random impact of external factors; the cycle length of IMF3 component is about 3 years, which can be regarded as the inventory cycle caused by the change of enterprise inventory; the cycle length of IMF4 component is 5 years, which can be regarded as political cycle; the cycle length of IMF5 component is about 20 years, which can be regarded as construction cycle. The set of portfolio paths is generated by path search algorithm in Bayesian network, and the big data set with real data characteristics is generated according to the probability distribution of each path. The experimental results show that the method is feasible and ensures a certain accuracy. The forecast results show that the amount of foreign direct investment in China will continue to fluctuate seasonally and in a trend in recent years.

Introduction
Theory and practice show that economic growth often presents a cyclical cycle characterized by expansion, contraction, recession and recovery. Both the academia and the political circles are actively exploring the internal mechanism of the economic cycle fluctuations, looking for good measures to deal with the sharp economic fluctuations, so as to reduce its negative impact as far as possible. The research on periodic fluctuation of economic time series focuses on time domain analysis and frequency domain analysis.
Real financial data are generally expressed in the form of time series. At present, the main methods of time series data simulation and generation are: regression moving average model, autoregressive conditional heteroscedasticity model, support vector regression model, long-term and short-term memory network model and so on. As the data generated by the simulation in this paper show obvious quarterly or monthly periodic changes, the seasonal differential autoregressive moving model will be used to automatically simulate and generate the time series data. Supplement part of the future data for the original historical data set and update the subsequent portfolio investment data generation model.
In this paper, the overall GM (1,1) prediction model of the interval fuzzy number is used to predict the return rate and liquidity of stocks, and the corresponding interval fuzzy number is obtained. On this basis, a portfolio selection model based on interval fuzzy number is constructed, and the validity of the model is verified by an example analysis.
The financial data used in this paper is essentially a kind of time series data. So, when it comes to generating extensions to a dataset, the data can be generated by training the time series model. At present, there are time series models and time series analysis methods, which reveal the law of phenomenon change with time through the historical statistical data of the series, and extend the law into the future according to the need. By giving different economic meanings according to the component characteristics, we can identify the different cycles of fixed asset investment fluctuation and study the sub-series of different time scales, so as to find out the hidden cycle fluctuation law of fixed asset investment fluctuation and dig out the deep reason of fixed asset investment fluctuation. The subsequent prediction analysis on this basis can, on the one hand, enrich the application research of EEMD model in economic sequence analysis and prediction in theory, and on the other hand, improve the accurate understanding of China's fixed asset investment fluctuation cycle in practice, and provide more scientific support for the formulation and introduction of macro-control policies.

Time series generation algorithm
As one of the manifestations of data, time series data truly records all kinds of important information at different time points (or time slices), which contains rich and valuable knowledge. This paper uses SARIMA to generate multiple columns of financial time series data. SARIMA model is derived from the Differential Autoregressive Moving Average (ARIMA) model. In the ARIMA (p, d, q) model, p is the number of autoregressive terms, q is the number of moving average terms, and d is the number of difference times for time series to become stationary series. If the time series {Yt} is a non-stationary series, the ARI-MA model can be expressed as: is the moving average coefficient, the random term εt is an independent white noise sequence, and obeys the normal distribution N (0, σ2ε), and the random term is correlated with the hysteresis variables Yt-1, Yt-2..., YT-P is not correlated. SARIMA (P, D, Q) (P, D, Q) S model is mainly used to analyze the time series with periodic changes caused by periodicity (including weekly, monthly, quarterly, annual, etc.) or other factors, and to carry out seasonal difference based on periodicity on ARIMA model. Let the change period of seasonal series be s, and the seasonal difference operator is defined as BS is s step lag operator. If the seasonal time series is expressed by {Yt}, the primary seasonal difference is expressed as If {Yt} is non-stationary seasonal time series, D-order seasonal difference is required, and a P-order autoregressive Q-order moving average seasonal time series model with a period of S is established In the existing SARIMA model building methods, the stationarity test and processing of the data are usually carried out first: to observe whether the sequence is a stationary time series; if the sequence is non-stationary time series, ordinary difference is required for the original sequence and seasonal difference, and thereby determine the parameters d, D; By observing the truncation or trailing characteristics of the autocorrelation and partial autocorrelation functions of time series. Most of the above steps require manual observation to determine the value of parameters, and for different time series data, the parameters of the model will be different. However, through literature reading and many experiments, it is found that the range of values of p, q, d, p, q and d is relatively conventional. Therefore, grid search method is used in this paper to determine the optimal parameter combination of the model. Then, the determined parameter combination is used to check and test the model. If the relative error between the predicted value and the actual value obtained from the data in the practical test set is less than 5%, it indicates that the established model has a high accuracy, and the model can accurately simulate and generate future data.

Bayesian Network algorithm for data Generation
The incremental learning method is used to update the Bayesian network model. The main idea of this method is that a certain amount of data samples are randomly selected from the historical data and combined with the new data set to train the Bayesian network structure, and then the Bayesian network structure is combined with the historical model to perfect the Bayesian network structure.

Figure 1. Historical data generated by a Bayesian network
Due to the inconsistency of time and frequency of data, we resampled the mean value according to the month. In addition, due to the need of constructing Bayesian network, the data is discretized. First, make first-order difference for data. For time series data {Xt}, the p-order difference operation formula is

Model result
The data values of central bank deposit, loan interest rate and deposit reserve ratio are relatively stable, and the weight random generation method is adopted, which is more in line with the reality than random generation method. In this experiment, data of 3 years are generated.

Figure 2. Data generation results
SARIMA algorithm is used for the remaining data to generate new data for subsequent Bayesian network updates. Since there are many types of data generated, the first column of data currency and the change amount of quasi-money supply M2 are taken as examples to show the generation process of new data in the next 3 years through learning historical data by using SARIMA time series algorithm. Since the processed data is relatively stable but the value is large, Min-Max standardization is used to process the data in the experiment.
After data processing, parameters of Sarima (p, d, q) (p, d, q) s model were determined. Since the first-order difference has made the data stable, d = d = 1, and s is the period of time series, monthly data are used in this experiment, and s is 12.Algorithm 1 is used to determine the remaining parameters, and the determined parameter combination is put into the model to predict the two-year data.  In order to make the sequence stable, the original sequence Y is processed by first-order difference, and the sequence DY is obtained. From figure 4, we can see that the sequence DY after difference always fluctuates up and down a constant value, and the trend is eliminated. Further unit root test shows that the original hypothesis that there is a unit root is rejected at this time, and the sequence DY is stable.
It can be seen from the seasonally adjusted time series diagram that the fluctuations affected by seasonal factors decrease, but the truncation and trailing trends of autocorrelation and partial autocorrelation graphs are still not obvious, and attempts to fit the ARIMA model have poor results. It is observed that AC and PAC are still large at the 12th order delay, which is still obvious in the figure. It can be considered that there is still seasonal effect in the seasonally adjusted sequence. Therefore, the short-term correlation of this sequence has complex correlation with seasonal effect, which cannot be extracted simply. It can be seen that although the actual value is different from the predicted value, the relative error is small, the average error is 3.5%, and the prediction accuracy is good. For the sake of safety, the monthly data of FDI from 2002 to 2019 are predicted by ARIMA model to see the overall fitting effect. Due to the difference of the data, the range of predicted samples is reduced. It can also be seen that the covariance ratio of the model is 0.9865), which is much larger than the bias ratio and variance ratio, so the fitting effect of the model for each annual historical data is also ideal as a whole.

Discussion and analysis
In the short term, in the face of major random events such as the epidemic, it is imperative to support and help foreign enterprises to resume normal production and operation. We need to solve the difficulties of resuming work and production of foreign enterprises in a timely manner, and implement preferential policies such as taxation to offset the impact of the epidemic.At the same time, we will ensure the implementation of major landmark foreign investment projects, make full use of various investment agencies and platforms, and continue to promote investment promotion and investment attraction. In the long run, the first step is to further expand the scope of foreign investment encouragement and guide the industrial restructuring of foreign direct investment.While the outbreak has impacted international economic and trade activities, it has also brought opportunities for China to accelerate the development of science and technology and promote industrial optimization and upgrading. It is conducive to promoting foreign investment in bio-medical and high-tech fields.

Conclusions
The prediction results in the sample show that the fitting effect of the model is good, and the model is used to predict the amount of foreign investment in China in recent months, and the results show that the future FDI inflow in China will maintain a relatively stable combined fluctuation state of seasonality and trend, and the time series generation algorithm can generate multi-column time series data automatically and accurately. The incremental Bayesian network is constructed, so that the Bayesian network can be updated dynamically with the generation of new data, and it is convenient to generate long-term accurate data later; the path set of each node is generated by path search of Bayesian network, and the big data set is generated on the basis of the value of each path node in the set and its corresponding probability. EEMD model is used to decompose the fixed asset investment series in multiscale, and each component is divided into random shock, inventory cycle, political cycle, construction cycle and long-term trend items according to its fluctuation frequency or cycle length. Finally, the robustness test is carried out, and the impact of major random events is analyzed to further test the reliability of the prediction results.


extfieldsofstudy:
Computer Science
Engineering

text:
NICE-Beam: Neural Integrated Covariance Estimators for Time-Varying Beamformers

Estimating a time-varying spatial covariance matrix for a beamforming algorithm is a challenging task, especially for wearable devices, as the algorithm must compensate for time-varying signal statistics due to rapid pose-changes. In this paper, we propose Neural Integrated Covariance Estimators for Beamformers, NICE-Beam. NICE-Beam is a general technique for learning how to estimate time-varying spatial covariance matrices, which we apply to joint speech enhancement and dereverberation. It is based on training a neural network module to non-linearly track and leverage scene information across time. We integrate our solution into a beamforming pipeline, which enables simple training, faster than real-time inference, and a variety of test-time adaptation options. We evaluate the proposed model against a suite of baselines in scenes with both stationary and moving microphones. Our results show that the proposed method can outperform a hand-tuned estimator, despite the hand-tuned estimator using oracle source separation knowledge.

INTRODUCTION
Beamforming provides a powerful way to leverage the spatial diversity of a microphone array to enhance, separate, dereverberate or otherwise process multi-channel data [1]. Recently, mobile microphone arrays mounted on devices such as headsets, hearing aids, and other wearables have been leveraged to perform these tasks [2]- [4]. However, these devices present challenging conditions for multi-channel speech processing due to their mobile nature. While adapting to source movement is well studied [5]- [7], compensating for arraymovements, such as pose change, is less explored. Successfully modeling such pose-varying and spatially dynamic scenes requires integrating both spatial (e.g. pose) and temporal information. Processing is further complicated by the fact that spatial statistics are dependent on both the pose of the array and the current sound source activity. This implies that a single pose-change renders it difficult to leverage previously collected statistics in a conventional linear framework (such as buffered or recursively averaged statistics). In this work, we aim to learn a non-linear covariance estimation algorithm that is robust to pose-changes.
In spatially stationary scenes, spatial statistics can be aggregated and estimated across multi-second windows. However, in spatially dynamic scenes, the statistics must be re-estimated in a rapid manner since long windows can span various spatial responses. Developing estimators which can rapidly re-estimate statistics for spatially stationary scenes is an area of active research. These developments † Work performed while at Reality Labs Research. include approaches that are more computationally efficient [8], probabilistic [9], [10], and able to incorporate both long and short-term information [11]. Several recent works have studied the effects of pose-change in spatially dynamic scenes on conventional beamformers. In the case of uniform circular arrays, sound-field interpolation is effective for rotation compensation [12]. With more complex arrays, techniques based on spherical harmonics can estimate rotation-invariant covariance representations for uncorrelated noise fields [13]. When the array is not rigid, pose-change may induce a deformation, which can be compensated for [14]. These algorithms often rely on good spectral separation estimates, such as those produced by a voice-activity detector or a ratio-mask estimator.
The growing success of deep neural networks (DNN) in singlechannel spectral separation and speech enhancement has led to several works integrating single-and/or multi-channel enhancers into beamforming pipelines for spatially stationary scenes [15], [16]. These works leverage a two part procedure where a pre-trained DNN mask-estimation module is inserted into an existing beamforming pipeline and used for spectral estimation. Training modules of a beamforming pipeline in one-step or end-to-end has proved challenging due to the numerical instability of spatial covariance matrix inversion, but has lead to improved speech recognition performance [17]. Moreover, it is also possible to do away with the beamforming structure entirely and learn DNN based multi-channel modules that learn to leverage spatial information implicitly [18]- [20]. However, some works have found that it is advantageous to incorporate explicit spectral and spatial estimation steps either in the form of explicit masking and beamforming [21], [22] or more abstractly [23]. The impact of motion, such as pose change, on these methods is, to the best of the authors' knowledge, unexplored.
We explore training online covariance estimation methods for scenes with pose changes and propose NICE-Beam, a hybrid DNN and Digital Signal Processing (DSP) system capable of joint enhancement and dereverberation. Inspired by work in learning optimizers [24], we develop a covariance estimator based on a recurrent neural network (RNN) and use it within a conventional pipeline composed of a single-channel enhancer and a beamformer. We train the estimator in an end-to-end fashion and sidestep previously encountered numerical instability issues by working directly on the inverse of the covariance matrix. We demonstrate our approach on a time-varying-spatialized version of the Microsoft DNS Challenge dataset [25] and show that this approach is robust to pose-changes and surpasses the performance of several conventional methods in terms of perceptual and objective metrics. We also find that our method can be improved by providing additional information to the learned estimator. We note some advantages of this hyrbid DNN-DSP approach by showing that our model is test-time modular, and recovers gracefully from hardware failures. Finally, we confirm the practical feasibility of our technique by testing that it can operate online and faster than real-time with low computational requirements.

NICE-BEAM FORMULATION
Let xm[t, f] ∈ C where m ∈ {1 · · · M } be the short-time Fourier transform (STFT) of a signal at time t and frequency f received by microphone m of a moving M channel array. This signal can be decomposed as f] represents some time-varying impulse response w.r.t microphone m, nm[t, f] is noise, and * is the convolution operator. The goal of our beamformer is to recover s[t, f] via a spatial filter w[t, f] ∈ C M applied to the reverberant mixture (w H [t, f] · xm[t, f]). In this work, (·) H is the Hermitian operator.
The spatial filter, w[t, f] is estimated using the spatial covariance matrices of the speech (Φss[t, f]) and noise (Φnn[t, f]). Typically, it is assumed that the speech/noise covariance matrices are time-invariant and can be computed in a simple running manner using speech estimateŝ[t, f] ∈ C M and noise estimaten[t, f] ∈ C M . However, due to the time-variation of dm, this is not a valid assumption. In this work, we explore using a RNN to estimate speech and noise covariance matrices in an online fashion by nonlineary aggre- We omit the f index when possible.
Filtering in the spatial domain enables constraints which improve beamforming performance. As such, translating the RNN output into something suitable for the computation of a spatial filter is a key contribution of this work. We explore three interpretations of the RNN output and discuss how they correspond to enforcing different degrees of structure. In each of these formulations, we operate on the inverse noise covariance and effectively sidestep previously discovered issues of training RNNs with a beamformer. We call this approach NICE-Beam. Let g θ (n[t], h[t]) represent the application of a multi-layer RNN parameterized by θ, with h[t] ∈ R D as the D dimensional time-frequency dependent state, to an estimate of the noise,n[t]. We omit noise and speech specifiers when possible.

Learned Rank-1 Update Estimates
First, we explore interpreting the network output as a rank-1 symmetric update to an existing Φ −1 [t − 1]. This is done by configuring g θ (·, h[t]) to output φ[t] ∈ C M and computing the update as The update in (1), preserves most of the structure of a covariance matrix and is a natural first step towards a learned module. In subsequent sections we call this approach "Rank-1".

Learned Cholesky Estimates
Second, we allow the network to manage updates internally, in a non-linear fashion. Though, we enforce that the output be a valid inverse covariance by predicting it's Cholesky decomposition. In this formulation, we interpret the network output, ϕ[t] ∈ C M ×M , as a lower triangular matrix within a Cholesky decomposition. To get a lower triangular matrix we zero out elements above the diagonal and pass elements on the diagonal through the absolute value function.
The estimate is then given as whereφ[t] is the lower-triangular absolute-value modified ϕ[t]. In later sections we call this approach "Cholesky".

Learned Arbitrary Estimates
Lastly, we explore removing all constraints and allow the RNN to output an arbitrary square matrix, ξ[t] ∈ C M ×M . Specifically, We call this approach "Arbitrary" in subsequent sections as any structure within the estimate is learned and not by construction.

Training and Leveraging Learned Estimators
To train a NICE-Beam estimator, we construct the pipeline from Fig. 1. Inputs are passed through an enhancer, used to estimate a covariance, and then fed to a beamforming module. We compute a loss on the beamformer output, since it is not clear what the optimal covariance estimate is. However, we do know the desired output (noise-less dry speech). Our formulation produces online estimators, which are custom learned for the training data and metrics. This allows us to train estimators, which specialize in speech separation for wearable arrays of a specific geometry and outperform more generically formulated estimators. Another advantage of this approach is the ability to leverage additional input signals by passing additional inputs to g θ (·) besides the current spatial STFT frame and state. We explore providing adjacent frequency information when computing φn[f, t]. Specifically, we use information from frequencies adjacent ton[f, t] and learn a frequency information sharing , parameterized by ψ, whose output is passed to g θ (·). Since we are training a custom estimator for speech signal enhancement, f ψ (·) could learn to leverage the spectral structure of speech. The particular selection of enhancer, beamformer, loss function, g θ (·), and f ψ (·) is discussed in the next section.

EXPERIMENTAL SETUP
To evaluate our learned online covariance estimators, we implement the pipeline shown in Fig 1. This pipeline is composed of an enhancer, covariance estimator, and a beamforming module. To isolate analysis of the covariance estimator, we perform initial ablations and experiments using a fixed semi-oracle enhancement module. We later extend our analysis to DNN based enhancers. For all evaluated pipelines, we use the minimum variance distortionless response (MVDR) beamformer, 512 point Hann analysis/synthesis window, 256 point hop size and a sampling rate of 16 kHz. We train separate models for spatially stationary and dynamic scenes.

Datasets
We created a spatialized version of the the DNS challenge dataset [25] using pyroomacoustics [26] for simulations. Each scene lasts 5 seconds, contains 1 to 3 noise sources, 1 speech source, and is randomly mixed at SNR between −5 dB and 5 dB. We simulate rooms with dimensions uniformly distributed between 4 m and 8 m, and with a T60 between 0.25 s and 0.75 s. Within each room, we simulate a 7 cm diameter circular microphone array with 6 microphones. For the dynamic dataset, we simulate array rotation by computing fixed impulse responses at 250 rotations and perform a time-varying convolution. We generate rotation patterns according to a Markov model derived from the EasyCom dataset [27]. To prevent leakage we use separate room geometries, rotation patterns, and speech/noise files for train/validation/test. We generated two separate datasets, one spatially static (no rotations) and one spatially dynamic (with rotations). The train, validation, and test set splits of each dataset contain 20, 000, 5, 000, and 5, 000 scenes respectively. Steering Vector Fig. 1. NICE-Beam processing pipeline composed of an enhancer, covariance estimator (inside the dashed box), and a beamformer. We use separate covariance estimators for speech/noise. The estimator outputs are translated into spatial covariance matrices and fed to the beamformer. In the covariance estimator block, blue denotes an inter-frequency operation whereas yellow denotes an intra-frequency operation.

Enhancer
To evaluate the effect of enhancement quality and isolate analysis of the covariance estimator, we experiment with semi-oracle, learned and oracle enhancers. For the semi-oracle enhancers we use an IRM (ideal-ratio mask) computed using either the echoic (Echoic IRM) or anechoic (Anechoic IRM) clean speech. For the learned enhancer, we predict a single-channel real-valued mask using a 3layer LSTM with hidden size of either 256 or 512 trained with SI-SDR [28] loss w.r.t the anechoic clean speech. This is a simple but capable model. All masks are applied via M xm [t] where M is the mask and is the element-wise product. For the Oracle enhancer, we setŝ = s andn = x − s. The IRM and oracle enhancers use ground truth knowledge and only serve as comparison points. The LSTM serves as an example of a suitable model whose performance lies between the Echoic and Anechoic IRMs.

Beamformer
For beamforming, we use MVDR with steering vector based on the estimated relative-transfer function. When running NICE-Beam, we use the normalized first column of the target covariance matrix as a steering vector. We do this to reduce computational complexity during training. When testing all other covariance estimators, we use the normalized largest principal component of the target covariance matrix. We denote the extraction of this time-varying steering vector with the operation P(·). We then compute MVDR weights as where v[t] = P(Φss[t]).

Covariance Estimates
We experiment with two conventional covariance estimators as well as learned methods. On the conventional side, we experiment with offline Fixed and online Buffered estimates, which we pair with the enhancers discussed above. For the Fixed case, we compute a single acausal estimate across the entire scene. For the Buffer case, we use a sliding buffer to compute an online time-varying estimate. In each experiment, we tune the buffer size with a grid search over [5,50] on the validation dataset. We tested exponentially smoothed updates but found they performed worse and chose not to include them. The learned estimators, g θ (· · · ), are composed of a 2-layer D = 128 dimension hidden size unidirectional LSTM. We stack the real/imaginary components of each STFT bin as input. On the output, we predict the real and imaginary parameters independently using a set of linear layers. When experimenting with frequency information sharing we set f ψ (· · · ) to be a stack of three convolutional layers each with kernel size 3 and 64 channels. The convolutional layers run across frequency while the LSTMs run on each time/frequency independently effectively performing parameter sharing across all frequencies. This makes the model independent of the STFT frame/hop but constrained to a predetermined number of microphones. We implemented this framework in PyTorch.

Training
We use an identical training recipe for training enhancers and estimators individually and jointly. We use SI-SDR loss for training and early stopping and cease training after 50 epochs. We use the Adam optimizer with a learning rate of 3 × 10 −4 , gradient clipping, 16 bit parameters and set the batch size between 8 and 16 depending on model size. We performed tuning on the validation set and did not use the test set until all hyperparameters were finalized. Using one Nvidia P100 GPU, it would take 2 to 5 days to train a model.

RESULTS
We evaluate our approach in spatially stationary and dynamic scenes. For comparison, we construct several validation-set-tuned conventional estimators. We also evaluate the test-time modularity of our approach by modifying portions of the processing pipeline without retraining. In all, we find that NICE-Beam estimators outperform their conventional counterparts by a significant amount in all metrics, are modular, robust, and computationally efficient.

Static Scenes
First, we study static scenes and evaluate all methods using SI-SDR, STOI, and PESQ. For this initial analysis, we equip all learned estimator models with a semi-oracle Echoic IRM as the enhancer. These results are shown in Table 1. The first grouping (rows 0-3) display the results for a reference microphone and three single-channel models, which do not estimate spatial statistics. The next grouping (rows 4-7) shows results for a variety of conventional estimators paired with a variety of enhancers. We evaluated all pairs of enhancers/estimators and chose to only show the best. The third grouping (rows 8-9) shows a NICE-Beam estimator which predicts Rank-1 updates and how incorporating learned information from adjacent frequencies (+F-Info.) improves performance in all metrics. Based on this, we use learned frequency information sharing in all other NICE-Beam models. The final grouping (rows 10-11) demonstrates that relaxing the enforced structure of the estimator and shifting all covariance processing within the learned module improves performance. Both the Cholesky and Arbitrary models outperform the conventional estimators equipped with perfect source separation knowledge. This is remarkable since the learned modules are causal and rely on worse speech/noise estimates.
The model in row 4 corresponds to [15], [16] where a DNN enhancer is combined with a conventional covariance estimator. Our proposed method is compatible with this approach as we demonstrate in the next section where we study spatially dynamic scenes.

Dynamic Scenes
Next, we study spatially dynamic scenes. These results are shown in Table 2 and can be compared to Table 1. The single-channel approaches in rows 0-3 are minimally impacted by spatial changes since they do not perform spatial processing. The next grouping (rows 4-7) shows results for several pairs of non-learned estimators and enhancers. Unsurprisingly, the fixed estimators shown in rows 4 and 5 drop in performance from static to dynamic scenes. This occurs since collected statistics are dependent on array pose. The buffer based model shown in row 6 maintains performance as it was re-tuned for dynamic scenes. The second to last grouping (rows 7-9) shows NICE-Beam estimators with Rank-1, Cholesky and Arbitrary structure. While the Rank-1 estimates (row 7) do not outperform the models with oracle knowledge (row 6), the Cholesky (row 8) and Arbitrary (row 9) estimates do. Though, they do drop in performance when compared to their results in static scenes. Interestingly, the performance drop is largest for the Rank-1 model. We hypothesize that the fully learned state of the Cholesky and Arbitrary models can adapt more rapidly to motion. Though, sometimes additional structure is useful. The Cholesky model scores marginally better in perceptual metrics than the Arbitrary model, indicating that Cholesky structure could be more faithful to the MVDR objective.
In the final grouping (rows 10-11) we replace the Echoic IRM, a practically unrealizable enhancer, with an LSTM enhancer. In effect, we create a NICE-Beam model with multiple learned components and extend [15], [16]. Though, instead of a two part training procedure we train the enhancer and estimator jointly, in an end-to-end fashion. This model is entirely realizable as it is online and uses no ground truth knowledge. The only other comparable models are in rows 2 and 4. To push performance, we double the estimator state size from 128 to 256. This largest model achieves our best dynamic scene SI-SDR of 7.13 dB. We believe these models could perform better with a more exhaustive hyperparameter search and tuning.

Test Time Adaptation
Our approach has interesting modularity properties which enhance it's real-world usability. Since it is part of a modular pipeline, we can modify modules at test time. We study such modifications in dynamic scenes and show results in Table 3. Our initial model shown in row 0 (same as Table 2 row 9) is the Arbitrary + F-Info model trained with a 512 window, 256 point hop, and Echoic IRM.
First, we experiment with doubling and halving the window and hop size at test time (rows 1-4). Effectively, we evaluate test-time latency and resolution adaption. Next, we experiment with changing the enhancer at test time. In row 5, the masker is changed from an Echoic IRM to an Anechoic IRM. This is a case of swapping in better enhancers at test-time without retraining. However, the enhancer modification can not be too large. In row 6, we provide oracle separation and all metrics drop. We believe this stems from the model being trained with an IRM based enhancer. Finally, we study robustness and simulate hardware failures by replacing a random microphone signal with white noise (row 7). As expected, all metrics drop, however, they remain higher than the Echoic-IRM (Table 2 row 1) and buffer estimator, which, in our experiments, errored out.

Practical Feasibility
The estimators have 550 K parameters and use 2.5 MB of storage. The computational budget is 80 MFLOP per frame or 2 GFLOP per second. They can run faster than real-time with a real-time-factor of 0.05 on an Nvidia P100 GPU and 0.9 on a 2.4 GHz Xeon CPU.  Table 3. Results evaluating test-time modularity. All adaptation is tested on the Arbitrary + F-Info model using the Echoic-IRM.

CONCLUSION
In this paper, we proposed NICE-Beam, a learning based method for estimating time-varying spatial covariance matrices, with application to joint speech enhancement and dereverberation. We applied these methods on a dataset containing both spatially -stationary and -dynamic scenes. Our empirical results showed that the proposed methods outperform a variety of conventional estimation approaches. Moreover, by incorporating the learned modules into a conventional beamforming pipeline, we can perform real-time inference, and apply test-time adaptation with respect to latency, resolution, and masking modification. The model is also robust to hardware failures. We believe our proposed method could also be a valuable tool for complex spatial processing in other domains.