s2fieldsofstudy:
Computer Science
Physics

text:
Real-time audio processing of real-life soundscapes for EEG analysis: ERPs based on natural sound onsets

With smartphone-based mobile electroencephalography (EEG), we can investigate sound perception beyond the lab. To understand sound perception in the real world, we need to relate naturally occurring sounds to EEG data. For this, EEG and audio information need to be synchronized precisely, only then it is possible to capture fast and transient evoked neural responses and relate them to individual sounds. We have developed Android applications (AFEx and Record-a) that allow for the concurrent acquisition of EEG data and audio features, i.e., sound onsets, average signal power (RMS) and power spectral density (PSD) on smartphone. In this paper, we evaluate these apps by computing event-related potentials (ERPs) evoked by everyday sounds. One participant listened to piano notes (played live by a pianist) and to a home-office soundscape. Timing tests showed that the temporal precision of the system is very good. We calculated ERPs to sound onsets and observed the typical P1-N1-P2 complex of auditory processing. Furthermore, we show how to relate information on loudness (RMS) and spectra (PSD) to brain activity. In future studies, we can use this system to study sound processing in everyday life.

Introduction
Mobile electroencephalography (EEG) allows to record brain activity beyond the lab while participants go about their everyday life (De Vos, Gandras, & Debener, 2014;Debener, Minow, Emkes, Gandras, & de Vos, 2012;Hölle, Meekes, & Bleichner, 2021;Wascher et al., 2021). It thereby provides unique insights into 1 human cognition that cannot be gained by introspection or by the observation of behavior alone. Importantly, it can help us to understand the coupling of action and cognition (Gramann, Ferris, Gwin, & Makeig, 2014;Ladouce, Donaldson, Dudchenko, & Ietswaart, 2017;Parada, 2018). In everyday life, action and cognition are tightly coupled -the brain has to continuously adapt to changing events in the environment and adjust goal-directed behavior accordingly. For example, we have to react immediately to the honking of a car. By studying the brain in complex naturalistic environments, we can learn more about the dynamics of brain processes over longer time periods  and in response to environmental events that are difficult to recreate in the laboratory.
If we want to understand the brain in relation to everyday life events (we will focus here on sound events), we need to capture both the brain activity and information about the events concurrently with high temporal precision: simply put, to analyze the brains' response to the honking car at the correct moment, we need to know when the honk occurred relative to the EEG data. By relating naturally occurring sounds in everyday life to ongoing EEG activity, we can investigate questions about auditory perception and attention in real world scenarios. There is an increasing number of studies using naturalistic stimuli to study auditory perception (e.g., De Lucia, Tzovara, Bernasconi, Spierer, & Murray, 2012;Perrin et al., 2005;Roye, Jacobsen, & Schröger, 2013;Scheer, Bülthoff, & Chuang, 2018;Zuk, Teoh, & Lalor, 2020). However, to gain experimental control in this lab-based research, everyday life sounds and contexts are approximated by using artificial stimuli and conditions. With mobile EEG we can record the brain and the acoustic environment throughout the day in everyday live, and start to understand sound processing in real life contexts .
Ideally, brain recordings in everyday life should not interfere with natural behavior. The used recording setup should neither be noticeable for the participant nor for outside observers, only then we can expect a natural and therefore representative behavior of the participant. Hence, the optimal setup is maximally transparent, portable, and non-restrictive for a participant . Such a transparent EEG solution is smartphone-based, as smartphones have sufficient computational power for EEG recordings and processing, and they can be comfortably carried in a pocket (Blum et al., 2017;Blum, Jacobsen, Bleichner, & Debener, 2019;Debener, Emkes, De Vos, & Bleichner, 2015;Hölle et al., 2021;Piñeyro Salvidegoitia et al., 2019).
To investigate sound processing in daily life, we have developed two apps for Android smartphones (Record-a and AFEx) that allow us to record and process naturally occurring sounds and brain data. Record-a is a generic app that uses the LabStreamingLayer (LSL) framework for the simultaneous acquisition and synchronization of different data streams (Blum, Hölle, Bleicher, & Debener, 2021, submitted for publication). The AFEx app dissects raw audio into acoustic features (Power spectral density, PSD; average signal power, RMS; sound onsets) in a privacy-protecting way. These features are broadcast as LSL streams and can be recorded concurrently with EEG data by the Record-a app.
Before studying sound processing in everyday life in a hypothesis-driven manner with this system, each component has to be carefully tested and validated (Scheel, Tiokhin, Isager, & Lakens, 2020). Hence, in this paper, we evaluate this smartphone-based system regarding timing precision and plausibility of the EEG data. We show how we use this system to relate EEG and acoustic features to study ERPs. We use sound onsets to calculate ERPs, spectral information to distinguish tones, and information on loudness to contrast ERPs elicited by loud and soft sounds.

System Validation
The AFEx app needs time to buffer and process the raw audio, which creates a lag between the recorded sound onsets (onset marker) and the actual sound onsets in the real world. We conducted timing tests (see Methods) to quantify this lag and to check if it is stable over time (jitter).

EEG recordings
Apart from timing tests, we also recorded neural data to validate the system. For the recordings, we used a smartphone running three apps: AFEx computes and streams the acoustic features; Smarting (mBrainTrain, Belgrade, Serbia), a commercial app, streams EEG data; and Record-a concurrently records the former two data streams into one file. We recorded EEG from one person equipped with a mobile EEG cap. See Figure 1 for an illustration of the system components and how they play together. The participant listened to three different auditory inputs (a random sequence of two tones, a random sequence of multiple tones, and a home-office soundscape). In figure 2, we show the average ERPs to all detected audio onsets for the three conditions. These conditions are analyzed in more detail below.   Figure 2: Grand average event-related potentials (ERPs) in the EEG recorded in response to the random sequence of two tones (piano oddball, 402 onsets), the random sequences of multiple tones (piano freeplay, 201) and to home-office soundscape events (1357).

Piano Oddball
In this condition, the participant listened to two tones (c and g) that were played live by a pianist in the same room (total duration 6 min). One note was played frequently and the other infrequently. The participant had to count the infrequent tone (target). The participant correctly counted all targets. On average, the AFEx app detected a tone every 0.99 seconds (SD = 0.24 s, Range = [0.25 s 1.42 s]). Figure 3a shows the results of the piano oddball, split into target and standard tones based on the PSD data. The automatic identification of targets and standards corresponded exactly to the actual notes played as recorded by a MIDI file (ground truth). For both tones, auditory evoked potentials (P1,N1,P2) can be observed. Compared to the standard tone, the target tone also shows a P3 response.

Piano Freeplay
The participant passively listened to the free playing of individual tones by the pianist. The pianist varied the speed and the loudness at which he played the tones. He played 329 tones (6 min Figure 2 for the grand average of all tones).

Home-office Soundscape
The participant passively listened to a pre-recorded soundscape (27 min

Discussion
We presented a system incorporating two Android apps that allow to relate acoustic features of the soundscape to continuous EEG data. Our goal was to evaluate the timing precision and EEG data plausibility of this system. In the  Figure 3: Grand average ERPs per condition. a) Average ERP to piano oddball, divided into standard (frequent,n=317 blue) and target (infrequent, n=78, orange) tones based on the PSD information. b) Average ERP in the piano freeplay. RMS information was used to sort audio onsets from lowest to highest RMS. The ERP is shown in response to the 50 softest (left) and 50 loudest sounds (right). The histogram shows the distribution of the RMS values (n = 201) and the color coding indicates which value range was used for the plots. c) Average ERP in response to detected sounds in the home-office soundscape (left: 250 softest sounds; right: 250 loudest sounds). The histogram shows the distribution of the RMS values (n = 1357) and the color coding indicates which value range was used for the plots. Each line represents a channel. The grey shaded arrays correspond to the time windows of the topographies. The dotted black lines represent the amplitude threshold for a signal deviation that is significantly different from baseline. Data below this threshold was considered as noise. 6 timing tests, we showed that the temporal precision of the system is sufficient to compute ERPs. We identified a constant lag (due to signal buffering of AFEx), that can be easily accounted for by shifting the recorded onset marker, and a negligible jitter. For the EEG recordings, we presented sounds with increasing stimulus complexity from single piano tones to complex office sounds. In all experimental conditions, we found clear auditory evoked potentials in response to sound onsets. Furthermore, we showed how acoustic features in addition to sound onsets can be used to analyse EEG data. In the piano oddball condition, we used PSD to distinguish target from standard tones. The resulting ERPs show the expected P3 response for target tones only (Polich, 2007). Advancing from this simple validation task to everyday life settings, PSD information could be used to differentiate between different sound sources (e.g., Fahim, Samarasinghe, & Abhayapala, 2018). For example, in a two speaker scenario, PSD can be used to identify which speaker (low vs. high voice) is currently talking.
In the piano freeplay and home-office soundscape condition, we contrasted high-RMS sounds with low-RMS sounds. In line with literature (May & Tiitinen, 2010), we observed a significantly higher N1 response for high-RMS onsets compared to low-RMS ones in the home-office soundscape. We did not observe this difference in the piano freeplay, which may be due to the lower number of onsets and thus lower variance of RMS values. Note that ERP amplitudes in the home-office condition were lower than in the other conditions, which is most visible in the grand average of all sound onsets. This finding can be explained by the higher number of low-RMS onsets (see histogram) in this recording, resulting in lower amplitudes overall.
To calculate these ERPs, we relied on sound onsets. Importantly, there are many different methods of defining sound onsets (e.g.; Böck, Krebs, & Schedl, 2012;Thoshkahna & Ramakrishnan, 2008); for example, some methods are specifically geared for detecting onsets in music (Bello et al., 2005;Haumann et al., 2021). For the online detection of onsets, we optimized the parameters to detect clear, isolated sound onsets in the presence of background noise. Consequently, not all onsets that that humans perceive as onsets are picked up by the detector. For example, in the freeplay condition, 120 notes were missed by the detector. Thus, the onsets that were detected are a subset of all onsets a person perceives. Depending on the use case, the parameter settings can be adapted and optimized for different sound onsets.
Obviously, the acoustic features that are provided by AFEx only provide limited information about the soundscape. The app does not provide information about many aspects that are known to be relevant for sound perception; for example, whether a sound was self-generated or generated by another sound source (e.g., Sanmiguel, Todd, & Schröger, 2013), whether the sound was relevant or irrelevant (e.g., Dehais, Roy, & Scannella, 2019;Holtze, Jaeger, Debener, Adiloglu, & Mirkovic, 2021;Scheer et al., 2018), or expected or unexpected for a person (e.g., Dalton & Fraenkel, 2012;Koreimann, Gula, & Vitouch, 2014). How to assess these factors remains a challenge. Some of these factors could be assessed by asking the person about subjective experience of the soundscape on a regular basis using momentary ambulatory assessment (Trull & Ebner-Priemer, 2013), but this will interrupt the person in their normal activity. An alternative is to use additional microphones that provide more information about the origin of a sound, or sensors that provide context information. However, it is unlikely that we will be able to capture the full complexity of the participants environment in the near future.
To sum up, we demonstrated that the presented technical setup reliably combines EEG and acoustic features and thereby allows to measure event related potential in response to everyday sounds. This smartphone-based setup can be used fast and easily. It allows us to study brain activity everywhere: We could record EEG next to a construction site, while driving through the city, or in the classroom. These contexts are already of interest to researchers (e.g., Getzmann, Arnau, Karthaus, Reiser, & Wascher, 2018;Ke, Du, & Luo, 2021;Ko, Komarov, Hairston, Jung, & Lin, 2017). In these everyday contexts, we can also investigate individual differences in sound processing: Why do some individuals notice some sounds that other people miss? How does their brain response to sounds differ? Which sounds are distracting or even annoying to an office worker on the neural level? As evident from these various examples, the AFEx app in combination with Record-a and a commercial EEG app facilitate new research of auditory processing in everyday life, which was henceforth typically only assessed with surveys (e.g., Banbury & Berry, 2005;Oseland & Hodsman, 2018).

Software
The Smarting app provided EEG data as LSL stream, the AFEx app computed acoustic features (sound onsets, PSD, RMS) and provided them as LSL streams, and the Record-a app recorded these LSL streams into one xdf-file. In our analyses, we could use the audio onsets provided by AFEx to cut the continuous EEG data and average over these epochs to gain ERPs (see Figure 1).

Smarting
For streaming EEG data, we used a Smarting (mBrainTrain, Belgrade, Serbia) system. The ongoing EEG signal was provided as an LSL stream by the Smarting android app (Version 1.6.0). Note that any EEG system that provides the EEG data as LSL stream can be used.

AFEx
AFEx is an Android app that calculates acoustic features (https://doi.org/10 .5281/zenodo.5128051). It implements a framework that allows to concatenate various stages such as audio capture, signal filtering, and signal transformation to produce the desired metrics. The produced data is pushed to the LSL in chunks equivalent to the block sizes specified below. Fig. 4 shows the signal path.
The different metrics are calculated and parameterized as follows: Audio capture A stereo microphone signal is captured at a sampling rate of 16 kHz, buffered and transferred to subsequent stages in chunks of 250 ms. Each stage is rebuffering data to provide the blocksizes (incl. overlaps) requested by succeeding processing stages. Note that the properties of the signal are depending on the recording setup. A signal being recorded using an Android's device internal microphone or input via the headphone/mic jack will usually result in two identical, i.e. duplicated, mono channels. Stereo signals can be recorded using external USB devices.
Highpass filter The second order biquad filter has a cutoff frequency of 250 Hz to suppress low frequency noise. The signal is filtered in blocks of 25 ms.
Signal power (RMS) The RMS is calculated with a blocklength of 25 ms and an overlap of 50 %, resulting in a sampling rate of 80 Hz.

Spectra (PSD)
The power spectral density is calculated for blocks of 25 ms with an overlap of 50 %, resulting in a sampling rate of 8 Hz. The resulting spectra are exponentially smoothed and averaged, with one spectrum being produced every 125 ms. This method prevents reconstruction of the time domain signal to protect a person's privacy.
Acoustic onset detection To detect acoustic onsets, a variable state filter produces low-, band-, and highpass filtered signals (2 nd order, f g , f c = 800 Hz, Q = 1/ √ 2). Those are then smoothed with both, slow and fast time constants, tuned to different values per band (Table 2). If the faster signal emerges from the slower in any band, an onset is triggered.
All calculations are performed online on the smartphone and the results are written to LSL. The audio signal does not need to be stored or cached on the device's permanent storage. Additionally, the parameters have been chosen in such a way as to prevent reconstruction of semantic content or information, i.e., from PSDs, thus ensuring a participant's privacy (Bitzer, Kissner, & Holube, lowpass bandpass highpass slow 160 ms 20 ms 1 ms fast 4 ms 2 ms 2 ms 2016; Kissner, Holube, Bitzer, Technology, & Sciences, 2015). Note that modified parameters or new features would require a re-validation of said privacy. AFEx has been tested with Android OS up to version 10. Permissions to access storage and the microphone are explicitly required. Processing runs as foreground service, giving it some precedency over other apps. There are, however, no additional mechanisms in place to prevent interaction with the app, provide some sort of kiosk mode, or notify about system events like low-battery.

Record-a
The Record-a app allows to record LSL data streams in the network into one xdf-file. With the LSL framework, different data streams with different sampling rates can be recorded simultaneously. Each data stream has its own time information in reference to a master clock and upon import, they are time corrected and thereby synchronized. This app is described in detail in Blum et al. (2021, submitted for publication). The app is available at https://github.com/s4rify/Pocketable-Labs.

EEG System
We used a 24-channel cap (easycap GmbH, Germany) connected to a mobile 24channel Smarting mobi amplifier with 24 bit resolution (mBrainTrain, Belgrade, Serbia). We recorded the EEG data with a sampling rate of 250 Hz. The amplifier transmitted the data to the smartphone via bluetooth.

Smartphone and Microphone
We used a Google Pixel 3a smartphone (Android Version: 10). To record stereo sound with this phone, we used customized ear microphones comprising two omni-directional EK-23024 microphones (Knowles Electronics, Illinois, USA) placed into behind-the-ear shells. The ear-microphones were connected to the phone via the USB-C port with a sound adapter (Andrea Pure Audio USB-MA, Andrea Electronics, Bohemia, USA). Naturally, the microphones process sounds differently than human ears: they may pick up on sounds not perceptible to human ears, but they may also miss sounds that the ear would perceive. However, the microphone clearly detects the onset sounds that are essential to our analyses.

System Validation
To relate EEG data and audio onsets reliably, both data streams need to be stable over time and have sufficient temporal precision in the millisecond domain. As the audio signal is buffered and then processed, there is a lag between the actual audio onset and the recorded audio onsets (onset marker) as they occurred in the real world. In the system validation, we conducted timing tests to quantify this lag. Furthermore, we also checked whether this lag remains stable over time (jitter).
We conducted five short (∼45 min) and one long (∼180min) timing tests. The smartphone was restarted between timing tests. For these timing tests we used wav-files that were generated with Matlab (Version: 9.6.0.1335978; The Mathworks Inc., Natick, MA, USA). The wav-files contained square wave impulses that were presented at 1 sec intervals at a sampling rate of 44100 Hz. We used square waves as they produce a sharp response in the EEG amplifier. The square wave impulse had a duration of 50 ms. For the short timing test, the wav-file consisted of a 10 s silence at the beginning followed by 2500 impulses. The wav-file of the long-term timing test consisted of 4 times the wav-file for the short timing test for a total of 10000 impulses. These sound files were presented via PC on external speakers (Wavemaster 1520, Bremen, Germany) in a soundproof cabin and simultaneously fed into the amplifier via a customized adapter.
Hence, with this procedure, an impulse produced both a response in the data recorded with the EEG amplifier and an audio onset as detected by the AFEx app. We can then cut the sound signal recorded with the EEG amplifier based on the audio onsets and calculate lag and jitter. The AFEx app detected onsets for all impulses plus a few extra sounds (experimenter leaving the room, nearby construction work). In our analysis (also in Matlab), we deleted these additional onsets.
We then epoched the data from -1 s to 0.1 s (baseline correction -1 to -0.9) relative to the detected sound onsets. To quantify the delay between audio onset and onset marker, we identified the time point when the audio onset in the EEG reached its half maximum (same time point for all recordings) and then calculated the time difference between this maximum and the onset marker for every epoch.

Participant
We recorded one volunteer (male, age between 25 and 35, ambidextrous) who had previous experience with EEG recordings. He provided written informed consent prior to participation. We deemed neural data from one participant sufficient to demonstrate the functionality of the technical setup.

Procedure
For the EEG recordings we had three conditions that increase in stimulus complexity. In the first two conditions, the participant listened to piano notes played live by a pianist. In the third condition, the participant listened to an homeoffice soundscape. In figure 2, we show the average ERPs to all detected audio onsets for the three conditions.
Before the participant was fit with the EEG cap, his skin was cleaned with alcohol and electrolyte gel (Abralyt HiCl, easycap GmbH, Germany). All impedance was kept below 10 kOhm. The ear microphones were plugged into the smartphone and placed on the participant's ear. The smartphone was always placed on a table adjacent to the participant. For condition one and two, the participant sat in a chair approximately at a distance of 1 m to the piano (Yahama Digital Piano P-35, Hamamatsu, Japan; set to 75 percent volume), fixating his gaze on a fixation cross on a wall. The participants sat with his back to the piano; hence, he could neither see the pianist nor the experimenter. In the third condition, the participant sat in a soundproof cabin looking at a fixation cross on a computer screen while listening to a soundscape that was presented free-field (two Sirocco S30 loudspeakers, Cambridge Audio, London, United Kingdom) at a participant-comfortable volume. For all conditions, the participant was instructed to move as little as possible.

Piano Oddball
The participant listened to live played notes on the piano. The pianist either played a c (159 in total, standards) or a g (39 in total, targets) on the keyboard. The sequence of tones was predefined and provided on a paper sheet. We recorded this oddball twice with a different predefined sequence in the second run. The participants task was to count the target tones. To check whether the pianist played exactly the notes from the sheet, we also recorded the MIDI file of the oddball. Each oddball took 3 minutes. For later analysis, we merged both oddball datasets.

Piano Freeplay
The participant listened to the pianist playing a monophonic sequence of random tones with variable loudness and tempo for 6 minutes. The exact sequence of sounds was recorded as a MIDI file.

Home-office Soundscape
In this condition, the participant listened to a soundscape of a home-office environment for 27 minutes. This soundscape (wav-file) included typical home-office activities such as operating a computer (clicking with the mouse, typing on the keyboard, notification sounds), walking around, using a hole puncher, or unloading the dishwasher. It did not contain music or speech. Due to technical interference in this recording, we recorded the sound onsets again in an empty room and mapped them onto the EEG recording.

Preprocessing
The EEG data were re-referenced to linked mastoids, low-pass filtered at 25 Hz (Order 134) and high-pass filtered at 0.1 Hz (Order 8250). To account for the previously determined time lag, the recorded markers for sound onsets were shifted by -248ms (-62 samples; see system validation results). We deleted some markers at the beginning and the end of each recording (14 in total) as these onsets were due to the starting and stopping of the recording. We used artifact subspace reconstruction (ASR) to clean the data. For ASR, we used the EEGLAB-plugin clean rawdata (Version: 1.0; parameters: flatline criterion = 60, high-pass = [0.25 0.75], channel criterion = off, line noise criterion = off, burst criterion = 20, window criterion = off). We epoched the data from -0.1 s to 0.8 s relative to each sound onset (baseline correction -0.1 s to 0 s).
Piano Oddball Based on the recorded PSD data, we identified target and standard tones. First, we visually identified the first target tone in the spectrum. Second, we created a template based on this target tone. We then correlated this template with every tone in the spectrum. This procedure allowed us to automatically identify target tones when the correlation was higher than 0.9.
Piano Freeplay and Home-office Soundscape In both conditions, for each audio onset, we computed an average RMS based on the mean of five samples (62.5 ms) after the audio onset of both audio channels to gain a corresponding RMS value for each sound onset.

Random events
To obtain an estimation for the reliability of the ERPs, we calculated an amplitude threshold based on ERPs extracted from random events for each condition. Data within this threshold can be considered as noise. For 1000 permutations, we inserted 500 markers at random time points in the data, epoched with these markers, and computed the ERP over all channels. We then determined the maximum and minimum amplitude of the mean over all permutations plus two times the standard deviation. This value corresponds to approximately 95% of the amplitude distribution.

Open Practices Statement
The EEG datasets are available from the corresponding author on request. The code used for the analyses, the stimuli used, and the datasets from the timing test are available at https://osf.io/bcfm3/. The code for the AFEx can be found at https://doi.org/10.5281/zenodo.5128051. The Record-a app can be found at https://github.com/s4rify/Pocketable-Labs.

Author Contribution
DH and MB conceived the experiments with the help of SB and SD. DH carried out the data collection and analyses. DH wrote the manuscript with the input of MB, SD, SB and SK. SB lead the development of the Record-a app. MB designed the AFEx functionality and SK developed the AFEx app.


extfieldsofstudy:
Computer Science
Engineering

text:
Combined Sampling and Optimization Based Planning for Legged-Wheeled Robots

Planning for legged-wheeled machines is typically done using trajectory optimization because of many degrees of freedom, thus rendering legged-wheeled planners prone to falling prey to bad local minima. We present a combined sampling and optimization-based planning approach that can cope with challenging terrain. The sampling-based stage computes whole-body configurations and contact schedule, which speeds up the optimization convergence. The optimization-based stage ensures that all the system constraints, such as non-holonomic rolling constraints, are satisfied. The evaluations show the importance of good initial guesses for optimization. Furthermore, they suggest that terrain/collision (avoidance) constraints are more challenging than the robot model's constraints. Lastly, we extend the optimization to handle general terrain representations in the form of elevation maps.

I. INTRODUCTION
Equipping robots with legs enables mobility over unstructured terrains, the same as the animals can traverse them. On the other hand, wheeled robots are not as mobile as their legged counterparts, but they are far more energyefficient. By combining legs and wheels into a hybrid system, roboticists have tried to keep the best characteristics of legged and wheeled systems [1], [2], [3]. Unfortunately, increased flexibility comes with an increase in complexity, especially true for motion planning since the combinatorial aspect of legged robots (contact schedule) is combined with wheeled robots' non-holonomic nature (rolling constraints). A powerful tool used to cope with the increased complexity is Trajectory Optimization (TO). TO has been shown to work well on hybrid systems with many Degrees of Freedoms (DoFs) ( [1], [4], [5]). However, optimization is prone to local minima. While some control over solution quality can be achieved through cost function shaping (e.g., [6], [7]), the optimization can ultimately fail due to the non-convexity of the planning problem. Failures happen especially often in challenging terrain where collision (avoidance) constraints become particularly difficult for optimizers. In this letter, we overcome this problem by providing good initializations (initialization step) for the optimization-based planner (refinement step); i.e we combine Sampling Based Planning (SBP) with TO. Our initialization step computes base poses, joint positions, and contact schedule, which are all then fed to the optimization. The gain is that our method can handle systems with many DoF while still being able to cope with challenging terrain.

A. Related Work
Early research on locomotion for hybrid systems treats the whole system as a driving robot and uses legs as an active suspension. Such a practice is widespread in the aerospace community [8], [9] and has more recently been applied to wheeled quadrupeds [10]. The main disadvantage of such an approach is that it does not fully leverage the hybrid system's legged nature, i.e., the robot cannot negotiate obstacles.
To simplify the combinatorial nature of contact schedule planning for legged robots, researchers often use cyclic gaits to restrict the solution space size. Planning with cyclic gaits has been widely used for quadrupedal robots with point feet ( [11], [12], [13], [14]) and more recently, it has been applied to hybrid robots as well ( [1], [15], [5]). Presented controllers run under the flat ground assumption in an Model Predictive Control (MPC) fashion with a prediction horizon of about 2 s. Such a strategy relies on reactive control behavior from MPC, and while it can traverse small irregularities in the terrain, large obstacles still pose an issue. Furthermore, these approaches are not suitable for temporally global plans because of the relatively short prediction horizon.
More recently, terrain-aware planning has been proposed for hybrid robots ( [7], [2]), which demonstrate the ability to traverse challenging terrain and plan motions in a wholebody fashion. However, the presented methods solely rely on trajectory optimization and often fall prey to the local minima. Furthermore, the environment is known excatly, and it remains unclear how the planner would handle maps generated from real sensory data since discretization and noise can lead to discontinuous gradients in the optimization.
Unlike the optimization, SBP cope well with the nonconvex environment. Attempts to use SBPs can be found in [16], [17], where the proposed approach samples base poses in SE(3) space and computes a guiding path for the base of the robot. The footholds are computed in the next stage using the guiding path. However, the proposed approach does not use any optimization, which would make the planning for robots with non-holonomic constraints difficult. Unlike [16], our approach does not accept a guiding path before having computed the footholds.
Recently, [18] has introduced a Contact Dynamic Roadmap (CDRM) data structure for rapid collision checking and foothold generation at runtime. The crux of the approach is computing a Probabilistic Roadmap (PRM) of collisionfree configurations offline and then using it for planning online. A similar idea is employed for self-collision avoidance during the initialization phase in our work. We extend the PRM with additional data, so that the approach is applicable to robots with heavy limbs such as walking excavators.
Finally, Klamt et al. [3], [19] use a graph-search based approach to plan hybrid motions for the Momaro robot. The robot decides when to drive and when to step based on a carefully crafted cost function. The transition sequences between driving and stepping are not computed in a wholebody fashion (as they are handcrafted). Lastly, the graphsearch algorithm choice is motivated by the fact that the robot can turn in place using its wheels only. Hence, the approach is unsuitable for robots with a minimal turning radius greater than zero in its current form.

B. Contribution
We present a combined sampling and optimization based planner for legged-wheeled machines with many DoFs. We use terrain representation to generate a wide variety of locomotion behaviors for navigating complex terrain. The planner is divided into two stages: Initialization Step based on a sampling-based planner and Refinement Step through nonlinear optimization. These two stages produce kinematically feasible and statically stable plans, motivated by our use case on a walking excavator [20]. Given the base's initial and goal pose, our formulation computes base trajectory in SE(3), joint trajectories, and contact schedule. To the best of the author's knowledge, existing approaches provide only high-level waypoints (e.g., [21], [22], [19]). Finally, our approach is the first (to the best of our knowledge) to

II. PROBLEM STATEMENT
A legged-wheeled robot comprises N limbs with wheels and M limbs without wheels, e.g., a walking excavator has four limbs with wheels, and one non wheeled limb (see [20]). The base of the robot can move in SE(3) space. Robot's knowledge about the environment is contained in a map, which is a mapping h : R 2 → R that maps coordinates to various functions describing the environment (e.g., height, traversability). In this letter, a multilayered grid map [23] data structure is used. Robot's i th limb is in contact with the environment if it is close enough to the surface, i.e.
where p z denotes z component of the contact position p and h(p x , p y ) is height at the contact position. We assume that the contact point is below the center of the wheel (negative gravity direction). For a 30°slope (which is the limit beyond which we consider terrain untraversable), the approximation error is about 15 % of the wheel radius, which is well within the adaptation capabilities of the tracking controller, as demonstrated in [24]. The environment is divided into traversable part denoted with T and untraversable part denoted with ¬T . A contact is valid if the contact point lies in the traversable set T , i.e.
where sdf (·, ·) represents a 2D Signed Distance Function (SDF) with the positive distance meaning the point lies in T . We require all the contact points to stay at least δ away from the ¬T . The 2D SDF is stored as a grid map layer and calculated using marching parabolas [25]. Note that, unlike the limbs, robot's base is not required to stay in T since the base is not in contact with the terrain. Our goal is to find a trajectory τ with a length of T seconds such that p B (t = 0) = p B,start and p B (t = T ) = p B,goal where p B,start , p B,goal are given starting and goal position for the base of the robot. Apart from the base, we do not enforce any other constraints on robot's pose or joint angles although this is not a hard requirement of our approach.

III. INITIALIZATION STEP
The backbone of the initialization step is a sampling-based planner that samples base poses. We use Rapidly-Exploring Random Trees (RRTs) [26], [27] although the problem formulation also permits the use of multi-query planners such as PRMs [28]. Results presented are mostly generated using and RRT# [29], whose implementation is taken from [30]. In addition to RRT, a PRM of limb end-effector positions is precomputed for efficient online planning.

A. Offline Computation
We use the CDRM data structure introduced in [18]. CDRM can be used at runtime to generate collision-free movements of the robot's limbs. In our case, CDRM helps us avoid collision between arm and legs while using the arm as a supporting limb. For each limb, the mapping between joint angles q i and end-effector position B p i is stored. This mapping does not change over time since B p i is expressed in the base frame of the robot. Furthermore, we store the mapping between i th limb's Center of Mass (CoM) p i,com and joint configuration for the same limb. This allows us to evaluate the stability criterion during the online planning phase rapidly. The offline roadmap is created using the PRM algorithm; it is shown in Fig. 3. The roadmap does not change over time which motivates the use of PRM. In addition to the roadmap, we compute terrain normals and filtered height. This computation can be performed by fitting a tangent plane at each point (x, y) locally. Local fitting is done using least squares, and as a result, we get the normal of the plane and fitted height at point (x, y). We filter once with a local radius R of 0.3 m and once with 2.5 m, which is roughly the robot's footprint radius. Filtering result with large R is shown in Fig. 2. The idea behind this is to use the filtered height for computing base poses. Terrain discontinuities (e.g., steps) should not be reflected in the base movement since the base moves above the terrain. On the other hand, smoothed terrain (shown in yellow) is a good approximation for base movement, assuming that it stays at some (roughly) constant height above the smoothed terrain.

B. Online Planning
Having computed the roadmap shown in Fig. 3, an RRT planner finds a plan between base poses p B,start and p B,goal . We use RRT framework to enable re-planning in potentially Algorithm 1 Select n and h at position (x, y) if cost < c best then 10: h best = hc, n best = nc, c best = cost 11: return (h best , n best ) changing maps. Similar to previous work [16], our planner proposes a base pose before computing the limb contacts. Subsequently, we check whether contacts can be established and whether the robot is stable. In general, a sampling-based planner typically has three main components: sampling, connecting a new sample to the tree, and feasibility checking. In this section, we describe how each step works.
1) Sampling: Candidate base poses are sampled in SE(2) space instead of full-fledged SE (3). The idea behind this decision is straightforward: since limbs interact with the environment, the sampler uses terrain information to constrain some DoFs of the base pose, which reduces the dimension of the search space. Hence our planner samples x, y position of the base, and yaw angle γ from a uniform distribution. The remaining DoFs are computed based on local terrain features: roll angle α, pitch angle β, and z coordinate. Roll and pitch are computed from terrain normal n such that the base remains roughly parallel to the terrain underneath. Finally the z coordinate can be computed as z = h(x, y) + h desired where h(x, y) is the terrain elevation at sampled point (x, y) and h desired is user defined desired height above the terrain.
Selecting h(x, y) and n can be tricky. E.g., when crossing a deep gap, terrain height can be so low that the planner cannot generate any valid poses (despite the heavy filtering). Luckily, one can leverage a simple observation to chose good h and n. When moving over untraversable terrain ¬T , the robot only cares about the nearest T (traversable area). The rationale is that contacts should only be made with T , and the base pose should be selected such that limbs can reach the nearest T . Alg. 1 implements this proposition; it selects h and n such that contacts with T can be established. The Alg. 1 enumerates all candidate normals and heights (lines 1-6) and then does a small brute force search to select a pair (h best , n best ) that minimizes some criterion. The computePoseCost function in line 9, gives low cost to poses where all legs are grounded and penalizes big roll and pitch angles. Full base pose is then determined from (h best , n best ).
2) Connection to Tree: Upon drawing a random base pose (x, y, γ), the planner tries to connect it to the tree (using the weighted cost of euclidean distance and angular distance). The connection is done in SE(2) space using Reeds-Shepp (RS) curves [31]. RS curves give an optimal path between two poses while respecting the minimum turning radius con- Fig. 4. Addition of a new node into the RRT tree. Red circles and full black lines are nodes and paths that make the current RRT tree. The newly sampled node (green) has to be connected to the rest of the tree. For a successful connection, all subnodes on the connecting path (small green circles) have to be valid. selectContactArmConfiguration(T ) 14: selectSwingLegsCofiguration(T ) 15: return isStable() 16: case 1, 0 17: return FALSE straint. For a robot that can turn in place, one could use a very small turning radius. Attempting straight-line connections between base poses would make the subsequent refinement step (see Sec. IV) very hard since the satisfaction of the nonholonomic rolling constraint cannot be guaranteed. By using RS curves, this is implicitly ensured; however, the resulting trajectory might be longer. Computing the RS connection is done terrain agnostic completely.
3) Feasibility Checking: Fig. 4 depicts the feasibility checking. Upon drawing a new sample (large green node) as described in step 1, we compute a RS connection (dotted line) to the new node, as described in step 2. Subsequently, the dotted line is discretized into subnodes (small green circles) using an RS interpolation method [30]. Next, for each subnode we generate full 6 DoF pose using Alg. 1. Finally, each subnode undergoes feasibility checking, ensuring that the robot is statically stable and can establish enough contacts with the ground. In case a feasibility check passes for every subnode, the RRT adds the new state and the connecting path to the tree. It is crucial to discretize the RS path with high resolution since straight-line connections are assumed between two subsequent subnodes; in our implementation, we allow for a maximal distance of 20 cm. The length of the whole path is a tuning parameter (in our case, 15 m). Alg. 2 summarizes feasibility checking.
The feasibility check shown in Alg. 2 does not allow poses with a large roll or pitch angle (line 2). We then ground all the legs (line 5). A leg is grounded if a sufficient number of configurations in PRM are in contact with the surface and the contact location lies in T (see Sec. II). Among grounded legs configurations the algorithm picks the one closest to the default configuration. In case of four contact legs, the pose is deemed to be stable. In case three legs are in contact, the algorithm selects good joint configuration for the swing leg according to some criterion (e.g. ground clearance or proximity to the default configuration). The swing arm configuration is selected such that the CoM is as centralized as possible. In case only two legs are in contact, the algorithm checks whether the arm can be grounded (line 14) and then proceeds with selecting swing leg configurations. The isStable() function computes the CoM of the whole robot and verifies that it lies in the support polygon. Aside from performing feasibility checking, Alg. 2 computes the full joint state of the robot q for feasible poses. The CoM of the full joint configuration q can be computed using: where M is the mass of the whole robot, p i,com and m i are the position of CoM and mass of the i th limb, respectively. Thanks to the mapping between q i and p i,com computed offline, the sum in Eq. 3 can be evaluated rapidly. Note that unlike [16], we do not require all limbs to contact the environment while generating base poses, thus allowing for more flexibility. Once the RRT has reached p B,goal , the final path is post-processed. In the first step, the contact schedule is modified to ensure stability. We do not allow establishing/breaking more than one contact between two different successive nodes. If the robot wants to change more than one contact state at any point, we insert a node in between. Those situations happen only when the arm contact is established/broken. The robot tries to change the contact state of the arm and leg(s) simultaneously. We add a short full contact phase (legs + the arm) in between to ensure static stability. Secondly, we compute Inverse Kinematics (IK) for the non-wheeled limbs in contact. Any of those limbs has to satisfy the contact constraintṗ i = 0. For each non-wheeled limb in contact, we find base poses at the beginning and the end of its respective contact phase. The reference position p * for the IK is found by solving: and setting p * = (p * s +p * e )/2.0, where p s are all positions in the i th limb roadmap at the beginning of the contact phase and p e at the end of the contact phase. We then compute i th limb's joint angles for every contact node as q i = IK(p * ).
Alg. 2 ensures that the robot is stable and that limbs are not in a collision. Nevertheless, it assumes that straight line connections in joint space are collision-free. The assumption might be invalid, especially for the arm, which moves around the base and is used as a counterweight. To overcome this problem, we use the precomputed roadmap in which we, similar to [18], invalidate all vertices and edges that are in a collision with other limbs or the environment. Once the graph is updated, each limb's path is found using a graph search algorithm (A* in our case). In practice, legs always end up being collision-free, but the arm often collides with legs. Hence we run the graph search only between nodes where the arm moves.
IV. REFINEMENT STEP The refinement step uses TO to satisfy all system constraints. TO methods scale well with system dimension and can handle nonlinear constraints such as forward kinematics or non-holonomic rolling constraints. However, computing the correct contact schedule and dealing with obstacles remains challenging for the gradient-based methods, so we initialize optimization with trajectory computed in the initialization step. The optimization receives contact schedule, base position/velocity (6 DoF) and joint position/velocity (25 DoF in our case) and solves a feasibility problem. Adding an optimization objective allows for fine motion tuning, however it typically results in increased computation times. TO planner used in this paper is based upon our previous work [4], and below we present modifications that enable us to cope with more challenging scenarios.
Terrain maps To the best of the author's knowledge, there is no optimization-based planner for hybrid systems that can handle general terrain representations. Compared to their legged counterparts, hybrid robots keep their limbs in contact over long distances. Hence, map errors influence more variables and constraints, which makes the optimization more sensitive. So far, proposed terrain-aware optimization planners have used analytical descriptions of the environment [7], [32], [2]. Our planner integrates grid maps [23] into the optimization, thus allowing planning in any environment where a 2.5D map is a suitable representation. The elevation map comes into the optimization in the form of height constraint for all limbs in contact C.
The constraint (5) is problematic since height mapping h(·, ·) is discrete and discontinuous (e.g. gaps or steps) which can cause optimization to diverge. Far away from the cell center, Nearest Neighbor (NN) search or linear interpolation are poor approximations of the true elevation. Hence, approximation of partial derivatives [ ∂h ∂x ∂h ∂y ] using central finite differences renders them non smooth. For large grid cells (0.1 m), the key to improving the optimization convergence is using a higher-order approximation of the h(·, ·) function. We found that bicubic interpolation and bicubic convolution algorithms [33] work well. Implementations of both algorithms are integrated into the open-source package Grid Map and made available for the community 1 . Unfortunately, neither filtering nor higher-order approximation can help if the terrain is discontinuous (steps or gaps). To handle discontinuities, we use gradient clipping, a technique known from the machine learning community. Smaller clipping thresholds prevent getting stuck in bad minima. However, they usually require a few more iterations for convergence.
Traversability Constraint We require that all contacts stay in the traversable area; the constraint implements Eq. 2. The gradient is also computed using a central finite difference with bicubic interpolation.
Collision Avoidance Constraint imposes a minimum distance between the collision geometries of the robot. We use shape primitives such as spheres or cylinders. We impose that the minimal distance between collision geometries has to be greater than d min . The minimal distance between collision geometries is calculated using the Bullet physics engine.

V. RESULTS
We tested our planner on HEAP [20], which is a customized Menzi Muck M545 walking excavator; five limbs, 25 joints, and a floating base make it a challenging test bench. The whole planning pipeline is implemented in C++ programming language, and tests are performed on the Intel Xeon E3-1535M processor with 32 GB of RAM.
1) Roadmap Generation: We use a roadmap size of 300 vertices per leg and about 3000 vertices for the arm since it has a much larger workspace (see Fig. 3). More vertices yield better workspace approximation; consequently, finding stable configurations is more likely. However, with more vertices, more computation is required to find stable configurations. We found the proposed number of vertices to be enough for finding solutions for the scenarios tested. For each vertex in the roadmap, we attempt the connection to its ten nearest (a) Prefer Shortest Path Looking for connections that are shorter than d max is the most computationally intensive operation.
2) Terrains: The proposed planning pipeline can compute plans in various terrains (see Fig. 1, also see the video attached 2 ). The planner uses the same set of parameters for all scenarios. Traversing such challenging terrains would not be possible with a purely optimization-based planner and handcrafting a contact schedule for those scenarios would be difficult. We show success rates on complex terrain features in Fig. 5b, by averaging five trials for each difficulty. All terrains except flat ground are shown in Fig. 1: rough (middle left image), gap (bottom right), step (middle right), hole (top right). For terrains flat, rough, gap and step, the planner was asked to find a path of about 15 m in length. For terrain hole, the length was about 30 m such that the planner has to navigate around the hole to get to the other side (see Fig. 1, top right). The maximal distance parameter in the RRT was set to 10 m. We consider planning successful if both initialization and refinement step find a solution. Times and costs in Fig. 5a have been obtained by averaging ten successful trials across all terrain difficulties. All the plans are found using the same RRT optimizing planner with planning time of 4 s. Fig. 5a shows that the first stage finds initial solutions quickly and that they are close to the optimal ones. Short planning times suggest that the initialization step could be used in a receding horizon fashion.
3) Importance of Initialization: To quantify effect of good initialization on the convergence, we use linear interpolation (between start and goal pose) as a baseline strategy. The other strategy is using a whole-body plan from the first stage to initialize all the variables. Linear interpolation uses the contact schedule from the first stage since it cannot compute one alone. The computation times until convergence when initialized with linear interpolation are shown in Fig. 5c (red color). The times were obtained by averaging ten successful trials with different initializations over all terrains. The computation times variance is caused by different durations of initialization trajectories from the first stage (longer durations 2 https://youtu.be/B-NHY4xwgwY require more computation). A good initialization makes a small difference in continuous terrain (flat terrain and rough terrain with roughness ±0.5). However, it becomes essential for harder terrains (hole, gap, step) since linear interpolation fails to produce a solution. This result corroborates our hypothesis from the Sec. I that TO can easily handle nonholonomic and nonlinear constraints from the robot model, whereas terrain constraints and contact schedule discovery present challenges for the optimization. Unlike agile quadrupeds in [32], HEAP cannot execute full flight phases, which makes the terrain constraints especially challenging.

4) Contact Schedule Discovery:
The way of terrain traversing can be influenced by tuning the cost function inside SBP. This is illustrated in Fig. 6. HEAP is commanded to reach the other side of the gap. Trajectory of the base is shown in green color. In the first scenario (Fig. 6a), HEAP incurs no cost for lifting the legs off. Upon introducing the stepping penalty, HEAP realizes that it can use a small bridge to avoid breaking contact on all legs, Fig. 6c. This behavior emerges merely by introducing the stepping penalty and without any other modifications. Such flexibility is made possible by optimizing over contact schedule and full-body poses simultaneously. The resulting contact schedules are shown in Fig. 6b and Fig. 6d.

VI. CONCLUSION AND OUTLOOK
We present a combined sampling and TO based planner for legged-wheeled robots. The sampling-based stage computes whole body configurations and contact schedule; it is based on RRT planner and a roadmap that is pre-computed offline. Compared to existing work, the roadmap is extended to store the mapping between joint angles and CoM, which allows for quick stability checks in the presence of heavy limbs. Our SBP planner achieves fast planning times and could be used interactively. In the second planning stage, TO satisfies all system constraints, such as non-holonomic rolling constraint. We integrate elevation maps into TO and demonstrate planning on general map representations. Evaluations of the proposed approach suggest that the main difficulty for TO stems from terrain/collision (avoidance) constraints and contact schedule planning, problems that are mitigated using the proposed two-stage approach.
Future developments will verify the approach on real machines. Besides, we plan to investigate using TO for tracking plans from the first stage in an MPC fashion.


A Deep Learning Approach for Robust Detection of Bots in Twitter Using Transformers

During the last decades, the volume of multimedia content posted in social networks has grown exponentially and such information is immediately propagated and consumed by a significant number of users. In this scenario, the disruption of fake news providers and bot accounts for spreading propaganda information as well as sensitive content throughout the network has fostered applied research to automatically measure the reliability of social networks accounts via Artificial Intelligence (AI). In this paper, we present a multilingual approach for addressing the bot identification task in Twitter via Deep learning (DL) approaches to support end-users when checking the credibility of a certain Twitter account. To do so, several experiments were conducted using state-of-the-art Multilingual Language Models to generate an encoding of the text-based features of the user account that are later on concatenated with the rest of the metadata to build a potential input vector on top of a Dense Network denoted as Bot-DenseNet. Consequently, this paper assesses the language constraint from previous studies where the encoding of the user account only considered either the metadata information or the metadata information together with some basic semantic text features. Moreover, the Bot-DenseNet produces a low-dimensional representation of the user account which can be used for any application within the Information Retrieval (IR) framework.

I. INTRODUCTION & MOTIVATION
In recent years, social media platforms such as Twitter or Facebook have gained a large level of both popularity and influence among millions of users due to the benefits of publishing, propagating and exchanging large volumes of multimedia content along the network. Therefore, these platforms allow users to establish a digital community as remarked in [22], which has made possible not only to discover and embrace new relationships but to maintain and boost existing ones.
On the other hand, due to both the great influence these platforms have on the lifestyle of people and its evolving The associate editor coordinating the review of this manuscript and approving it for publication was Weiping Ding . as a potential communication tool, they have exponentially promoted its attraction for marketing and commercial purposes by analysing the behaviour and opinion of users in different topics or events such as political elections. Consequently, numerous research studies have been fostered in the social media field with different purposes including sentiment analysis [35], traffic control [48], or consumer behaviour mining [4].
However, the considerable growth of social media platforms has also provoked the desire of altering people's opinion in certain topics by spreading propaganda or bias information. Many of these controlling procedures are carried out by Bots which are widely described in numerous investigations [31], [32], [40] such as automatic systems which are capable of generating and spreading multimedia content VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ throughout the network without the supervision of a human being. Furthermore, with the disruptive growth of Artificial Intelligence (AI) algorithms, the identification of bots or non-reliable sources has become a crucial challenge to be investigated. It raised many studies and publications with the goal of building robust automatic systems to improve the quality of experience of consumers in such platforms by reducing their privacy risks as well as increasing the trustworthiness on the platform itself at the same time.
Therefore, this paper aims to contribute to the state-of-theart in this field by proposing a novel method for automatically (i) encoding an input user account as a low-dimensional feature vector independently of its language, (ii) identifying the input encoding vector as a suspicious bot account with a certain probability throughout a Deep Neural Network (DNN) referred as Bot-DenseNet. (iii) generating a low-dimensional embedding which represents the original input encoding vector of the user account and which can be used for any other purpose related to Information Retrieval (IR).
More specifically, our study is focused on identifying Bot accounts in Twitter by considering three main aspects of the account: its activity level, popularity and profile information. The global set of features can be separated into two modalities including metadata and text-based descriptors, where the latter set is encoded via novel Language Model Embeddings (LME) to mitigate the language constraint suffered in previous studies.
The remainder of this paper is organized as follows: in Section II, previous studies and investigations within the Bot identification framework are described. Section III outlines the main components involved in the proposed multilingual approach including Section III-A to describe the generation of the input encoding vector and Section III-B which summarizes the proposed architecture of the proposed Bot-DenseNet model. Afterwards, Section IV presents the distinct experiments conducted as well as the outcomes and breakthroughs reached. Finally, the general conclusions and future work are summarized in Section V.

Recently, AI techniques including Deep Learning (DL) and
Machine Learning (ML) methods have gained popularity and interest in many applied research and industry services related to social media analysis where, sentiment analysis and text classification have been the central focus of these investigations specially for searching engines or recommender systems.
More specifically, the essence of sentiment analysis consists in extracting an aspect term of an input sentence to determine its polarity as positive, neutral and negative as authors remark in [8] and it is generally solved as a multi-class classification problem. Moreover, sentiment analysis has widely been used in numerous studies for both reviews and user opinions analysis in online commercial platforms [16], [47] and user behaviour mining in social media platforms such as Twitter [6], [29], [37], [42].
Furthermore, the continuous growth of the social media platforms such as Twitter or Facebook in the last decade along with the considerable propagation of non-trusted information throughout them, have raised applied research to automatically identify these non-trusted sources which in many cases correspond to non-human or Bot accounts. One of the first studies in this field was proposed by [9] and it was based on a random forest approach to classify bots and non-bots accounts using a manually annotated dataset with around 2000 samples. In 2016, BotOrNot was proposed in [12] as a service to automatically detect bots in Twitter using similarities between characteristics of social bots. This model has inspired posterior investigations in this field that even employed this service to automatically annotate data from Twitter.
In [11], authors annotated more than 8000 accounts and proposed a classifier which achieved a considerable level of accuracy for such set of samples. Additionally, [38] presented a model for Twitter bot detection based on a large number of metadata from the account to perform the classification.
More recently, several scientific studies have incorporated more annotated samples to support this research such as [27], [44], [45] including some procedures for achieving better level of accuracy by strategically selecting a subset of training samples that better generalize the problem. In [22], a language-agnostic approach is employed to identify potential features to distinguish between human and bot accounts. The model is then trained and validated using over 8000 samples distributed in an unbalanced fashion and its performance reaches an accuracy of 98%.
Moreover, authors in [32] proposed a 2D Convolutional Network model based on user-generated contents for detecting bots from human accounts including its gender (male, female account) covering both Spanish and English languages. A similar goal is explored by authors in [40], where both Word and Character N-Grams are employed as main features to perform the classification.
A different manner of addressing the problem was recently proposed by authors in [5], where novel altmetrics data to investigate social networks are analysed and they are used to train a Graph Convolutional Network (GCN) which reaches over 70% of accuracy in this task. On the other hand, authors in [34] presented a novel one-class classifier to enhance Twitter bot detection without any requiring previous information about them.
Most of the aforementioned approaches were limited due to lack of large volumes of annotated data for this specific task by the time their experiments were conducted. This problem is also remarked in [45] and thus, this paper has considered all the available public datasets in the current days in order to build the system with the most updated, newest and relevant state-of-the-art annotated data from Twitter. Additionally, although many approaches employ both metadata 54592 VOLUME 9, 2021 and text-based features from the user accounts, the text-based features are either extracted at a lexical level or they only cover a limited number of languages such as Spanish or English.
Unlike previous studies, our proposed model encodes all the text-based features of an input user account via novel multilingual Language Models (LM) including transformer models such as the so-called BERT [14] or Contextual string embeddings proposed in [2]. Thus, by concatenating both the metadata set of features along with the output vector provided by these LMs, an input vector of the user account is obtained. Finally, this paper proposes a Dense-based DL model to produce both the final decision of the account and a low-dimensional embedding of the user based on the aforementioned input vector.

III. A MULTILINGUAL APPROACH FOR USER ACCOUNT ENCODING VIA TRANSFORMERS
As it was introduced in previous sections, our system consists in a multilingual approach capable of better identify suspicious Twitter accounts based on a set of features independently of the language of the account. More specifically, the methodology of building the whole system can be distributed in two separate processes: (i) a preprocessing stage where a multilingual input vector of the user account is generated and (ii) a final decision system for identifying whether the account has a normal or abnormal behaviour according to existence patterns in the input vector generated during the first stage.
Moreover, the former process is responsible for retrieving a large collection of annotated Twitter accounts in a binary fashion, where the positive class indicates that the account is Bot whereas the negative class means that it belongs to the human account category. Subsequently, several features were collected from each Twitter account to enhance some relevant aspects including: (i) Level of activity, (ii) Level of popularity, (iii) Profile information.
Finally, this first stage ends by combining all the features to generate an input vector with both textual and metadata information for each Twitter account. Section III-A describes the full process to provide all the details of the implementation. The latter process, described in Section III-B, is in charged of automatically identifying patterns in the input encoding vector to properly distinguish between bots and human Twitter accounts via Deep Neural Networks (DNNs). Moreover, this process automatically obtains a low-dimensional feature representation of the input vector which can be used for any IR purpose in a more efficient way due to its low-dimensional nature.

A. MULTILINGUAL USER ENCODING VIA TRANSFORMERS
As introduced in Section III, a first process is needed to combine distinct relevant aspects from Twitter accounts to build a solid multilingual encoding representation which can be used as potential inputs for classification purposes throughout DNNs.
In Figure 1, an illustrative block diagram of this process is presented to show the different tools and stages needed to achieve the objectives of this first phase of the system.
More specifically, a first stage to retrieve all the data from the set of users U is performed via Twitter API. 1 Subsequently, each account is represented as a vector considering two modalities: text-based and metadata features. The former set is passed through a multilingual pre-trained LM model to obtain a feature vector representation of the text information including the description, the username of the account as well as the language of the account. The latter set of features is directly passed forward to the final stage which is a concatenation of both modalities into a single feature vector x, which encodes the information of an input user account.

1) DATASET GENERATION
There are several public datasets to address the bot identification problem from a binary classification perspective as the ones presented in Table 1. Moreover, some of these datasets were already used to train and evaluate the so-called Botometer (formerly BotOrNot) service proposed by [12].
However, as authors described in [7], [28], the generation of bot accounts continuously changes over time and additionally, some of the provided accounts have been already suspended by Twitter. Thus, a preprocessing is needed to improve the usability of this large collection of datasets by removing the identifiers those accounts that were already removed by Twitter. This aspect is also critical since several previous Bot detectors have not been updated with the new tendencies and features that bots accounts may currently have, so that, they are no longer as reliable as they used to be.
On the other hand, due to policy restriction terms from Twitter, the datasets only contain the identifier of the Twitter account but not any significant feature. Consequently, an additional data crawling process via the Twitter API is performed to gather further information about the available accounts which is needed for the analysis. As introduced in Section III, the following information is collected: • Popularity features including total number of both friends and followers, • Activity features including the following fields: creation date, average tweets per day, tweets & favourites counts, account age.
• Profile information features including: screen name, description, language, location, verified indicator, default profile indicator. After crawling and preprocessing the data, the final complete dataset is composed of 37438 Twitter accounts, where 25013 were annotated as human accounts and the remaining 12425 are bots.

2) INPUT USER ENCODING GENERATION
The crucial part of this first stage lies in the generation of an user encoding vector based on the aforementioned collection  of features to serve as input of the proposed deep learning model. In previous related studies [7], [12], [13], [20], the proposed solutions had two main constraints: 1) either they were metadata-oriented approaches so that, the text-based features were extracted at a semantic-level in terms of Natural Language Processing, or 2) they employed more advanced NLP procedures based on N-grams or DL solutions but they only supported a limited number of languages when performing the analysis.
However, our proposal addresses the aforementioned constraints by combining relevant metadata features along with powerful models capable of transforming text-based features into vectors independently of the language of the input text.
More specifically, given an input set of Users U = {u 1 , u 2 , . . . , u m }, a certain user account is represented as . . , m where u t i indicates its text-based feature vector whereas u z i represents the remaining metadata-based vector. Our proposed solution employs a mapping function f (u) to generate a new set of Userṡ In this case, we denote || to indicate a concatenation operation between this pair of vectors. The reason behind using a concatenation layer at the end of this process lies in the fact that the system only consider information coming from the same target object: the user account. Other alternatives widely used in Collaborative Recommender Systems such as computing the outer product [19] were not consider for this approach since in those scenarios, the information from the embeddings comes from two different sources: Users and Items, and the goal of the outer product is thus, to catch similarities and discrepancies between this two sets.

3) ENCODING TEXT-BASED FEATURES
Regarding the generation of the text-based vector from an input user account u i , a certain function g(u) is needed. Different state-of-the-art sentence-level encoders from several NLP frameworks were explored and investigated. In particular, the so-called Flair framework described in [2] was employed to combine state-of-the-art Word Embeddings (WE) and Transformers [39], [43] for extracting robust document embeddings from the text-based features. More precisely, the following main families of embeddings have been employed in this study: (i) Contextual string embeddings [3] which are trained without any explicit notion of words and therefore, words are modelled as sequences of characters. Moreover, words are contextualized by the surrounding text. The employed model was trained using the so-called JW300 Dataset described in [1]. In this study, both multi-forward and multi-backward embeddings are used. The dimension of their outputs is equal to 2048. (ii) BERT (Bidirectional Encoder Representations from Transformers) embeddings which were proposed and developed by [14] and are based on a bidirectional transformer architecture [39], [43]. In this study, the socalled bert-base-multilingual-cased has been employed. (iii) RoBERTa which is an adaptive version of the BERT embedding where the goal is to improve the performance in longer sequences, or when there are vast volumes of data as [41] suggests. In this case, we employed the so-called roberta-large-mnli pre-trained model. Furthermore, several experiments were conducted considering three different solutions. The first approach is based on using one or multiple stacked embeddings similarly to the approach proposed by [25] so that, all the text-based features from the user account are encoded at a sentence-level representation. Subsequently, a document-level representation is computed via a Pooling model, where an average of all the stacked sentence-level embeddings was calculated.
The second approach regards the training of a Long Short-Term Memory (LSTM) recurrent network over all the word embeddings used to generate the sentence-level encoding. Finally, the last approach directly uses an intermediate layer from a pre-trained Transformer model, to produce the document-level embedding.
Moreover, both the multilingual BERT transformer pre-trained model named as BERT-base-multilingual-cased and the RoBERTa pre-trained model named as roberta-largemnli have been employed. More specifically, the former generates a 768-dimensional embedded vector whereas the latter produces a 1024 embedded representation. All the details of the aforementioned pre-trained models can be found at the official Hugging Face repository. 2

4) ENCODING METADATA-BASED FEATURES
On the other hand, all the corresponding metadata features from an input user account are properly preprocessed and encoded throughout function h(u z i ) to be interpreted by neural networks. In addition, they are concatenated along with the aforementioned text-based features as it is remarked in Section III-A.
In particular, this set of features includes all the information related to both the popularity and the activity of the user account.

B. BOT-DenseNet
Once the set of input user vectors denoted byU is obtained, a second process is required to automatically identify an account as bot or human.
To address this goal, we propose a Deep Fully-connectedbased neural network named as Bot-DenseNet, which is capable of finding robust decision boundaries based on hidden patterns of the input vectors to better recognize bot accounts in Twitter. The details of the model implementation including classical parameters in neural networks such as activation functions, number of neurons in the hidden layers or the selected optimizers to perform the backpropagation and the Gradient Descent algorithm are summarized in Table 2. One 2 https://huggingface.co/transformers/ of the main differences from previous studies is the incorporation of the so-called Scaled Exponential Linear Unit (SELU) similarly to the approach proposed at [23] which obtains better results than the classical ReLU activation function.
On the other hand, the architecture of the system, presented in Figure 2, is composed of a set of blocks including Dense + Batch Normalization + Activation + Dropout layers in a sequential fashion as usual in general dense-based models.
Furthermore, the inclusion of a Batch Normalization layer within the architecture lies in previous studies such as the one described in [10], [21], where authors proved that the training performance indeed improves when using such normalization since this layer provides many benefits including a faster convergence as it allows to employ higher learning rates during the Gradient descent algorithm.
The aforementioned hyper-parameters were carefully selected after conducting several heuristic experiments in order to design a model with the best performance in terms of F1-score.

IV. EXPERIMENTAL RESULTS
One of the main objectives of this paper relies on analysing, via an ablation study, different input feature vectors based on Transformers as well as additional novel approaches to investigate the performance of the same DNN model. Moreover, The DL architecture was trained in the classical supervised learning fashion considering a binary classification where the Positive class refers to Bots whereas the Negative corresponds to Human accounts.
Due to the unbalanced constraint of the dataset, two main steps have been followed: (i) the so-called Stratify sampling updated in [26] has been employed to force having samples from both classes during the training and evaluation of the system, (ii) the F1-score metric as the main measurement of the performance of the system since it balances both precision and recall metrics in a single value and provides more realistic information about the capability of the model to detect both Positive and Negative classes than the classical accuracy metric.
Moreover, to mitigate the classical overfitting problem, which happens when the neural network is not capable of generalizing properly for unseen samples, two main widely used techniques have been incorporated: (i) Dropout firstly proposed in [36], which pursues the aim of deactivating some VOLUME 9, 2021  neurons randomly at each epoch with a certain probability; (ii) Early Stopping, fully described in [46], which attempts to stop the training process whenever the performance on the validation set has no longer improved in a certain number of epochs. Therefore, a hyper-parameter is required to indicate the number of consecutive epochs with no improvements in the loss function or in the metrics considering the validation set, and it is usually named as patience.

A. TRAINING & VALIDATING BOT-DenseNet
The aim of these experiments is to find the most appropriate text embedding to be added on top of the Bot-DenseNet along with the remaining metadata feature vector in order to find optimal decision boundaries for downstream tasks such as the one presented in this paper.
The outcomes obtained during the training and validation stages for all the possible input feature vectors are summarized in Table 3. Since the dataset is not balanced, the F1-score metric has a crucial role in the evaluation of the system in order to objective measure the performance of identifying bots in a social network such as Twitter where only a few accounts from the total set of accounts belong indeed to the bot category as it is described in previous studies [7], [27], [34]. In particular, since the model is trained using an Early-Stopping callback to stop the process in the epoch when the loss function in the validation set is no longer decreasing, the F1-score was computed by using both the recall and the precision at this specific epoch.
Moreover, in Table 3, each row indicates the pre-trained LM employed such as Flair, BERT, RoBERTa etc., as well as the approach followed to generate the final input user encoding (via Pooling, bidirectional LSTM or directly the embedding obtained from an intermediate layer of the Transformer model). In particular, the results presented in Table 3 reflect that when combining the text embeddings directly extracted from intermediate layers of the Transformers along with the metadata features, the proposed model Bot-DenseNet achieves higher scores in terms of F1-score in both training and validation sets. On the other hand, when using either Pooling or LSTMs to produce the final text embeddings, the F1-score metric in the training phase is higher than in the validation phase which indicates that the model is suffering from overfitting. As a result, this issue may provoke a decrease in the performance of the system when analysing unseen observations in future predictions.

B. SELF-SUPERVISED USER EMBEDDING
Our proposed model is capable of identifying suspicious Twitter accounts based on a robust set of input features. However, as it is well-known in the Deep Learning framework, intermediate layers are usually an adequate embedded representation of the inputs which can be employed in other downstream tasks such as text classification or similarity analysis in a more efficient way due to their low-dimensional nature. Thus, after training our proposed model, it produces a relevant representation of an input Twitter account in a self-supervised fashion since such representation was automatically learned by the intermediate hidden layers. Hence, Bot-DenseNet can also be used to encode any user account as a 256-dimensional vector throughout its last intermediate layer. To better visualize the final embeddings obtained by the last layers of the different models, a 2D projected representation of them were computed using the so-called T-SNE algorithm, updated in [30] with a level of perplexity equal to 80.
More specifically, Figure 3 includes the embeddings as a result of training the model using directly the pre-trained Transformers models whereas Figure 4 shows the outcomes when using different combinations of Word Embeddings along with either Pooling or LSTMs on top of them to produce the final text vector. Moreover, Figure 3 remarks the potential of Transformers in any downstream NLP tasks such the one presented in this paper where their intermediate layers have been employed to obtain robust representations of the text-based features of the user accounts. As a consequence, they have increased the capability of Bot-DenseNet to better distinguish between bots and human accounts in a more efficient manner. Furthermore, Figure 3 shows that the boundaries determined by both BERT and RoBERTa transformers without any additional step to generate a text encoding (neither Pooling nor LSTMs) are simpler and more adequate than the ones provided by combining different text embeddings which is summarized in Figure 4. This fact is also presented in the F1-score metrics from Table 3, where it is clear that these two solutions are the ones which are not overfitting the data and thus, they are more suitable to be employed as top of our proposed Bot-DenseNet model.

C. DECISION MAKING CRITERIA FOR BOT-DenseNet
After conducting the aforementioned experiments, a final decision to select the best configuration of the model is required. To do so, the following criteria have been considered: (i) The performance of the model in terms of F1-score in both training and validation to provide an objective criteria when making the final decision; (ii) The simplicity of the model in terms of both trainable parameters and the length of the input feature vector as it is remarked in Table 4; (iii) The simplicity of the final decision boundaries to distinguish between Bot and Human accounts which is assessed by observing the low-dimensional embeddings distribution in a TSNE projection. This aspect arises as crucial in order to propose a robust model capable of generalizing in future applications.
Thus, considering these elements for making the final decision both Table 4 as well as Figure 3 have been analysed to provide an objective decision.
Firstly, regarding performance, the best model is the one that uses the so-called RoBERTa Transfomer on top of it according to the F1-score achieved during the validation phase. Secondly, when it comes to simplicity, it is noticed that the best approaches are the ones that have employed LSTMs during the generation of the input vectors but they have achieved considerable lower results in terms of F1-score and therefore, they were discarded from the further analysis. On the other hand, the family of approaches that have employed a Pooling procedure are the ones with the highest level of complexity in terms of trainable parameters as well the lowest performance based on the F1-score, thus, they were directly discarded from the final decision.
Regarding simplicity in terms of the samples distribution after applying the TSNE method, Figures 4 and 3 shows that the boundaries achieved when using Transformers as part of the input feature vector are more suitable and simpler than the rest of configurations.  Consequently, the final model uses the so-called RoBERTa transformer on top of it since it provides a remarkable trade-off between precision and simplicity, which are critical aspects to be considered when implementing Deep Learning models Finally, Table 5 shows a comparison between our proposed DL architecture in comparison with previous studies in terms of both the performance throughout the F1-score metric, the learning approach as well as the capability of incorporating language-dependent features. This comparison enhances our proposed method in the following aspects: (i) Language-dependent features throughout the analysis of text descriptors via Transformers to improve the input feature vector and thus, the robustness of the system when facing non-English languages. (ii) Hybrid learning, since it produces embeddings throughout its intermediate hidden layers (unsupervised learning) once the model has been trained using a limited set of annotated data (supervised learning). (iii) Promising scoring metrics in comparison with latest studies.
All the details of the implementation as well as the code for re-training or testing the proposed system can be found at this Github repository. More specifically, a Python library named as User2Vec was released to foster further research in this field.

V. CONCLUSION & FUTURE WORK
In this paper, a robust solution for detecting Bots in Twitter accounts has been described. In particular, this study has taken advantage of Transfer learning techniques via powerful state-of-the-art NLP models such as Transformers to extract compact multilingual representations of the text-based features associated with user accounts. By doing so, several constraints presented in previous studies related to process text-based features to improve the input feature vector from multiple languages were mitigated.
Furthermore, by employing the text encodings along with additional metadata on top of a dense-based neural network, a final classifier named as Bot-DenseNet has been trained and validated using a large set of samples collected via the Twitter API. More specifically, several experiments were conducted using different combinations of Word Embeddings, document embeddings (Pooling and LSTMs) and Transformers to obtain a single vector regarding the text-based features of the user account. Subsequently, a detailed comparison of the performance of the proposed classifier when using these approaches of Language Models as part of the input has been presented to investigate which input vector provides the best result in terms of performance simplicity in the generation of decision boundaries and feasibility.
In particular, the comparison of these experiments suggested that the Bot-DenseNet achieves the most adequate trade-off between performance and feasibility when using the so-called RoBERTa Transformer as part of the input feature vector.
Consequently, this paper provides two main contributions to the scientific community including a DL model for automatically detecting bots as well as a robust manner of representing any Twitter account as a low-dimensional feature vector throughout an intermediate layer of the aforementioned model. Moreover, this compact representation of the Twitter account can be used as a baseline for recommender or search engines, similarity analysis or any other application related with social media mining.
Finally, this study also remarks the outstanding performance of novel Transformers in downstream NLP tasks as the one presented, by providing a more robust input vector which leads the final classifier model to be more capable of extracting relevant low-level features from it. As Future work, the latest Transformers such as the GPT-3 [17] and T5 [33] will be considered for generating the input vector of the proposed DL model in order to compare the performance with the work described on this paper. Moreover, novel approaches such the one described by authors in [24] to automatically generate non-parametric Two-Sample tests based on the so-called Maximum Mean Discrepancy (MMD) [18], will be considered once all the user embeddings are generated, to find discrepancies and similarities between the distributions of both bots and non-bots embeddings.  ALBERTO BELMONTE HERNÁNDEZ (Member, IEEE) received the degree in telecommunication engineering, the master's degree in communication systems, and the Ph.D. degree (cum laude) from Universidad Politécnica de Madrid (UPM), in 2014, 2016, and 2020, respectively. He is currently an Assistant Professor in several subjects. Since 2016, he has been with the Visual Telecommunications Applications Group (GATV), UPM. He is actively working on artificial intelligent applied to multimedia content and sensors for pattern detection, recognition, and fusion. He has been developing technical parts in national and EU projects.  He is the author and a coauthor of more than 60 articles and several books, book chapters, and patents in the field of ICT networks and audiovisual technologies. He has been participating with different managerial and technical responsibilities in several national and EU projects, being coordinator of five EU projects in the last six years. He had participated in national and international standardization for a DVB, CENELEC TC206, and so on. He is also a member of the Program Committee of several scientific conferences. VOLUME 9, 2021