Scrooge Attack: Undervolting ARM Processors for Profit

Latest ARM processors are approaching the computational power of x86 architectures while consuming much less energy. Consequently, supply follows demand with Amazon EC2, Equinix Metal and Microsoft Azure offering ARM-based instances, while Oracle Cloud Infrastructure is about to add such support. We expect this trend to continue, with an increasing number of cloud providers offering ARM-based cloud instances. ARM processors are more energy-efficient leading to substantial electricity savings for cloud providers. However, a malicious cloud provider could intentionally reduce the CPU voltage to further lower its costs. Running applications malfunction when the undervolting goes below critical thresholds. By avoiding critical voltage regions, a cloud provider can run undervolted instances in a stealthy manner. This practical experience report describes a novel attack scenario: an attack launched by the cloud provider against its users to aggressively reduce the processor voltage for saving energy to the last penny. We call it the Scrooge Attack and show how it could be executed using ARM-based computing instances. We mimic ARM-based cloud instances by deploying our own ARM-based devices using different generations of Raspberry Pi. Using realistic and synthetic workloads, we demonstrate to which degree of aggressiveness the attack is relevant. The attack is unnoticeable by our detection method up to an offset of -50mV. We show that the attack may even remain completely stealthy for certain workloads. Finally, we propose a set of client-based detection methods that can identify undervolted instances. We support experimental reproducibility and provide instructions to reproduce our results.

I. INTRODUCTION
Cloud providers continuosly upgrade their commercial offerings to adapt to market and customer needs. While the vast majority of them offer computing instances based on x86 processors, the availability of ARM-based cloud instances is quickly expanding. ARM processors are increasing their market share of server-grade machines [1,2,3,4,5,6,7,8], thanks to additional energy and performance improvements. Before ARM announced its Neoverse [9] microarchitecture, there were no server-grade ARM processors to license. Companies had to customize application-grade ARM processor designs for their server-grade platforms [5,7]. Recent ARM servergrade processors [1,3,7,8] are based on custom-developed ARMv8 microarchitectures. For example, Amazon [5] deploys ARM-based processors currently shipped in off-the-shelf ARM hardware. Their AWS Graviton processor is essentially a more powerful quad-Raspberry Pi 4B [10]. Scaleway offered instances based on custom-made ARM SoCs with servers smaller than a business card [11]. Table I summarizes a subset of available server-grade ARM processors, supported instruction set architectures (ISA), and providers deploying this hardware. Several generations of ARM processors [1,2,5,6,7] are currently available across cloud providers. ARM processors also started reaching into the supercomputing market segment. We expect an increasing availability of ARM Neoverse processors and future server-grade ARM instances to close the performance gap to x86. On the one hand processor manufacturers specify conservative voltage margins due to process variation [12]. On the other hand processors offer different power management mechanisms to adjust frequencies and voltages. While marginal energy savings on a single device appear unimportant, it is of importance at scale, especially since power savings of the cloud infrastructure accumulate for each CPU. The energy footprint of a single execution step (i.e., one single instruction on a processor) is fairly independent of the CPU frequency but dependent on the CPU voltage [13]. Decreasing the CPU voltage below the nominal value to conserve power is called undervolting 1 . Besides energy savings, undervolting directly influences core temperature and can also reduce core aging [14]. Undervolting, however, incurs the risk of introducing soft [15] and hard-errors related to timing violations [16]. These types of errors can be mitigated by carefully analyzing the guardband of processors [17]. In this practical experience report, we consider a scenario where processors supporting a cloud infrastructure are undervolted by an excessively economic and malicious cloud provider (a scrooge §III-A) to profit from additional electricity bill savings, while cloud users (from here on referred to as users) observe similar performance. Unfortunately, undervolting cannot be applied arbitrarily. In fact, it comes at the cost of processor reliability when the supplied voltage is insufficient to drive the processor's frequency. We believe this is a risk that malicious cloud providers are willing to take. For users, undervolting opens up a new attack vector against their cloud applications (see our threat model in §III). The main research questions we address in this work are: RQ1: What is necessary for a malicious cloud provider in order to pull off a stealthy undervolting strategy?
RQ2: Does a cloud user have the ability to uncover such an undervolting strategy?
To answer those questions, we need to lay the foundation to better understand consequences of (arbitrary) undervolting, both from the cloud provider and client perspective. In fact, depending on supply voltage, frequency, load, and temperature of the CPU, execution steps can yield erroneous computations. While recent attacks [18,19] have demonstrated how undervolting can be effectively exploited to gain access to sensitive information, we deal with a different threat model: the infrastructure is undervolted on purpose by a powerful attacker (i.e., the cloud provider), at the risk of exposing hardto-detect unreliable computing instances for users. Without physical access to instances, nor being able to directly manipulate the supply voltage or frequency, a user's options remain limited. Nevertheless, a user can adjust the processor's load and operating performance points ( §II-B) to influence its heat dissipation. In order to operate under full load, the processor has to be set to the highest operating performance point, which implies the highest frequency and supply voltage setting. Consequently, undervolted processors present higher probability for erroneous computations to occur because they are unable to maintain high frequencies. This probability is further increased by the propagation delay due to high operating temperature. If erroneous computations result in faults, one can observe application crashes, or kernel panics, leading to cloud instance unavailability. While service level agreements (SLA) [20] typically cover such scenarios, a malicious provider might try to balance its actions to only yield erroneous computations not resulting in faults, basically overcoming SLA protections. For this reason, we designed a non-selective fault injection method for detecting the scrooge attack. The sole purpose of the detection method is to yield intentional application crashes or kernel panics on undervolted instances such that the user is covered by the SLA. While interesting, we consider cloud providers or users exploiting undervolting to leak sensitive information [21,22] to be out of scope of this work.
Interestingly, ARM-based Raspberry Pis have already been collocated in cloud data centers [23]. With the intent to Our contributions are as follows: • We describe a novel attack scenario based on undervolting by a scrooge cloud provider to lower energy costs. • We demonstrate how cloud users can with a certain probability detect this novel scrooge attack. • We provide a temperature-based guardband analysis to narrow down the operation voltage range of an ARMbased processor ( §V-D). • We describe how our analysis can be used to automatically identify undervolted instances ( §V-E) • We present potential energy gains of undervolting systems using a reliability benchmark ( §V-F). In general gains can reach up to 37 %.
This practical experience report is organized as follows. Section II provides background on the low-level mechanisms used to undervolt a processor and the Raspberry Pi platform as well as the associated side-effects. Our threat model is given in Section III. We overview our detection method in Section IV. Our in-depth experimental evaluation is presented in Section V. We discuss and review related work in Section VI and Section VII, before concluding in Section VIII.

II. BACKGROUND
This section defines more precisely a few concepts related to power management ( §II-B), i.e., frequency and voltage scaling and associated techniques such as Dynamic Voltage and Frequency Scaling (DVFS) and Adaptive Voltage Scaling (AVS). In §II-D we explain the relation between such techniques and how they affect the overall reliability of a system.

A. ARM in data centers
Collocation offers allow users to either ship or buy Raspberry Pis in order to deploy lightweight workloads on this lowenergy hardware and thus free up resources on high-energy x86 hardware. Furthermore, Raspberry Pis are the size of credit cards and have much lower cooling demands, which allows hosting a large number of units in a single rack [11]. Such off-the-shelf hardware setups allow for large-scale node deployments as needed in data processing or cloud computing workloads. While off-the-shelf hardware typically lacks in performance and storage capability, its energy consumption remains comparable to server hardware. Figure 2 compares the energy consumption of ARMbased off-the-shelf hardware (i.e., three different Raspberry Pi models) against server-grade hardware using different x86 architectures. We run a cryptographic (CPU-bound) and a memory allocation (Memory-bound) stressor while measuring the entire device power consumption. The x86 processors used were an AMD EPYC and three different Intel Xeon processor generations, i.e., Broadwell, Kaby Lake and Harpertown. This is a direct comparison of the execution of two distinct binaries of the same source code on two different architectures based on a common metric (J/op). Despite a large difference in power consumption between ARM and x86 hardware, their energy consumption per operation is comparable. For a detailed discussion on energy metrics with comparison of different architectures we refer to [13]. We observe no major difference for CPU-bound operations between different architectures [24]. However, memory-bound operations on offthe-shelf hardware have higher energy consumption. In the case of the Raspberry Pi models, these are due to cache size and memory transfer rate. Nevertheless, off-the-shelf hardware achieves lower energy consumption for both operations compared to older server-grade x86 hardware, i.e., Harpertown. These results indicate that replacing old x86 hardware with recent off-the-shelf ARM-based nodes in data centers will result in energy savings.

B. Power management
The power dissipated by an integrated circuit depends on static power (leakage current) and dynamic power (switching power). Since about 2005 [25] the power dissipation contribution of dynamic power has become much higher than static power. Nowadays, with the decreased transistor size and lowered threshold voltages, static power is becoming more and more important [26]. In the following, we outline techniques to reduce dynamic power.
Frequency scaling regulates (dynamically) the frequency of an integrated circuit in order to change performance, conserve power or reduce the amount of heat dissipation. Reducing the frequency at a constant voltage is called underclocking or throttling, while increasing the frequency is called overclocking. The dynamic power dissipated by an integrated circuit over a period of time is given by P = CV 2 f , where C is the capacitance, V is the voltage, and f is the frequency. Thus, increasing the frequency results in a higher power consumption and operating temperature.
Voltage scaling is an open loop system, in which the voltage of an integrated circuit is regulated (dynamically) based on an external setting. Increasing or decreasing the voltage while keeping the frequency constant is called overvolting and undervolting, respectively. Regulating the voltage enables increasing the frequency or conserving power of an integrated circuit, a particularly useful aspect especially for battery-powered devices. Changing the voltage influences the rate at which capacitances can be charged and discharged. Thus voltage determines the speed and frequency at which an integrated circuit can be operated. Modern operating systems do not provide direct support to adjust a processor's voltage individually. The processor's voltage is either regulated by model-specific registers [27] or through firmware.
DVFS is the simultaneous software-controlled regulation of voltage and frequency scaling of an integrated circuit. Depending on the process variation (variation of integrated circuits when fabricated) ARM system on a chip (SoC) manufacturers specify a set of operating performance points (OPPs) under worst case conditions. These OPPs are pairs of clock frequencies and voltages under which the integrated circuit is operational with a sufficiently large margin while taking into account thermal conditions. In Linux the CPUFreq kernel driver [28] will chose a set of OPPs based on a specified governor. DVFS has been extensively studied [26,29,30] to accelerate multi-threaded applications. x86 manufacturers use their own DVFS implementations [31,32,33].
AVS [34] is a closed loop system where the voltage is regulated based on its process variation, aging and a feedback loop of sensor data. A hardware monitor or software backed by sensor data determines if the changes made to the system are sufficient or if additional changes are necessary. AVS requires support from both the processor and the power regulators, in order to adjust the voltage accordingly. The Raspberry Pi models B used in this report are equipped with an AVS system.

C. Raspberry Pi
The Raspberry Pi's firmware is configured at boot time by a text file containing property-value pairs. For example, the frequency and voltage can be set in this configuration file. A particularity is that voltages can only be set to a nominal offset in steps of 25 mV. These offset steps are referred to as overvoltage levels in the Raspberry Pi firmware configuration. This configuration file is then parsed by the firmware. While this undervolting configuration is specific to the Raspberry Pi, other hardware can more easily be undervolted dynamically at runtime. Notice that the requested CPU frequency in the operating system can deviate from the actual frequency regulated by the firmware. This is in particular the case if the device is throttled for reaching the thermal hard limit at 85°C. Additionally, the 3B+ has a soft limit temperature at 60°C that will throttle the CPU frequency and voltage.

D. Reliability
There are several approaches to determine a processor's reliability in an undervolted operating regime. Known benchmarks (e.g., SPEC CPU2006 [35], PARSEC [36], etc.) are still used [17,37,38]. Recently, new specialized power viruses [39,40] have been proposed to maximize power consumption and voltage noise. Even small proof-of-concept programs are sufficient for fault detection under voltage and/or frequency scaling [18,21]. Finally, such programs can also be used to characterize the guardband of a system [21,41].
In this practical experience report we distinguish between three regions with respect to the guardband: safe, critical, and failure. A safe region has a sufficiently high voltage margin, such that erroneous computations or transient faults cannot occur. The critical region designates a small voltage band in which the processor occasionally experiences erroneous results or transient faults. Inside the failure region it is impossible to boot the operating system either because the voltage cannot support the processor's frequency or because erroneous computations and transient faults lead to kernel crashes or panics.
Undervolted instances become unavailable in case a transient fault leads to an instance crash. From our perspective, current SLAs cover single instances that have crashed because of undervolted hardware, provided users can sufficiently support these claims. The situation is trickier with multiple instances. Deployed instances would have to crash simultaneously, yet process variation plays into the cloud provider's hands. These crashes are non-deterministic and, therefore, process variation helps obfuscating the undervolted setup. Only simultaneous crashes satisfy today's cloud provider restrictions in order for users to be covered by the SLA.
III. THREAT MODEL In this section we discuss our threat model. In particular, we intend to clarify: (1) which techniques can a malicious cloud provider use to hide an undervolted processor from an unsuspecting user, and (2) which are the methods for a curious user to reveal an undervolted processor? Notice that we validate these methods on a specific hardware configuration (i.e., Raspberry Pi boards using Broadcom BCM2837/BCM2711 processors), but the discussion holds for other platforms relying on similar voltage regulation mechanisms.

A. The scrooge cloud provider
We assume the cloud provider has full access to the physical infrastructure and can connect remotely to the physical machines [42]. Furthermore, the cloud provider purposefully undervolts its ARM-based hardware to benefit from additional savings. Firmware configurations can be hidden from users for malicious or security purposes. By maliciously intercepting any voltage reading requests (see §III-C), the cloud provider ensures that the undervolted state of the cloud infrastructure remains oblivious to users. A cloud provider must find the sweet spot [17] for the undervolt configuration in or near the critical region to provide sufficiently stable instances.

B. The curious cloud user
The curious cloud user is suspicious of the cloud provider and intends to uncover its potentially obfuscated activity. Instances of the cloud provider can exclusively be accessed remotely by the user. The only way for a user to detect an undervolted processor is by querying the firmware, normally using a specific executable command file for that. By reading values from the firmware and comparing them to values in the boot configuration file, a user can detect an undervolted processor. If results of firmware queries can be forged, it becomes difficult for a user to uncover the scrooge cloud provider. A confidential and tamper-proof message exchange with the firmware is essential to detect an undervolted processor.
A user can suspect an undervolted processor to operate in the critical region in case of kernel warnings or kernel panics appearing during the system boot or while the system is running, despite these being generic kernel warnings rather than specific ones. In particular if the booting time is longer than expected, then this might hint at a failed boot attempt where the kernel crashed. Most systems have a kernel log that can be consulted by the system administrator. However, a cloud provider can tamper with those kernel logs, and the system utilities are outside the trusted computing base.

C. The scrooge attack
The scrooge cloud provider makes undervolted ARM instances available to users. These undervolted instances should be indistinguishable from nominal voltage instances. This includes configuration, firmware, and tools querying CPU voltages. Thus, the undervolt configuration needs to be exchanged for a nominal configuration and any CPU voltage reading request needs to be intercepted. Figure 3 shows different actions the cloud provider has to perform during an instance lifecycle in order to hide the undervolt configuration. When a user boots such an instance, the cloud provider must ensure that the undervolt configuration is loaded by the firmware on the machine the instance is running on. However, this undervolt configuration should not be accessible once the user is connected to the instance. The undervolt configuration has to be swapped for the nominal configuration (Fig.3-). Depending on the configuration mechanism, the file system that was booted may be different from the file system the user finds after booting. This includes firmware, operating system kernel, binaries, etc. A hidden or obfuscated system service could perform this task while the operating system is booting. An even stealthier approach involves a trusted operating system [43] or auxiliary devices [44] which exchange the configurations before the operating system is booted. Therefore, without proper system attestation, there is no guarantee about the authenticity of the system users believe they have booted. During reboot or shutdown of the machine configurations might have to be swapped back (Fig.3-) again.
Any CPU voltage reading request needs to be intercepted and substituted by a plausible nominal voltage value. This will typically involve a kernel driver that will handle the communication with the firmware or accessing model-specific registers. The request can then be intercepted directly in the user space tool or the kernel driver (Fig.3-). From the kernel driver the request is forwarded (Fig.3-) and the actual undervolted CPU voltage value is returned to the kernel driver (Fig.3-). The kernel driver then substitutes this value by a some nominal voltage value, e.g., by adding the undervolt offset to the value. The alleged nominal voltage value is then returned to the user (Fig.3-). A more costly but stealthier variant involves the trusted operating system, to which the cloud provider could delegate voltage reading requests instead of a kernel driver.
If users are allowed to deploy their own kernels, then the cloud provider needs a different approach. Voltage reading request can no longer be intercepted in kernel space. Instead, the cloud provider needs to use the hypervisor to intercept CPU voltage requests and substitute them similarly to the kernel driver approach.
In our threat model we assume that the cloud provider will make use of these mechanisms and obfuscate as much as possible the undervolted state of the infrastructure from users, a practical effort with significant benefits. Without access to the firmware configuration nor any untampered message exchange with the CPU voltage regulating mechanism, a user can never be sure to obtain a genuine voltage reading.

D. Relevance of discussed techniques
While a scrooge cloud provider has powerful mechanisms in place to hide its undervolted instances, the curious user can still expose this misbehaviour. For instance, the processor's frequency and package temperature are viable options to test for undervolted conditions. The techniques presented in Section V demonstrate to which extent users can deploy applications stressing aforementioned options on instances and how accurately conclusions can be drawn.

IV. SCROOGE ATTACK DETECTION
This section describes the user's detection method as well as under which conditions the detection method works and where difficulties may arise. Furthermore, we mention potential concerns of cloud providers.
We assume that users cannot trust any firmware or system reading on instances. As such, users have no reference to any parameters for adjusting the detection method to the attack. Users can for this reason make use of simple CPU-bound programs that will put the processor under maximum load while monitoring for faults. Inspired by [18] we propose implementing an arithmetic computation (i.e., multiplication) for which we can validate the result. First we generate two random numbers which are then multiplied until the instance crashes while alternating the position of multiplier and multiplicand. Murdock et al. have observed, that the position of the multiplier and the multiplicand can lead to a faulting instruction. After each multiplication the result is compared to the original result. While the processor operates at maximum load it will run at the highest frequency and dissipate heat which will raise its temperature. Under these conditions we achieve the highest probability to inject faults related to timing violations. Depending on the complexity of the RISC circuitry in the ARM processor, certain instructions are more likely to fault then others. To this end, the detection method might not inject faults in its own computation (due to its simple nature), but more likely in other processes. This behavior is favorable, as it allows to run the detection method until the instance itself becomes unavailable due to multiple critical faults in system relevant processes. Thus, we detect an undervolted cloud instance using the detection method by gradually failing processes to crash the instance and make it unavailable.
The detection method depends strongly on how aggressively machines are undervolted and the cooling system employed by the cloud provider. The less a machine is undervolted, the higher the temperature needs to be raised by the detection method to fault processes and vice versa. A good cooling system is a lesser problem than a weakly undervolted machine. With a good cooling system the detection method requires a longer time to raise the processor's temperature. On the one hand implementing a soft limit temperature throttle in order to prevent this detection method is not an ideal solution. Users are less inclined to pay for a service which underperforms compared to alternative services. On the other hand weakly undervolting machines defies the scrooge cloud provider's original idea of minimizing the electricity bill. The cloud provider's options are limited to completely prevent the detection method from unveiling the scrooge attack. Even the powerful setup of the cloud provider to tamper with CPU voltage readings is not sufficient denying the detection method. The scrooge attack has the disadvantage that detection methods have a simple design, but it has the advantage that proving the undervolt state without the firmware is difficult.

V. EVALUATION
In this section we explore the behavior of Raspberry Pi processors under different nominal and undervolted setups. The information gained from these experiments allows quantifying the attack parameters and determining the type of processes to use for the detection method. Then, we derive the probability at which our detection method can successfully uncover the attack. We begin by describing our experimental setup to undervolt Raspberry Pis before evaluating the firmware's throttling behavior when reaching the soft limit and limit temperatures. The temperature-based guardband analysis allows detecting the critical region of the device and defines the margin for an undervolt setup. Faults that occurred during the guardband analysis are analyzed to describe the fault injection of the detection method. Finally, we measure the energy efficiency of the undervolted hardware with a reliability benchmark. The dataset gathered for this evaluation is publicly available at https://github.com/ChrisG55/Scrooge-Attack.

A. Experimental settings
We use the three Raspberry Pi models 3B, 3B+, and 4B, while booting from the same SD card a Raspbian Buster distribution (https://github.com/raspberrypi/linux). All units rely on recent firmware releases (since June 2020). To simulate a realistic cloud scenario we take all measurements in an airconditioned room at (24 ± 1)°C and connect the Raspberry Pis to Ethernet and run the SSH daemon. The Raspberry Pis are monitored over UART from an auxiliary machine. No other peripherals are connected to the Pis in order to minimize any interference. Both the Raspberry Pi's and the auxiliary machine's clocks are synchronized using NTP, to easily correlate the power consumption logs recorded on the auxiliary machine to the benchmark running on the Raspberry Pi. The power consumption of the Raspberry Pi is recorded by an Alciom PowerSpy2 [45] over bluetooth. The entire experimental setup can be fully automated for a manageable effort using a network-capable power strip.

B. Soft limit temperature throttling
We start by evaluating the firmware behavior when reaching the soft limit temperature while running under the CPUFreq

C. Limit temperature throttling
Next, we evaluate the firmware behavior when reaching the limit temperature while running under the CPUFreq performance governor. At the limit temperature, the firmware will throttle the processor to prevent thermal runaway. Notice that model 3B+ is not included here, as it is taking too much time reaching the limit temperature while already being throttled for going beyond the soft limit temperature. Neither the 3B nor the 4B reduce the voltage when reaching the limit temperature as shown in Table III. However, both models reduce their frequency. For every voltage in the scale, there is a range of frequency values that do not crash the processor. The sets of frequencies indicated in Table III are frequencies downstepped to by the firmware until thermal limits are satisfied when being throttled. The 3B is reducing its ARM CPU frequency f L arm in steps of about 54 MHz (except for the first step) while the 4B significantly reduces its frequency by 500 MHz. In addition the 4B also reduces its GPU frequency f L core by 167 MHz. We find that reaching the limit temperature will reduce the load put on the processor by the detection mechanism and reduce its temperature which lead to a lower fault injection rate.

D. Temperature-based guardband analysis
The temperature-based guardband analysis helps identifying voltage margins of the Raspberry Pi models. While this analysis supports the cloud provider in selecting an undervolt offset, its core principle can also be exploited by users to uncover the scrooge attack. This benchmark consists of three stages: 1) booting the operating system while undervolted before 2) adjusting the SoC's temperature either actively or passively and 3) running a billion iterations of the multiplication benchmark described in §IV as a single-threaded process. We set the CPUFreq governor to performance right before starting the multiplication benchmark. This will guarantee that the multiplication benchmark is started at a well defined temperature and that it runs at a constant, maximum frequency.  After each benchmark execution we reduce the ARM CPU voltage level in the configuration and reboot the system. This process is repeated until the system no longer boots because the supply voltage has gone below threshold or because the system has crashed. The results of this analysis are shown in Figure 4. All Raspberry Pi models keep a sufficient margin with their nominal voltage (connected black bullets) configuration to the critical region. Further undervolting of the ARM CPU into the critical region results in occasionally failing processes. Undervolting the ARM CPU beyond threshold voltage makes it impossible to boot the hardware. Our multiplication benchmark, that verifies the correct operation of the ARM CPU, never detected an incorrect result. We explain this characteristic of the multiplication benchmark, which is purely based on arithmetic operations, by not being on a timing-critical path to force an incorrect operation of the ARM CPU.
We can also see that the undervolting depends directly on the SoC's temperature. For instance on the Raspberry Pi 3B we clearly observe slightly rising regions, which result from small adjustments made by the AVS system. This is mainly due to the resistivity of the circuitry that increases with temperature.
The Raspberry Pi 4B can only be undervolted once to level −1 at −15 mV due to missing overclocking 2 support in the firmware. This undervolt limit is indicated by the blue line. However, we observe some basic overheating protection mechanism that slightly lowers nominal voltage by −12.5 mV in the range of 50°C to 70°C.

E. Implications on the detection method success rate
In order to produce Figure 4 and to fully characterize safe undervolt parameters we (as cloud provider) ran overall 741 guardband analyses. In 265 of those 741 guardband analyses the system had experienced a failure or crashed as a result of operating in the critical region. The ARM architecture provides the Exception Syndrome Register [46], which the operating system can consult to diagnose the type of exception generated by a process. Among the 265 failed or crashed runs we identified 407 process failures of the following 5 types: (1) paging requests (46.4 %), (2) freeze during boot  . These types of failures, with the exception of (2), usually generate a kernel oops. A kernel oops happens when the operating system kernel detects an incorrect behavior of a process and can possibly resume execution of the system. In some cases execution cannot be resumed because of system dependencies or unavailable system resources as a result of the failing process. The kernel will raise a panic and halt the system if a kernel oops occurs in an interrupt handler. If possible, the operating system kernel will log this kernel oops diagnose in the system log.
With the information from the 265 failed or crashed guardband analyses we derived a temperature-dependent failure rate for the 3B and 3B+ in Figure 5. Notice that the 4B is not included, as it's firmware does not provide undervolting support and we could not provoke any failures in this system. Our analysis indicates that at 60°C the detection method has the highest probability with a 40 % chance on the 3B+ respectively a 90 % chance on the 3B to provoke a failure in a system operating in the critical region. Compare with the results in Section V-G from the point of view of a cloud user running the detection method. With an even more aggressive undervolting at −100 mV, failures can already be reliably provoked at 40°C on both devices. These failures were provoked in at least 33 different processes (34 % user, 15 % kernel processes and 51 % unknown processes) of which the multiplication benchmark is never among the known failed Table IV: STRESS-NG ETR heat map indicating the relative energy efficiency for an undervolted setup compared to a nominal setup. The darker the shade, the more energy-efficient the stressor ran. processes. For the 3B+ the failure rate is dropping at 70°C due to the temperature soft-limit throttling in the firmware to bring the system back into the safe region. At nominal voltage and in the safe region no failures could be provoked in any system during temperature-based guardband analysis execution.

F. "Reliability" and energy efficiency
A common option to test the reliability of the selected undervolt parameters for our systems is to run benchmarks. Despite stressing the systems for several hours, we (as cloud provider) could not provoke any failures in these systems using STRESS-NG. Consequently, as cloud provider we would assume, that our chosen undervolt parameters keep the systems in the safe region as we covered successfully a large set of stressors with our benchmark. However, this assumption on our system's reliability turns out to be deceptive. These stressors and other benchmarks are not optimized to push all components of these processors to their limits. For example, the collection of statistics in a stressor's main loop can introduce sufficient overhead for not being able to test a component's limits. Therefore, given the circumstances, an undervolted processor can suddenly be operating in the critical region. In our scenario, these circumstances can be changed by raising the heat dissipation of the processor to increase its temperature. The detection method is based on this exact principle, where a cloud user tries to trip the cloud instance processor into the critical region by putting the processor under heavy load and raising its temperature. The interesting aspect of this detection method is that there is no specific requirement on the program putting the processor under heavy load. Instead, the detection method exploits the nature of all other processes running simultaneously on the system by eventually failing. While undervolting improves energy efficiency of processors in general, we believe that processors should also be equipped with features improving the reliability while operating under such critical conditions, i.e., error correcting codes. We show in heat map Table IV the energy efficiency for all three Raspberry Pi models based on the ETR ratio of the undervolted to the nominal setup. For the measurements we use two cooling setups: active and passive cooling. As there are many cooling solutions and because cooling solutions differ among cloud providers, our measurements do not include the energy consumed by our cooling system. Whether these additional gains outweigh the cost of the active cooling system remains an open but also interesting question. We ran up to 169 stressors sequentially of which 27 are shown in the heat map. Each stressor was configured with a timeout of 60 s.
Our results show that when the Raspberry Pi's are actively cooled, we can achieve higher energy efficiency. It is even possible to undervolt the device further. For example, with the 3B+ we were able to undervolt up to −100 mV and in some rare instances also run the benchmark successfully. One particular observation is made on the 3B that uses an older thermal design. On the 3B passive cooling results in higher energy efficiency compared to active cooling. 18 of the 27 stressors were more energy efficient with passive cooling than active cooling on the 3B with an average of −4 % energy efficiency per stressor. Occasionally we also observe better energy efficiency in the passive cooling setup than in the active cooling setup on the 3B+ (e.g., hrtimers with −20 % and judy with −7 %) and 4B (e.g., futex with −14 % and sysfs with −13 %). We noticed that some stressors have a large variance in the number of operations. Hence, if these stressors achieve a higher than average number of operations during the measurement, their energy efficiency improves proportionally. With the 4B we notice only minor improvements in energy efficiency. Again, this is due to the lack of overclocking support but also because of the lower core supply voltage compared to the other models. For the 3B+ and 4B we can say that a setup with active cooling results in larger energy gains compared to a passive cooling setup. On average across the 27 stressors 5 % / 9 % (active/passive) were saved on the 3B, 6 % / 6 % were saved on the 3B+, and 2 % / 1 % were saved on the 4B. The highest energy efficiency observed on the 3B was −10 % / −37 % on the hrtimers stressor. On the 3B+ −13 % / −20 % were saved on the fork / hrtimers stressor. Finally, the −30 % / −16 % were saved on the 4B with the hrtimers / futex stressor.

G. Detection method parameters
In this subsection we quantify the detection method parameters (i.e., run-time and temperature) based on undervolted baremetal and container instances. Deploying virtual machines on the Raspberry Pi is impracticable and were therefore not included in our evaluation. To run containers on the Raspberry Pi we deployed a small Kubernetes cluster. The detection method is deployed by the cloud user on instances using SSH. Figure 6 shows histograms with crashes on bare-metal and container instances deployed on the 3B and 3B+. We show the run-time of our detection method and the temperature at which instances crashed. Our observations made with the temperature-based guardband analysis in subsection V-E are confirmed by the temperature histograms. The run-time strongly depends on the processor's capability to heat up to a certain temperature and is therefore not an ideal parameter. We observe clear differences between the thermal designs of the two models. For the 3B our detection method requires about 175 s / 30 s (bare-metal / container) to reach 62°C to crash bare-metal or container instances. On the 3B+ we require about 145 s / 250 s to reach 62°C to crash bare-metal or container instances. Interestingly, container instances crash on the 3B earlier than bare-metal instances. We assume the computing requirements from the container environment and the thermal design work in favor of the detection method.
These histograms assist a cloud user to narrow down the exact parameters for tripping a processor into the critical region and successfully crash instances. In order to achieve a simultaneous crash of multiple instances, the cloud user has to adjust the timing of the detection method's deployment. The broad distribution of the run-time in the histograms suggests that the program used in the detection method for putting the processor under load needs to synchronize with other instances to throttle the load in order to achieve a simultaneous crash.

VI. DISCUSSION
From our evaluation we conclude that the detection method is best used in combination with other processes such as in STRESS-NG. The user even has the option to scale the number of threads in the detection method to adjust the crash time of an instance as well as the injection rate. A simple CPUbound program like the multiplication benchmark turns out to be ideal for injecting faults in an undervolted setup. The advantage of such a simple CPU-bound program is that it is unlikely to inject faults during its own execution and can run until a kernel panic while raising heat dissipation. In terms of energy efficiency we observed that by undervolting the cloud provider can save on average 5 % and up to 37 % for specific workloads on ARM processors. RA1: as shown by our extensive experimental evaluation, in order to pull off a stealthy undervolting strategy, a malicious cloud provider must exchange any firmware configuration to undervolt the hardware and intercept any voltage requests coming from users.
RA2: a cloud user can uncover such an undervolting strategy by running a simple CPU-bound benchmark until enough processes have failed to render the cloud instance unavailable. The drawback of this detection method is that it is nonselective and cloud instances can fail either soon or late.

VII. RELATED WORK
Undervolting the supply voltage for energy savings has been explored on CPUs for ARM [12,47], x86 processors [17,48] the Itanium micro-architecture [49], and for POWER-7 processors [37]. This experimental undervolting approach has been extended to GPUs [50] and FPGAs [41] as well. On the CPU side, frameworks to automate and optimize the process of undervolting have been developed [15,12]. Recently, AMD has announced an undervolting product/framework for their most recent Ryzen 5000 CPUs [51]. In [52] the authors discuss the trade-off between the reduced energy cost and the SLA violation penalties introduced by higher node failures of undervolted X86 and ARM nodes. In CLKSCREW [21], the undervolting capabilities of modern ARM processors is exploited to compromise system security, by targetting undervolting faults to specific hardware components to extract cryptographic keys.

VIII. CONCLUSION AND OPEN CHALLENGES
A cloud provider can obfuscate the undervolting of processors and even run workloads up to 37 % more energyefficiently. However, by undervolting its infrastructure, the cloud provider incurs a major risk. Not only does the cloud provider reduce the margin of error but also the system's stability is at stake. Cloud users can with high probability detect such situations and exploit them using a simple CPUbound benchmark. To some extent, the cloud provider can mitigate stability issues with appropriate cooling systems. However, it is questionable if the gains of undervolting the infrastructure outweigh the costs of such cooling systems.
Cloud users' options to detect an undervolted ARM instance remain limited and, as shown in this paper, essentially depend on the probability to inject faults non-selectively in processes. As our temperature-based guardband analysis and failure evaluation have shown, the higher the processor's temperature, the more likely faults can be injected into processes. Despite such a powerful cloud provider attacker model, cloud users have an exploitable weak link. Their only option for presuming a potentially undervolted instance is by increasing the processor's heat dissipation. Heat dissipation is increased by tuning the CPU frequency and load to the processor's limit. Under these thermal conditions and an undervolted setup the fault injection probability in processes is rising. Ideally cloud instances will become unavailable and violate the SLA as a result of continuously failing processes. Our detection method depends strongly on hardware and how systems such as firmware and AVS react to excessive heat dissipation. As future plans, we intend to expand this study to a more diverse set of ARM-based hardware targets, focusing in particular on current and future cloud offerings. We would also like to make our detection method more deterministic by injecting faults in processes more selectively.


extfieldsofstudy:
Mathematics
Computer Science
Physics

text:
KAM Theory Meets Statistical Learning Theory: Hamiltonian Neural Networks with Non-Zero Training Loss

Many physical phenomena are described by Hamiltonian mechanics using an energy function (the Hamiltonian). Recently, the Hamiltonian neural network, which approximates the Hamiltonian as a neural network, and its extensions have attracted much attention. This is a very powerful method, but its use in theoretical studies remains limited. In this study, by combining the statistical learning theory and Kolmogorov-Arnold-Moser (KAM) theory, we provide a theoretical analysis of the behavior of Hamiltonian neural networks when the learning error is not completely zero. A Hamiltonian neural network with non-zero errors can be considered as a perturbation from the true dynamics, and the perturbation theory of the Hamilton equation is widely known as the KAM theory. To apply the KAM theory, we provide a generalization error bound for Hamiltonian neural networks by deriving an estimate of the covering number of the gradient of the multi-layer perceptron, which is the key ingredient of the model. This error bound gives an $L^\infty$ bound on the Hamiltonian that is required in the application of the KAM theory.

Introduction
Many physical phenomena are described by energy-based theories, such as Hamiltonian mechanics (e.g., Furihata and Matsuo (2010) where u : t ∈ R → u(t) ∈ R N , H : u ∈ R N → H(u) ∈ R, and S is a skew-symmetric matrix. H represents the energy function of the system. In recent years, there has been a lot of research on predicting the corresponding physical phenomena by learning the energy function H in such equations with a neural network H NN (e.g., Chen et al. (2020); Cranmer et al. (2020); Greydanus, Dzamba, and Yosinski (2019); Matsubara, Ishikawa, and Yaguchi (2020); Zhong, Dey, and Chakraborty (2020)); however, to the best of our knowledge, theoretical analysis of such models has not been performed sufficiently, except for SympNet (Jin et al. 2020b) for the Hamilton equation, where the universal approximation theorems for discrete-time neural network models are provided.
In this paper, we focus on theories of the properties of the most fundamental model, comprising Hamiltonian neural networks (HNNs) (Greydanus, Dzamba, and Yosinski 2019) and their extensions in practical situations, where the learning error is not completely zero. In this case, the trained model can be regarded as a perturbed Hamiltonian system due to the modeling error of the energy function. In addition, S is a general skew-symmetric matrix and hence (2) can model Hamiltonian partial differential equations (Matsubara, Ishikawa, and Yaguchi 2020). In mathematical physics, perturbed Hamiltonian systems are well studied. For example, because whether the solar system will continue to exist in the future is of fundamental interest in astronomy, the stability of the solar system under perturbations is a very important issue that has been studied for a long time (e.g., Féjoz (2013); Laskar (1996)). The Kolmogorov-Arnold-Moser (KAM) theory gives an answer to questions of this type; essentially, periodic motions of such systems are stable under small perturbations. The stability of periodic motions is of particular importance in science. In addition to the stability of celestial systems, the recursive nature of physical phenomena is also of interest in physics. For example, in the famous numerical experiments using the Korteweg-De Vries (KdV) equation by Zabusky and Kruskal (1965), it was confirmed that the waveform given as the initial condition initially collapsed due to severe oscillations, and then returned to its original shape after a long time. Whether such phenomena can be reproduced by deep learning models is an important problem that greatly affects the usefulness of deep physical models.
In this paper, we give an answer to this question by combining the KAM theory and statistical learning theory. Because trained models are also perturbed Hamiltonian systems, it is expected that the periodic behaviors of the systems are preserved even if the loss function does not vanish completely provided it is sufficiently small. However, this expectation cannot be proved in a straightforward way, because the application of the KAM theory requires that the energy function be close enough to that of the true dynamics in the whole phase space. Noticing that the error of the Figure 1: Outline of our main theorem. The first main result is the generalization error of the energy function, which is proved by the statistical learning theory. By combining the first result and the KAM theorem, we prove that the quasi-periodic behaviors of target systems are preserved. energy function in the whole space is essentially the generalization error, we overcome this difficulty by combining the KAM theory with statistical learning theory. This illustrates that combinations of statistical learning theory and dynamical systems theory can lead to powerful results. Indeed, combinations of this kind may be applicable to other stability results in dynamical system contexts, for example, the stability of solitary wave solutions.
Importantly, for the neural network models to be close to the true dynamics, we need a universal approximation theorem and also a generalization error bound. In this paper, we also provide such results for HNNs.
Regarding the generalization error bound, because the derivative of a multi-layer perceptron is used in HNNs, a bound for the derivative is required. To this end, we estimated the covering number of the derivative of multi-layer perceptrons. An L ∞ bound on the error in the Hamiltonian is also provided, which is required for application of the KAM theory.
In addition, we show a universal approximation theorem for a model with the coordinate transformation This model is indispensable in practice; to apply HNNs, the data are given in the canonical coordinate system because the equation of motion is in the form of the Hamilton equation (1) only when the state variables are represented by the canonical coordinate system. However, this coordinate system depends on an unknown Lagrangian and hence the energy function. Hence, the coordinate system must be also learned from data by using, for example, neural networks.
In addition, this model also represents other energy-based physical models beyond the Hamilton equation. See the Appendix for details. The main contributions of this paper are as follows.
1. Combination of the KAM theory and statistical learning theory for HNNs with non-zero training loss to prove the existence of quasi-periodic behaviors (see Figure 1). 2. Derivation of a generalization error bound for HNNs.

Development of a universal approximation theorem
for HNNs and other energy-based physical models with coordinate transformations.

Related work
Many studies of neural network models for phenomena that can be modeled by energy-based equation (1) have been put forward. Among them, the most basic studies are neural ordinary differential equations (Chen et al. 2018) and HNNs (Greydanus, Dzamba, and Yosinski 2019). In particular, extensions of HNNs have been intensively developed. Describing them all is beyond the scope of this paper, but some examples are given here. In Toth et al. (2019) HNNs were extended to latent variable models. Other studies, such as DiPietro, Xiong, and Zhu (2020); Xiong et al. (2021);Zhong, Dey, and Chakraborty (2020), focused on the symplectic structure of the Hamilton equation. For Noether's theorem, which is a fundamental theorem in classical mechanics, several studies (Bharadwaj, Li, and Demanet 2020;Bondesan and Lamacraft 2019;Finzi et al. 2020) developed methods related to symmetry and conservation laws. In addition, a discrete-time model that preserves the energy behaviors was constructed in Matsubara, Ishikawa, and Yaguchi (2020). In Galioto and Gorodetsky (2020), HNNs were combined with a Bayesian approach.
Methods applied to the framework of classical mechanics other than Hamiltonian mechanics include those in Cranmer et al. (2020); Desai and Roberts (2020); Saemundsson et al. (2019), which are methods for Lagrangian formalism. In Jin, Lin, and Li (2020), reinforcement learning was applied to the variational principle. A simplified model formed by introducing constraints was proposed in Finzi, Wang, and Wilson (2020). In Jin et al. (2020a), HNNs were extended to the Poisson system, which is a wider class of mechanical equations. There are also a number of proposals that integrate them with more advanced deep learning techniques, namely, graph networks (Sanchez-Gonzalez et al. 2019), recurrent neural networks (Chen et al. 2020), and normalizing flows . As an application-oriented approach, Feng et al. (2020) designed a microscopic model for structural analysis.
However, as far as the authors know, there is no theoretical research other than the universal approximation theorems for Hamiltonian mechanics in SympNet (Jin et al. 2020b), in which a certain kind of neural network is shown to have universal approximation properties for symplectic maps. The difference between their results and ours is that (1) we analyze the behaviors of a HNN with non-zero training loss by a combination of the KAM theory and statistical learning theory, (2) we provide a generalization error bound for NHHs, and (3) the universal approximation theorems in Jin et al. (2020b) are for discrete-time models, while ours are for continuous-time models.
Meanwhile, as an existing energy-based model, Hopfield neural network is known. Both Hopfield neural network and Hamiltonian network are derived from energy-based theories, and their dynamics are described by (1). Hamiltonian neural network is associated with a skew-symmetric matrix S and is a model of an energy-preserving, continuous-time, and deterministic physics phenomenon. Its output is the time-series of the state. Hopfield network is associated with a negative definite matrix S and exhibits a dynamics which is often energy-dissipating, discrete-time, and stochastic. It is a machine learning tool rather than a physical model, and its equilibrium point is treated as its output. Because their outputs are different, their theoretical properties should be discussed separately.

Brief introduction to Hamiltonian systems and the KAM theory
We briefly introduce some properties of Hamiltonian systems. Theorem 1 (Darboux). By an appropriate coordinate transformation, the matrix S is transformed into the normal form Definition 1. The function ω : is called the symplectic form. Using the symplectic form associates a vector field X F with each function F : R N → R by requiring For two functions F, G, the following operation is called the Poisson bracket: (4) Definition 2. A Hamiltonian system for which the state variable is N = 2M dimensional is integrable in the sense of Liouville if this Hamiltonian system has the first integrals (i.e., conserved quantities) F 1 , F 2 , . . . , F M with ∇F 1 (u), ∇F 2 (u), . . . , ∇F M (u) independent at each u and for all i, j: {F i , F j } = 0. For integrable systems, Theorem 2 is known. Theorem 2 (Liouville-Arnold). Suppose that for an integrable Hamiltonian system, constants c 1 , . . . , c M exist such is connected and compact. Then, there exists a neighborhood N comprising K, U ⊂ R n and a coordinate transform φ : (θ, J) ∈ T n × U → φ(θ, J) ∈ N (5) such that the transformed system is the Hamilton equation of which Hamiltonian H • φ depends only on J.
The variables J and θ are called action-angle variables. Theorems 1 and 2 roughly mean that integrable Hamiltonian systems can be written in the following form: Further, becauseH = H • φ depends on I only, it holds that d dt This shows that J is constant, and hence θ moves on the torus at a constant velocity. Because the velocities are typically not co-related to each other, the dynamics are "quasiperiodic." See, for example, Scott Dumas (2014) for more details.
As seen above, integrable Hamiltonian systems are quasiperiodic. Note that general Hamiltonian systems are not necessarily quasi-periodic and neither are HNNs. However, for the HNNs that are trained to model integrable systems, the quasi-periodic behaviors are preferably maintained. When the modeling error is sufficiently small, this is considered as a perturbation problem. The perturbation theory of Hamiltonian systems has been investigated from various perspectives. For example, perturbed integrable Hamiltonian systems are in general no longer integrable; hence, approximation of integrable Hamiltonian systems by integrable neural network models appears to be difficult. Fortunately, however, the KAM theory shows that even though the perturbed system is not integrable, it maintains the quasi-periodic behaviors described above under certain conditions.
The KAM theorem has many variants under various conditions. The following variant (Scott Dumas 2014) is typical: Theorem 3 (KAM Theorem). Let θ and J be the actionangle variables for a C ∞ integrable Hamiltonian H 0 : for the perturbed system H(θ, J) = H 0 (J) + εF (θ, J, ε) by F ∈ C ∞ , there exists ε 0 such that if εF < ε 0 , there exists a set of M -dimensional tori that are invariant under the perturbed flow. For each invariant torus, the flow of the perturbed system H is quasi-periodic. In addition, the set of invariant tori is large in the sense that its measure becomes full as ε → 0.
Remark 1. The last sentence -the set of invariant tori is large in the sense that its measure becomes full as ε → 0corresponds to the non-existence of so-called resonance. If the perturbation added to the system is in resonance with the original system, the perturbation may grow rapidly and the behavior of the system may change significantly. This statement assures that for small perturbations, such a situation almost never occurs.
Remark 2. It may be difficult to check whether the target system is integrable by using given data. One possibility is application of the Koopman operator, which makes it possible to find the conserved quantities that the given data may admit. If a sufficient number of conserved quantities exist, it is highly likely that the target system is integrable.

Universal Approximation Properties of HNNs
For HNNs to be close to the true dynamics, a universal approximation theorem and a generalization error analysis are needed. First, we show universal approximation theorems. We first define some notation to describe the theorem. C m (X) with the topology of the Sobolev space W p,m (X) is denoted by S m p (X), where W p,m (X) is a space of functions that admit weak derivatives up to the mth order that bounds L p -norms. Hence, S m p (X) is the space of functions in W p,m (X) with (usual) derivatives; for details on the Sobolev theory, see Adams and Fournier (2003). L p -norms of functions are denoted by · L p , and those of vectors by · p .

Universal approximation theorem for HNNs
The following theorem shows the universal approximation property of HNNs.
Theorem 4. Let H : R N → R be an energy function with the equation where u : t ∈ R → u(t) ∈ R N and S is a skew-symmetric N × N matrix. Suppose that the phase space K of this system is compact and the right-hand side S∂H/∂u is Lipschitz continuous. If the activation function σ = 0 belongs to S 1 2 (R), then for any ε > 0, there exists a neural network H NN for which holds. In addition, if the energy function is C ∞ , the function can be approximated by a C ∞ neural network provided that the activation function is sufficiently smooth. An outline of the proof is as follows. From the existence theorem of a solution to ordinary differential equations, the right-hand side of the equation must be Lipschitz continuous to guarantee the existence of a solution. Therefore, the differential of the H is Lipschitz continuous, which implies the smoothness of H. Then, the approximation property is obtained from the universal approximation theorem for smooth functions by Hornik, Stinchcombe, and White (1990). For the detailed proof, see the Appendix.
HNNs with a coordinate transformation The practical use of HNNs is hampered by the fact that the state variables must be represented by a specific coordinate, such as the generalized momentum; however, the derivation of the generalized momentum requires the energy function, which is unknown. For example, the double pendulum in Figure  2 exhibits the dynamics shown in Figure 3. These are predicted by the models that are trained from the data of the state variables and their derivatives, not those of the generalized momenta. HNNs failed to solve such problems because the data were not given in the canonical coordinate system. See the Appendix for details. Based on this, we here propose a model with a coordinate transformation, such as the transformations that appear, for example, in Rana et al. (2020); Jin et al. (2020a).
Suppose that, although the given data point x(t) is not represented by the canonical coordinate system, the data point x(t) can be transformed into the canonical coordinate system by an unknown transformation We show that model (7) admits the same energetic property as the original equation and also the universal approximation property. Theorem 5. The model (7) admits the energy conservation law in the sense that dH NN /dt = 0.
Theorem 6. Let H : R N → R be an energy function for the equation Suppose that the phase space K of this system is compact and the right-hand side ∂H/∂u is Lipschitz continuous. Suppose also that u is a C 1diffeomorphism. If the functions σ = 0 and ρ = 0 belong to S 1 2 (R), then for any ε > 0, there exist neural networks H NN with the activation functions σ and u NN with ρ for which For proofs, see Appendix.

Generalization Error Analysis of HNNs
Next, we derive a generalization error bound for the standard HNN (2) by employing a technique from statistical learning theory. More precisely, we adjust the technique so that an estimate on the energy gradient can be obtained. Remark 3. Although the standard HNN without the coordinate transformations is considered below, the results can be extended to the general energy-based model with the coordinate transformations if the matrix (∂u/∂x) −1 is bounded.
Lemma 1. Let X and Y be arbitrary spaces, F ⊂ {f : X → Y } be a hypotheses class, and L : Then, for any δ > 0 and any probability measure P , we obtain, with a probability of at least (1 − δ) with respect to the repeated sampling of P n -distributed training data, the following: The Rademacher complexity is known to be bounded by using the covering number.
Thus, if the covering number is estimated for a HNN, the bound on the generalization error is obtained. To this end, we suppose that the model is trained by minimizing the pnorm of the error in the right-hand side of the model. More precisely, for the hypothesis h : u j → S ∂HNN(uj ) ∂u , we consider the loss function where u i are training data. We denote the Lipschitz constant of the loss function associated with the above by ρ p . Of course, p = 2 is typically used; however, we show below that p > 2M is useful to obtain an L ∞ bound on the Hamiltonian.
Remark 4. The training can be performed also by using the symplectic gradient: In that case, the results will be slightly modified using the norm of S; however, we omit this for simplicity.
A bound of the covering number is derived as follows.
Theorem 7. Suppose that the hypotheses class F consists of multi-layer perceptrons f NN that have ρ σj -Lispchitz activation functions σ j (j = 1, . . . , n l ), for which the derivatives are ρ j -Lipschitz continuous and bounded by sup |σ j | < c σj . Suppose also that the matrices A j (j = 1, . . . , n l + 1) in the linear layers in the perceptrons have the bounded norm |A j | < c Aj : with the ρ p -Lipschitz continuous loss function L. In addition, suppose that the phase space is compact so that the data u i (i = 1, . . . , n) are in a bounded set with the bound u i < c u . Then, the covering number of G is estimated by To prove this theorem, we use the following lemmas, which are typically used to estimate the covering numbers (Shalev-Shwartz and Ben-David 2014).
Lemma 4. Suppose that functions φ i : R → R, i = 1, 2, . . . , n are ρ-Lipschitz continuous. Then, for V ⊂ V n , Proof of Theorem 7. To simplify the discussion, we will estimate the covering number of the following perceptron Because the proof for general cases is exactly the same, we need to estimate the covering number of the gradient of f NN , which is written as where Dσ 2 and Dσ 1 are Jacobian matrices. These Jacobian matrices are evaluated at u = A 2 σ 1 (A 1 u + b 1 ) + b 2 and A 1 u + b 1 , respectively. We first estimate the covering number associated with Dσ 2 . Dσ 2 has the same architecture as a multi-layer perceptron, except that the last activation function is replaced by the differential σ 2 of σ 2 : Assuming that the input data are in the ball B cu with radius c u , we obtain Then, because the norms of A 1 , A 2 are bounded by c A1 , c A2 , matrix multiplications by these matrices are c A1 -and c A2 -Lipschitz continuous, respectively. In addition, σ 1 is ρ 1 -Lipschitz and σ 2 is ρ 2 -Lipschitz continuous. Therefore, the covering number associated with Dσ 2 is estimated by Finally, because σ 1 is assumed to be bounded by c σ1 , the norms of the matrices other than Dσ 2 in ∇f NN are bounded as follows: Because the loss function is assumed to be ρ p -Lipschitz continuous, we obtain the estimate L ∞ estimate on the error in the Hamiltonian The generalization error analysis in Theorem 7 shows that, at a certain probability, the expectation of the loss function can be bounded. If this bound certainty holds and if the training is performed by minimizing the p-norm with p > 2M , we can derive an L ∞ estimate on the Hamiltonian for the standard HNN (2) by applying the Poincaré inequality and the Sobolev inequality under Assumption 1.
Assumption 1 There exists a density f P for measure P with inf f P > 0. Remark 5. The condition p > 2M is not required in practice because of the well-known equivalence of the norms in finite dimensional spaces; for example, if the standard 2norm is small enough, then the p-norm is also small. However, when the dimension 2M is large, the 2-norm needs to be very small to bound the p-norm because the constant in the inequality used to bound the p-norm depends on the dimension. Therefore, it is preferable to minimize the p-norm in such cases. Theorem 8 (Poincaré inequality). Suppose that 1 ≤ p ≤ ∞ and Ω ⊂ R 2M is bounded. Then there exists a constant c p such that, for any H ∈ S 1 p (Ω), The constant c p is called the Poincaré constant. Theorem 9 (Sobolev inequalities). There exist constants c 1 , c 2 such that, if lp > 2M , By using these inequalities along with the invariance of the Hamilton equation under the constant shift of the energy function, we obtain an error bound on the Hamiltonian. From the above estimate, we obtain By using the density f P for the measure P , we obtain which gives us We note that the left-hand side is the Sobolev norm of the error in W p,l ; then, under the assumption that p > 2M , we can use the Sobolev inequality to obtain which ensure that H and H NN are close in terms of the function values.

KAM Theory for HNNs
The universal approximation property shown in the previous sections guarantees that the value of MSE can be made arbitrarily small by training; however, in actual training, a finite error remains. In this section, as an application of the generalization bound, we apply the KAM theory to theoretically investigate the trained standard HNN model (2) in such cases by assuming that the target system is integrable.
We make a few assumptions that are needed for the application of the KAM theory. Assumption 2 The dimension of the phase space is assumed to be 2M with M ≥ 2. Assumption 3 The target system is an integrable Hamiltonian system with the conserved quantities F 1 , . . . , F M . The series c 1 , . . . , c M exists such that K = ∩ M i=1 F −1 i (c i ) is connected and compact.
Under the above assumptions, from the Liouville-Arnold theorem there exist a neighborhood N of K, U ⊂ R n and a coordinate transform such that the transformed system is the Hamilton equation.
Following the usual setting of the KAM theorem, we consider the target system and the Hamiltonian equation in the transformed coordinate T n × U. Assumption 4 The Hamiltonian H : T n ×U → R of the target system is C ∞ and non-degenerate. The activation functions of the HNNs used are in C ∞ . Assumption 5 From the generalization error analysis in the previous section, we have essentially shown that if p > 2M , with at least probability 1 − δ, it holds that sup |H(u) − H NN (u)| < c 1 L train + c 2 R n + c 3 ln 1 δ n with constants c 1 , c 2 , and c 3 , where R n is a bound on the Rademacher complexity. We assume that the training was performed with p > 2M and the above statement certainly holds.
Using these assumptions, we obtain Theorem 10. Theorem 10. Let the threshold of the KAM theorem be ε 0 and δ be Under the above assumptions, with a probability of at least (1 − δ), a set of invariant tori exists for the trained model H NN .
Proof. It is confirmed by a straightforward calculation that if δ is given as described above, it holds that sup |H(u) − H NN (u)| < ε 0 , and hence the assumption of the KAM theorem is satisfied.
Remark 6. As mentioned in Remark 1, the KAM theorem also shows that the invariant tori become larger when the perturbation becomes smaller. Hence, if the generalization error is small enough, the size of the tori is expected to be large.
Note that general Hamiltonian systems, and hence general HNNs, are not quasi-periodic. Therefore, a model that approximates a quasi-periodic Hamilton equation may be (in some sense) approximately quasi-periodic, but it is not necessarily strictly quasi-periodic. This theorem states that the trained model can be strictly quasi-periodic even if the training loss does not completely vanish.
Numerical Example: Learning the Zabusky and Kruskal Experiment As a numerical experiment, we trained a HNN 1 so that the dynamics of the KdV equation is learned by using the data from the experiment by Zabusky and Kruskal (1965), in which a nontrivial recurrence of initial states is reported.
The KdV equation is derived from the following energy function H: In fact, under the periodic boundary condition, the variational derivative is Then, the time evolution is expressed as a Hamiltonian equation: For spatial discretization, we used the forward and backward difference operators, respectively. The central difference operator D is their mean, specifically D = 1 2 (D f +D b ) and that for the second derivative is D 2 = D f D b = D b D f . Using these difference operators, the equation is semi-discretized as Following Zabusky and Kruskal (1965), we set the parameters to α = −1.0 and β = −0.022 2 , set the width of phase space to 2.0, and used the initial condition u(0, x) to u(0, x) = cos(xπ). We discretized the system with the spatial and temporal mesh sizes of ∆x = 0.1 and ∆t = 0.01. We obtained an orbit for 200 time steps from the initial condition using the fifth-order Dormand-Prince method with the absolute and relative tolerances of 10 −10 and 10 −8 .
We performed the experiments on an NVIDIA TITAN V with double precision. We employed a three-layered convolutional neural network with kernel sizes of 3, 1, and 1. The number of hidden channels was 200, the number of output channels was 1, the activation function was the tanh function, and each weight parameter was initialized as a random orthogonal matrix. We summed up the output in the spatial direction and obtained the global energy. We used the whole orbit at every iteration, and minimized the mean squared error of the time derivative as the loss function using the Adam optimizer with a learning rate of 10 −3 for 10,000 iterations; the error reached a maximum of 1.37 × 10 −3 . Given the true dynamics u, the absolute error between the energy function H and the neural network H NN was 1.31 × 10 −4 on average and 2.51 × 10 −4 at most.
Using the true model and the trained neural network, we also obtained orbits for 1100 time steps from the same initial condition, as shown in the second and third panels of Fig. 4, respectively. In the top panels, blue and orange lines denote the true state u and the state predicted by the trained neural network u NN at t =0.0, 2.0, 4.8, and 9.8. The bottom panel shows the energy function H given the predicted states u and u NN . Due to the non-zero training error, more waves incur a larger error. Nonetheless, at around t = 9.8, the true model and learned neural network reproduce sin waves, which are given as the initial condition, and the energy error is restored to zero; they exhibit quasi-periodic behaviors.

Concluding Remarks
We analyzed the behavior of HNNs with non-zero learning errors by combining the KAM theory and statistical machine learning. We investigated the approximation properties of deep energy-based models, including HNNs. More precisely, we proved the persistence of the quasi-periodic behaviors of integrable Hamiltonian systems with a high probability even when the loss function is not perfectly zero. Further, we provided a generalization error bound and universal approximation theorems for HNNs to ensure that the loss function can be sufficiently small for application of the KAM theorem. Meanwhile, in the recent research on this type of model, numerically integrated gradients are often used for training. Similar results should be obtained for such cases; however, rigorous discussion is needed.

Energy-Based Physical Models
Equation (1) [and its coordinate transformation (3)] and the related model (2) [and its coordinate transformation (7)] describe not only Hamiltonian systems but also various types of other phenomena. In fact, as shown in the theorem below, models similar to (1), where G is a skew-symmetric or negative-semidefinite matrix, have not only the energy conservation property but also the energy-dissipation property, depending on the matrix G. The same result for the models similar to (3) is obtained in a straightforward way. Theorem 11. Equation (12) admits the energy conservation law, if G is skew-symmetric, and it admits the energy dissipation law, Proof of Theorem 11. We use the chain rule to obtain Hence, when the matrix G is skew-symmetric, the energy conservation law holds: dH dt = 0 because v Gv = 0 holds for all vectors v when G is skewsymmetric. When the matrix G is negative semidefinite, the energy function is monotonically non-increasing: As mentioned above, equations of this form are used not only in Hamiltonian mechanics but also in many fields of mathematical modeling, for example, in the Landau theory and the phase-field method. Phenomena such as phase separation, crystal growth, and crack propagation are modeled using these theories (e.g., Caginalp (1986); Miehe, Hofacker, and Welschinger (2010); Steinbach (2009) ;Wheeler, Murray, and Schaefer (1993)). The above model also includes the semi-discretized partial differential equations. For example, the KdV equation, which is a model of shallowwater waves, where α, β are parameters. This equation can be written as where H(u, ∂u/∂x) is the Hamiltonian and δH/δu is the variational derivative of H, which is defined by where u x denotes ∂u/∂x. Equation (15) is an example of a Hamiltonian partial differential equation, in which ∂/∂x in front of δH/δu plays the role of the skew-symmetric matrix G. In fact, if ∂/∂x is discretized by using a central difference operator, ∂ ∂x Because the difference matrix D is skew-symmetric, the semi-discretized equation has the form of equation (7) with a skew-symmetric matrix G. Similarly, an equation with the form is semi-discretized to equation (7) with a negativesemidefinite G. In fact, if the ∂ 2 /∂x 2 in front of δH/δu is discretized, which is known to be negative semidefinite (Furihata and Matsuo 2010). The universal approximatio-genn properties and the generalization error analysis are also applied to this model in a straightforward way under the assumption that S is nondegenerate. In fact, the properties of the matrix S used in the theoretical analyses are (1) S is non-degenerate and (2) the norm of S is bounded. If S is a constant matrix, (2) is automatically satisfied and hence (1) is the only assumption of the matrix S. In a part of the theoretical analysis, we consider the errors in the gradient ∇H of the energy function. However, typically, the loss function of S∇H is minimized in the training process, and if S is degenerate, errors in ∇H may not be small even when those in S∇H are small. Hence, S should be non-degenerate.

Proofs of Other Theorems
Proof of Theorem 5 Proof. By substituting the equation, we obtain because S is skew-symmetric and hence for any vector v, v Sv = 0.
Proof of Theorem 6 We prove the following theorem, which is a generalization of Theorem 6.
Theorem 14. Let H : R N → R be an energy function for the equation and S is an N × N non-degenerate matrix. Suppose that the phase space K of this system is compact and the right-hand side ∂H/∂u is Lipschitz continuous. Suppose also that u is a C 1 -diffeomorphism. If the functions σ = 0 and ρ = 0 belong to S 1 2 (R), then for any ε > 0 there exist neural networks H NN with the activation functions σ and u NN with ρ for which Proof. We need to prove the approximation property for (∂u/∂x) −1 . From the assumption that ρ = 0 is in S 1 2 (R), there exists a function u NN that approximates ∂u/∂x. Because the determinant function of matrices is continuous, it is deduced that det ∂u NN /∂x = 0 and hence (∂u NN /∂x) −1 exists. Because the matrix inverse is also continuous, (∂u NN /∂x) −1 is also approximated by u NN .

Details on Numerical Experiments
Energy-based physical model with the coordinate transformations In this study, we considered a model with coordinate transformations: For a better understanding of the significance of the theoretical results presented herein, we performed numerical tests using this model.
Because the generalized momenta are difficult to observe, we suppose that the values of the state variables θ 1 , θ 2 and their derivatives φ 1 , φ 2 are given as data. In such a situation, HNNs (Greydanus, Dzamba, and Yosinski 2019) are not applicable, because this model can be used only when p 1 and p 2 are generalized momenta; in the case considered here, they are supposed to be unknown.
To confirm this, we tested the model where v 1 , v 2 are not the generalized momenta but the velocities v 1 =q 1 , v 2 =q 2 . The experiments were implemented using Python 3.8.5 with the packages PyTorch 1.7.1, NumPy 1.19.5, SciPy 1.6.0, autograd 1.3, and torchdiffeq 0.2.1. For data, we used numerical solutions to the equation of motion with the parameters l 1 = l 2 = 1.0, m 1 = 1, m 2 = 2, and g = 9.8 solved by SciPy odeint with 100 initial conditions randomly generated from the standard normal distribution. We numerically integrated each orbit on the time interval [0, 5], in which the computed values are evaluated at 100 points with an identical sampling rate. Then, the target data d dt are obtained by substituting the computed state variables into the right-hand side of the equation. The experiments were performed on an NVIDIA GeForce 2080 Ti. The energy function was modeled by using a two-layered fully connected neural network with 50 hidden units and tanh as the activation function. We minimized the mean squared error of the time derivative as the loss function using the Adam optimizer with a learning rate of 0.001 and a batch-size of 200 for 10,000 iterations.
Remark 7. For the coordinate transformations in the model, invertible neural networks should be used; however, we used a simple neural network here because this model is assumed in the theorem in the main text. In the theorem, we needed neural networks that can approximate given functions and their derivatives. Although the universal approximation property of invertible neural networks was recently proved (Teshima et al. 2020), this theorem shows the universal approximation property of the function values only, not of the derivatives, which are needed for computation of the Jacobi matrix in our model.
Examples of the predicted orbits are shown in Figure 3. The training losses were 13.6 for the HNN and 0.280 for the model with coordinate transformations. As shown in this figure, the naive model failed to capture the dynamics correctly. This is because the dynamics of θ 1 , θ 2 , φ 1 , andφ 2 cannot be described by equation (17). This illustrates that, to model physical phenomena by using a model of the form the choice of the coordinate system is important.
Mass-spring system. Due to the well-known chaotic behaviors of the double pendulum, the results, in particular the values of the losses, of the previous experiments are to a certain extent unstable, except for the Hamilton neural networks always failing. Therefore, second, we investigated the models in more detail using the simple mass-spring system depicted in Figure 5. The two mass points m 1 and m 2 are connected by springs, which respectively have the spring Figure 5: Target mass-spring system.
constants k 1 and k 2 and natural lengths l 1 and l 2 . This system is a Hamiltonian system with the energy function where q 1 , q 2 are the positions of the mass points and p 1 , p 2 are the momenta, which are defined by p 1 = m 1 v 1 , p 2 = m 2 v 2 , v 1 = dp 1 /dt, andv 2 = dp 2 /dt. Suppose that we do not know the exact values of m 1 and m 2 and only the positions q 1 and q 2 and their derivatives can be observed.
Although m 1 and m 2 may be estimated from the data, for evaluation of the models, we tried to model the dynamics only using q 1 , q 2 , and their derivatives.
We examined the naive model, the model with the coordinate transformation, and the neural ordinary differential equation (ODE, Chen et al. (2018) The experimental conditions were almost the same as those in the previous experiment, except for the batch size, which we set to 100. For data, we used numerical solutions to d dt which is equivalent to equation (18).
Examples of the predicted orbits are shown in Figure 6. While the model with coordinate transformations produced an orbit that was nearly identical to the ground truth, the naive model failed to predict the states. In fact, it should be impossible to rewrite equation (19) as (17) with a certain energy function; for example, when the system has just one mass point and the equation of motion is given by d dt , this can be transformed into a Hamiltonian system d dt with k 1 = k 1 /m 1 , m 1 and the mass m 1 = 1. Hence, for this system, the HNNs are applicable without knowledge of m 1 . However, for system (19), such a transformation cannot be applied. Meanwhile, in the model with the coordinate transformation, a coordinate transformation that makes the equation Hamiltonian is explored. The result obtained by the neural ODE is better than that by the naive model; however, the prediction error became larger as time increased. This is due to the non-existence of the energy conservation law for the neural ODE. In fact, the value of the energy function of the neural ODE model was increasing, as shown in Figure 7.
The losses for the above models are listed in Table 1 for 12 experiments for each model. The values of the loss functions of the neural ODE were very small and those of the model with coordinate transformations were not stable. Actually, the histories of the loss functions for this model were not monotonic at all. This is due to the non-uniqueness of the model with coordinate transformations; if a model is fit to the given data quite well, then so are its canonical transformations. Due to this feature of the model, after the loss function decreased, it sometimes increased significantly in the search for different coordinate transformations. The failure in the learning process can be detected as the large train-ing loss, and if the learning process is successful, the test loss is competitive with that of the neural ODEs. Moreover, the neural ODEs exhibit energy drift, as seen above. Therefore, the model with coordinate transformations is the most suitable for a long-term physics simulation in an unknown coordinate system.