1. Bibliographic Information

1.1. Title

Extended Friction Models for the Physics Simulation of Servo Actuators

1.2. Authors

• Marc Duclusaud

• Grégoire Passault

• Vincent Padois

• Olivier Ly

  The affiliations of Marc Duclusaud, Grégoire Passault, and Olivier Ly are not explicitly stated in the provided text, but they appear to be associated with the same institution (indicated by '1' next to their names). Vincent Padois is associated with a different institution (indicated by '2'). Given the topic of robotic systems, control algorithms, and physics simulation, their research backgrounds likely span robotics, control engineering, and applied physics.

1.3. Journal/Conference

The paper is published on arXiv, a preprint server. While it is a widely used platform for disseminating early research, it is not a peer-reviewed journal or conference proceeding. Therefore, it has not undergone formal peer review, which is a crucial step for validating research quality and rigor in established academic venues.

1.4. Publication Year

2024

1.5. Abstract

This paper addresses the challenge of accurately simulating the complex friction dynamics of servo actuators in robotic systems. It highlights that current state-of-the-art servo actuator models often fail to capture these dynamics, which limits the transferability of simulated behaviors to real-world applications, particularly for Reinforcement Learning (RL) based control. The authors propose extended friction models to enhance simulation accuracy. Their methodology involves a comprehensive analysis of various friction models, a method for identifying model parameters using recorded trajectories from a pendulum test bench, and demonstrating how these models can be integrated into physics engines. The proposed models are validated on four distinct servo actuators and tested on 2R manipulators, showing significant improvements in accuracy compared to the standard Coulomb-Viscous model. The key conclusion is the importance of incorporating advanced friction effects to achieve more realistic and reliable robotic simulations.

1.6. Original Source Link

• Original Source Link: https://arxiv.org/abs/2410.08650v1
• PDF Link: https://arxiv.org/pdf/2410.08650v1.pdf
• Publication Status: This paper is a preprint published on arXiv.

2. Executive Summary

2.1. Background & Motivation

The core problem the paper aims to solve is the inaccuracy of physics simulations for servo actuators in robotic systems, specifically concerning their friction dynamics. This inaccuracy poses a significant challenge for the development and validation of control algorithms, especially in Reinforcement Learning (RL) where extensive simulation is used to train robot control policies.

The problem is important because the transferability of learned policies from simulation to real-world robots is severely limited if the simulation does not accurately reflect reality. Current state-of-the-art servo actuator models, often relying on simplified Coulomb-Viscous friction models (as found in popular physics simulators like MuJoCo and Isaac Gym), fail to capture the complex, non-linear friction effects observed in real servo actuators (e.g., Stribeck effect, load-dependence, directional efficiency, quadratic friction). This discrepancy leads to a simulation-to-reality gap, where policies learned in simulation perform poorly when deployed on physical robots.

The paper's entry point and innovative idea revolve around proposing and validating extended friction models that go beyond the traditional Coulomb-Viscous model. By incorporating more advanced friction phenomena, the authors aim to bridge the sim-to-real gap and improve the realism and reliability of robotic simulations, particularly for RL applications where actuators are often controlled with low gains, making friction dynamics more prominent.

2.2. Main Contributions / Findings

The paper's primary contributions are:

• Friction Model Analysis: A comprehensive analysis of various friction models (Stribeck, Load-Dependent, Stribeck Load-Dependent, Directional, Quadratic) and their effects on servo actuator dynamics.

• Parameter Identification Method: A systematic method to identify the parameters for these extended friction models using recorded trajectories from a pendulum test bench. This involves an optimization process using the CMA-ES genetic algorithm to minimize Mean Absolute Error (MAE) between simulated and real trajectories.

• Simulation Integration: A detailed explanation of how to integrate these extended friction models into physics engines for simulating servo actuators in more complex robotic systems.

  The key conclusions and findings reached by the paper are:

• The extended friction models significantly improve the accuracy of servo actuator simulations compared to the standard Coulomb-Viscous model. For instance, the MAE was reduced by factors ranging from 1.51 to 2.93 on different servo actuators during identification.

• The Stribeck load-dependent model ($\mathcal{M}_4$) performed best for Dynamixel servo actuators, while the Quadratic model ($\mathcal{M}_6$) was most accurate for the eRob80:100 actuator. These models consistently outperformed Coulomb-Viscous on 2R manipulators across various trajectories and control gains, with MAE values more than twice as low.

• The findings highlight that accounting for advanced friction effects (such as Stribeck, load-dependence, directional efficiency, and quadratic friction) is crucial for enhancing the realism and reliability of robotic simulations. This is particularly important for RL applications where the accuracy of the simulated environment directly impacts the transferability of learned policies to the real world.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

• Physical Simulation: In robotics, physical simulation (or physics engine) refers to software that mimics the physical behavior of robots and their environment. It calculates how forces, gravity, friction, and collisions affect objects over time. This allows for safe and rapid testing of control algorithms before deploying them on expensive or fragile hardware. Examples include MuJoCo and Isaac Gym.

• Servo Actuator: A servo actuator is a device (typically a motor combined with a gearbox and an electronic controller) that provides precise control over angular or linear position, velocity, and torque. They are ubiquitous in robotics, enabling movement in robot joints.

• Friction: Friction is a force that opposes motion or attempted motion between surfaces in contact. In servo actuators, friction primarily occurs in the gearbox and bearings. It consumes energy, generates heat, and introduces non-linearities into the system's dynamics, making accurate modeling crucial for simulation. The paper distinguishes between static friction (opposing initial motion) and viscous friction (opposing ongoing motion).

• Reinforcement Learning (RL): A subfield of machine learning where an agent learns to make decisions by performing actions in an environment to maximize a cumulative reward. In robotics, RL is used to train robots to perform complex tasks by allowing them to learn control policies through trial and error, often heavily relying on simulations.

• Sim-to-Real Transfer: The process of deploying a control policy or behavior learned in a simulation environment to a real-world robotic system. A significant challenge in sim-to-real transfer is the reality gap, which arises from discrepancies between the simulated and real dynamics, often including inaccurate friction models.

• Pendulum Test Bench: An experimental setup where a servo actuator is attached to a rigid link with a mass at its end, forming a pendulum. This setup simplifies the system dynamics, making it easier to identify actuator and friction parameters by recording the pendulum's motion under various conditions.

• Drive/Backdrive Diagram: A graphical representation used to characterize the transmission efficiency and friction of an actuator or gearbox. It plots the external torque ($\tau_e$) against the motor torque ($\tau_m$). The static area shows where the system remains stationary, bounded by the drive torque (maximum external torque the motor can overcome) and backdrive torque (minimum external torque required to move the motor backward).

• PID Controller: A Proportional-Integral-Derivative controller is a widely used feedback control loop mechanism. It calculates an error value as the difference between a desired setpoint (e.g., target position $\theta^d$) and a measured process variable (e.g., current position $\theta$). The controller attempts to minimize the error by adjusting the process control inputs (e.g., voltage or current to the motor) based on proportional, integral, and derivative terms.

  • Proportional (P) Term: Responds to the current error. A larger error leads to a larger corrective action.
  • Integral (I) Term: Accounts for past errors, helping to eliminate steady-state errors.
  • Derivative (D) Term: Predicts future errors based on the rate of change of the current error, providing a damping effect. In the context of servo actuators, PID gains ($K_p, K_i, K_d$) are often tuned to achieve desired control performance.

3.2. Previous Works

The paper references several prior studies that form the foundation and context for its work:

• General Reinforcement Learning in Robotics [1, 2, 3]:

  • Kober et al. [1]: A survey on Reinforcement Learning in Robotics, highlighting its capability to implement robust, versatile, and adaptive behaviors. This establishes the context for why accurate simulation is crucial for RL.
  • Todorov et al. [2] (MuJoCo): Introduces MuJoCo, a physics engine widely used for model-based control. The paper points out that MuJoCo typically uses the simplified Coulomb-Viscous friction model, which is a limitation addressed by the current work.
  • Makoviychuk et al. [3] (Isaac Gym): Introduces Isaac Gym, another high-performance GPU-based physics simulation platform for robot learning. Similar to MuJoCo, it also uses simplified friction models, reinforcing the need for more accurate models.
  • Background: These works demonstrate the growing reliance on simulation for RL in robotics, underscoring the importance of simulation accuracy for sim-to-real transfer.

• Actuator Dynamics Identification for Sim-to-Real Transfer [4, 5, 6]:

  • Zhu et al. [4]: A survey on sim-to-real transfer techniques for bioinspired robots, emphasizing actuator dynamics identification as a common approach to ensure consistency between simulation and reality.
  • Andrychowicz et al. [5]: Describes learning dexterous in-hand manipulation, likely involving actuator identification to achieve realistic simulation. They use methods like iterative coordinate descent.
  • Fabre et al. [6] (Dynaban firmware for Dynamixel): Proposes Dynaban, an open-source alternative firmware for Dynamixel servo-motors. This work explicitly mentions using the Stribeck friction model for Dynamixel actuators, indicating prior recognition of its importance. They use genetic algorithms for parameter tuning.
  • Background: These papers highlight various methods for identifying actuator dynamics and tuning simulation parameters to reduce the sim-to-real gap. However, they are often constrained by the complexity of servo actuator dynamics and the limitations of simple friction models.

• Machine Learning for Actuator Dynamics [7, 8]:

  • Lee et al. [7]: Introduced a multi-layer perceptron to predict torque applied by a servo actuator using position errors and velocities as input. While outperforming an ideal model, it lacked comparison with an identified physical model.
  • Serifi et al. [8]: Proposed a transformer-based augmentation to correct simulated position of servo actuators. This showed promising results but lacked physical interpretation for corrections.
  • Background: These works explore data-driven approaches using machine learning to model actuator dynamics, moving beyond purely physics-based models. However, they may sacrifice physical interpretability, which the current paper aims to retain by focusing on extended physical friction models.

• Friction Modeling [9, 10, 11, 12, 17, 18, 19, 20, 24]:

  • Al-Bender and Swevers [9]: Characterization of friction force dynamics, emphasizing the complexity of friction and its decomposition into static and viscous components, as well as pre-sliding and sliding regimes.
  • Coulomb [10]: The original description of static friction as a constant force. This is the historical basis for the Coulomb-Viscous model.
  • Pennestri et al. [11]: A review and comparison of dry friction force models, including the Stribeck effect.
  • Biteout et al. [12]: Discusses modeling and identification of load and temperature effects on friction in robotic joints, noting that extra static friction produced by load-dependence is reduced when the system is moving. This directly supports the Stribeck load-dependent model in the current paper.
  • Wilfrido et al. [17]: Discusses load-dependent friction laws for harmonic drive gearboxes, showing that friction effects can be quadratic in the load. This provides the motivation for the Quadratic model ($\mathcal{M}_6$).
  • Zhu et al. [18]: Explores design, modeling, and analysis of a liquid cooled proprioceptive actuator, likely involving detailed friction modeling and drive/backdrive diagrams.
  • Mori and Venture [19]: Focuses on identification of gear transmission's efficiency by neural network, which may implicitly capture friction effects.
  • Wang and Kim [20]: Characterizes directional efficiency in geared transmissions and backdrivability, motivating the Directional model ($\mathcal{M}_5$).
  • Gonthier et al. [24]: Discusses dwell time dependent friction, which is acknowledged as a limitation and future work in the current paper.
  • Background: This extensive body of work demonstrates that friction is a complex phenomenon with various effects beyond simple Coulomb-Viscous behavior. These include the Stribeck effect (velocity-dependent reduction of static friction), load-dependence (friction varying with applied load), directional efficiency, and even quadratic effects. The current paper builds upon these observations by systematically integrating them into extended friction models.

3.3. Technological Evolution

The field of robotic simulation has evolved from simple kinematic models to complex physics engines capable of simulating dynamics, contacts, and multi-body systems. Early simulations often relied on highly simplified actuator models and friction models (like Coulomb-Viscous) due to computational limitations and a focus on gross motion rather than fine dynamics.

With the rise of Reinforcement Learning in robotics, the demand for high-fidelity simulations has increased dramatically. RL algorithms are highly sensitive to discrepancies between simulation and reality, making accurate actuator dynamics crucial for effective sim-to-real transfer. This has led to a focus on identifying more accurate actuator parameters and friction models.

This paper's work fits into this technological timeline by addressing a critical gap: the insufficient representation of friction dynamics in state-of-the-art physics engines. It leverages existing knowledge about various friction phenomena (e.g., Stribeck, load-dependence) that have been studied in mechanical engineering and applies them to the specific context of servo actuators for robotic simulation. By proposing extended friction models and a systematic identification method, it aims to push the boundaries of simulation realism, which is essential for the next generation of RL-trained robots.

3.4. Differentiation Analysis

Compared to the main methods in related work, the core differences and innovations of this paper's approach are:

• Comprehensive Friction Model Integration: While individual friction effects (like Stribeck [6, 11] or load-dependence [12, 17]) have been studied, this paper systematically analyzes and integrates a hierarchy of increasingly complex friction models (from $\mathcal{M}_1$ to $\mathcal{M}_6$) that account for Stribeck, load-dependence, directional efficiency, and quadratic effects into a unified framework for servo actuators. This systematic comparison and validation across multiple models and actuators is a key differentiator.

• Physics-Based Interpretability vs. Black-Box ML: Unlike machine learning-based approaches [7, 8] that provide black-box corrections without clear physical interpretation, this paper focuses on physics-based models. This allows for a deeper understanding of the underlying friction phenomena and provides parameters that have direct physical meaning, which can be valuable for engineering design and analysis.

• Practical Identification and Integration Method: The paper provides a clear methodology for identifying the parameters of these extended friction models using a pendulum test bench and genetic algorithms. Crucially, it also details how these models can be integrated into existing physics engines (like MuJoCo) by dynamically updating friction parameters, making the approach practical for robotic simulation.

• Empirical Validation on Diverse Actuators and Systems: The validation is performed on four distinct servo actuators (two Dynamixel and two eRob80 with different gear ratios) and tested on 2R manipulators. This diverse set of experimental evaluations strongly supports the generalizability and effectiveness of the proposed models beyond a single actuator type.

• Focus on Low-Gain Control: The paper explicitly highlights the importance of accurate friction modeling when servo actuators are controlled with low gains, a common scenario in RL applications. This specific focus addresses a practical pain point in the sim-to-real transfer for RL.

  In essence, while previous work has touched upon aspects of friction modeling or actuator identification, this paper offers a more comprehensive, physically interpretable, and practically applicable solution for integrating advanced friction dynamics into robotic simulations, directly addressing a critical limitation for RL-driven robotics.

4. Methodology

4.1. Principles

The core idea of the method is to enhance the realism of servo actuator simulations by replacing the simplified Coulomb-Viscous friction model (commonly used in physics engines) with extended friction models that capture more complex friction phenomena. The theoretical basis is rooted in the understanding that friction in lubricated environments like motor gearboxes involves various effects beyond constant static and linear viscous components, such as the Stribeck effect (velocity-dependent static friction), load-dependence (friction varying with applied load), directional efficiency, and quadratic load effects.

The intuition is that by accurately modeling these subtle yet significant friction effects, the simulated behavior of servo actuators will more closely match their real-world counterparts. This improved fidelity is expected to reduce the simulation-to-reality gap, thereby improving the transferability of control policies (especially those learned through Reinforcement Learning) from simulation to real robots. The method involves:

1. Defining a suite of extended friction models with increasing complexity.
2. Developing a systematic way to identify the parameters of these models from real-world data (recorded trajectories).
3. Demonstrating how to integrate these extended models into physics engines to simulate complex robotic systems.

4.2. Core Methodology In-depth (Layer by Layer)

The methodology can be broken down into three main parts: Test bench dynamics for data generation, Extended Friction Models for accurate representation, and Servo Actuator Simulation including control laws and physics engine integration.

4.2.1. Test Bench Dynamics

The system dynamics are first established for a pendulum test bench, which simplifies the environment to focus on actuator and friction dynamics.

The dynamics of the system are expressed by the following equation: $ \tau _ { m } + \tau _ { e } ( \theta ) + \tau _ { f } = J \ddot { \theta } $ where:

• $\theta$: The servo actuator position (angular position of the pendulum).
• $\dot { \theta }$: The servo actuator velocity.
• $\ddot { \theta }$: The servo actuator acceleration.
• $\tau _ { m }$: The motor torque (torque generated by the servo actuator).
• $\tau _ { e } ( \theta ) = m g l \sin ( \theta )$: The external torque caused by gravity acting on the pendulum.
  • $m$: Mass of the point load.
  • $g$: Acceleration due to gravity.
  • $l$: Length of the rigid link.
• $\tau _ { f }$: The torque caused by friction. This is the focus of the extended models.
• $J$: The total inertia of the system.

  • $J = m l ^ { 2 } + J _ { m }$: The sum of the load inertia ($m l ^ { 2 }$) and the servo actuator apparent inertia (J _ { m }).

  • If the rotor inertia $J_r$ is known, $J_m = N^2 J_r$, where $N$ is the inverse of the gear ratio. Otherwise, $J_m$ is an additional parameter to identify.

    This equation forms the basis for simulating the pendulum's motion and identifying the unknown parameters.

4.2.2. Friction Model Definition

Friction is defined as an upper bound on the torque available to prevent motion. This is crucial for simulation as it allows for a clear distinction between the friction budget and the applied friction.

The Coulomb-Viscous model ($\mathcal{M}_1$) is the baseline, representing friction as a force that needs to be overcome: $ \mathcal { M } _ { 1 } : \tau _ { f } ^ { m } = K _ { v } | \dot { \theta } | + K _ { c } $ where:

• $\tau _ { f } ^ { m }$: The maximum friction torque (or friction budget) available to prevent motion.

• K _ { v }: The viscous friction coefficient, which scales linearly with velocity.

• K _ { c }: The Coulomb friction coefficient, representing a constant static friction.

  This model is typically used in physics simulators. However, it implies zero friction at zero velocity and discontinuity, which is physically inaccurate.

The paper then introduces five extended friction models, progressively building complexity to account for observed phenomena:

1. Stribeck model ($\mathcal{M}_2$, 5 parameters): This model accounts for the Stribeck effect, where static friction is higher when the system is stationary and decreases once motion begins. $ \mathcal { M } _ { 2 } : \tau _ { f } ^ { m } = K _ { v } \dot { \theta } + K _ { c } + e ^ { - \left| \frac { \dot { \theta } } { \dot { \theta } ^ { s } } \right| ^ { \alpha } } K _ { c } ^ { s } $ where:

   • $K _ { c } ^ { s }$: The Stribeck-Coulomb friction, an extra static friction component present when not moving.
   • $\dot { \theta } ^ { s }$: The Stribeck velocity, which parametrizes the range of velocities over which the Stribeck effect is influential.
   • $\alpha$: A parameter controlling the curvature of the exponential decay.

2. Load-dependent model ($\mathcal{M}_3$, 3 parameters): This model introduces load-dependence, where friction is augmented by the load exerted on the gearbox (e.g., due to the difference between motor and external torques). $ \mathcal { M } _ { 3 } : \tau _ { f } ^ { m } = K _ { v } \dot { \theta } + K _ { c } + K _ { l } | \tau _ { m } - \tau _ { e } | $ where:

   • K _ { l }: The load-dependent friction coefficient, which adds friction proportionally to the magnitude of the differential torque $| \tau _ { m } - \tau _ { e } |$.

3. Stribeck load-dependent model ($\mathcal{M}_4$, 7 parameters): This model combines the Stribeck effect and load-dependence, acknowledging that load-dependent friction can also be reduced once the system is moving. $ \begin{array} { r } { \mathcal { M } _ { 4 } : \tau _ { f } ^ { m } = K _ { v } \dot { \theta } + K _ { c } + K _ { l } | \tau _ { m } - \tau _ { e } | } \ { + e ^ { - \left| \frac { \dot { \theta } } { \dot { \theta } ^ { s } } \right| ^ { \alpha } } [ K _ { c } ^ { s } + K _ { l } ^ { s } | \tau _ { m } - \tau _ { e } | ] } \end{array} $ where:

   • $K _ { l } ^ { s }$: The Stribeck-load-dependent friction, representing an extra load-dependent friction component specifically for the at-rest state, which then decays with velocity due to the Stribeck effect.

4. Directional model ($\mathcal{M}_5$, 9 parameters): This model accounts for directional efficiency in the gearbox, meaning friction might differ depending on whether the motor is driving the load or the load is backdriving the motor. $ \begin{array} { r } { \mathcal { M } _ { 5 } : \tau _ { f } ^ { m } = K _ { v } \dot { \theta } + K _ { c } + | K _ { m } \tau _ { m } - K _ { e } \tau _ { e } | } \ { + e ^ { - \left| \frac { \dot { \theta } } { \dot { \theta } ^ { s } } \right| ^ { \alpha } } [ K _ { c } ^ { s } + | K _ { m } ^ { s } \tau _ { m } - K _ { e } ^ { s } \tau _ { e } | ] } \end{array} $ where:

   • K _ { m } and K _ { e }: Motor and external load-dependent friction coefficients, respectively, allowing friction to be influenced differently by the motor's own torque versus the external load torque.
   • $K _ { m } ^ { s }$ and $K _ { e } ^ { s }$: Motor and external Stribeck-load-dependent friction coefficients, capturing the Stribeck effect on load-dependent friction in a directional manner.

5. Quadratic model ($\mathcal{M}_6$, 11 parameters): This model introduces quadratic load-dependent friction, observed in systems like harmonic drives, where the friction effect can be non-linear (quadratic) with respect to the load. $ \begin{array} { r } { \mathcal { M } _ { 6 } : \tau _ { f } ^ { m } = K _ { v } \dot { \theta } + K _ { c } + | K _ { m } \tau _ { m } - K _ { e } \tau _ { e } | } \ { + e ^ { - | \frac { \dot { \theta } } { \dot { \theta } ^ { s } } | ^ { \alpha } } [ K _ { c } ^ { s } + | K _ { m } ^ { s } \tau _ { m } - K _ { e } ^ { s } \tau _ { e } | + Q ] } \ { Q = { K _ { e } ^ { q } \tau _ { e } ^ { 2 } \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad } \ { K _ { m } ^ { q } \tau _ { m } ^ { 2 } \quad \quad \quad \quad \quad \quad i f \ | \tau _ { m } | < | \tau _ { e } | } \end{array} $ where:

   • $Q$: A term representing the quadratic friction.
   • $K _ { e } ^ { q }$ and $K _ { m } ^ { q }$: External and motor load-dependent quadratic friction coefficients. The choice between these two is conditional: if the absolute motor torque is less than the absolute external torque, quadratic friction is based on the external torque squared; otherwise, it's based on the motor torque squared. This implies the quadratic effect is more pronounced from the dominant torque source.

4.2.3. Simulating Servo Actuators with Friction

The paper then describes how to simulate a position-controlled servo actuator using these friction models.

4.2.3.1. Servo Actuator Model

A servo actuator model $\boldsymbol{\mathcal{S}}$ is a function that takes the current state $(\theta, \dot{\theta})$ and a target position $\theta^d$, and outputs the motor torque $\tau_m$ to be applied. $ \tau _ { m } = \boldsymbol { \mathcal { S } } ( \boldsymbol { \theta } , \dot { \boldsymbol { \theta } } , \boldsymbol { \theta } ^ { d } ) $ The servo actuator model consists of a control law and a motor model.

4.2.3.2. Voltage Control Law

If the servo actuator uses a voltage control law, a PID controller typically computes the voltage $U$ to apply to the motor: $ U = c l i p ( P I D ( \theta , \dot { \theta } , \theta ^ { d } ) , [ - U _ { m a x } , U _ { m a x } ] ) $ where:

• $PID ( \theta , \dot { \theta } , \theta ^ { d } )$: The output of the PID controller based on current position, velocity, and desired position.

• U _ { m a x }: The maximum voltage the motor can receive.

• clip: A function that bounds the voltage to the interval [ - U _ { m a x } , U _ { m a x } ].

  The motor torque $\tau_m$ is then derived from DC motor equations: $ \tau _ { m } = \frac { k _ { t } } { R } U - \frac { k _ { t } ^ { 2 } } { R } \dot { \theta } $ where:

• k _ { t }: The motor torque constant. This includes the reducer ratio.

• $R$: The internal motor resistance. If $k_t$ and $R$ are unknown, they are additional parameters to identify. This equation assumes a drive/brake controller.

4.2.3.3. Current Control Law

If the servo actuator uses a current control law, a PID controller computes a target current $I$: $ \begin{array} { l } { I = c l i p ( P I D ( \theta , \dot { \theta } , \theta ^ { d } ) , [ I _ { m i n } , I _ { m a x } ] ) , } \ { ~ } \ { { \mathrm { w i t h } } \quad { } } \ { { \mathrm { } } } \end{array} $ where:

• $I_{min}$ and $I_{max}$: Minimum and maximum allowable currents.

• $I _ { e m f } = \frac { 1 } { R } [ U _ { m a x } - k _ { t } \dot { \theta } ]$: The maximum current allowable by the input voltage (due to back-electromotive force).

• I _ { h e a t }: A limit current to dissipate heat.

  The motor torque $\tau_m$ is then derived from the current: $ \tau _ { m } = k _ { t } I $

4.2.3.4. Simulating the Pendulum Test Bench

With the test bench dynamics, an extended friction model, and a servo actuator model, the simulation of a single step of the pendulum dynamics with friction is performed using Algorithm 1.

Algorithm 1: Simulating one step of the pendulum dynamics with friction. Input: $\Delta t$: timestep, $(\theta_k, \dot{\theta}_k)$: state at step $k$, $\theta^d_{k+1}$: desired position at step $k + 1$, $M$: friction model (section III), $S$: servo actuator model (section IV-A) Output: $(\theta_{k+1}, \dot{\theta}_{k+1})$: state at step $k + 1$

1. $\tau_m \leftarrow S(\theta_k, \dot{\theta}_k, \theta^d_{k+1})$; // servo actuator model

2. $\tau_e \leftarrow mgl \sin(\theta_k)$; // test bench dynamics

3. $J \leftarrow ml^2 + J_m$; // test bench dynamics

4. $\tau_f^m \leftarrow M(\tau_m, \tau_e, \dot{\theta}_k)$; // friction model (computes maximum friction budget)

5. $\tau_{f,stop} \leftarrow - \bigg ( \frac { J } { \Delta t } \dot { \theta }_k + \tau _ { m } + \tau _ { e } \bigg )$; // torque required to stop motion

6. $\tau_f \leftarrow clip(\tau_{f,stop}, [-\tau_f^m, \tau_f^m])$; // applied friction is limited by the friction budget

   The remaining steps of the algorithm (e.g., updating position and velocity using the net torque and integrating over $\Delta t$) are implied but not explicitly detailed in the provided Algorithm 1 stub. Assuming a simple Euler integration step (or similar):

7. $\ddot{\theta} \leftarrow (\tau_m + \tau_e + \tau_f) / J$; // calculate acceleration from net torque

8. $\dot{\theta}_{k+1} \leftarrow \dot{\theta}_k + \ddot{\theta} \Delta t$; // update velocity

9. $\theta_{k+1} \leftarrow \theta_k + \dot{\theta}_{k+1} \Delta t$; // update position

4.2.3.5. Simulation using a Physics Engine

For more complex systems, physics engines (MuJoCo, Isaac Gym) are used. These simulators already implement a Coulomb-Viscous friction model. To integrate the extended friction models, the static ($K_c$) and viscous ($K_v$) friction parameters within the physics engine are updated on-the-fly based on the current state of the system, using the chosen extended friction model.

A challenge arises because external torque ($\tau_e$) and friction torque ($\tau_f$) are typically computed simultaneously by the physics engine's dynamics solver. To overcome this, the paper suggests using the previous value of $\tau_e$ to compute the current friction budget $\tau_f^m$. This approximation is deemed valid because external torque does not vary abruptly due to the soft handling of constraints in simulations.

4.2.4. Identification Method

The method for identifying the parameters of the friction and servo actuator models involves two main steps: trajectory recording and optimization.

4.2.4.1. Trajectories

Four types of trajectories are recorded on a physical pendulum test bench using different combinations of masses, pendulum lengths, and servo actuator control laws:

1. Accelerated oscillations: Oscillations with increasing frequency to reveal controller phase and amplitude shifts.

2. Slow oscillations with smaller sub-oscillations: A combination of large, slow movements and smaller, higher-frequency movements to probe different velocity regimes.

3. Raising and lowering slowly: Moving the mass up and down slowly, often resulting in a static plateau where the system remains at rest until sufficiently backdriven, highlighting static friction and load-dependence.

4. Lift and drop: Lifting a mass and then releasing it (setting current to zero for current control or releasing the H-bridge for voltage control) to observe the system's passive dynamics, particularly viscous friction and the absence of electromotive force.

   These diverse trajectories are designed to excite the various friction phenomena captured by the extended models.

4.2.4.2. Optimization

• Score Computation: For a given set of model parameters, Algorithm 1 is used to simulate the recorded trajectories. The Mean Absolute Error (MAE) between the simulated and measured trajectories is used as the score to quantify the model's accuracy. $ MAE = \frac{1}{N} \sum_{i=1}^{N} |y_{i, \text{simulated}} - y_{i, \text{measured}}| $ where $N$ is the number of data points, $y_{i, \text{simulated}}$ is the simulated position at point $i$, and $y_{i, \text{measured}}$ is the measured position at point $i$.

• Optimization Algorithm: The CMA-ES (Covariance Matrix Adaptation Evolution Strategy) genetic algorithm [22], implemented via the Optuna Python module [23], is used to identify the optimal parameters. CMA-ES is a stochastic optimization method for non-linear non-convex problems. It iteratively updates a multivariate normal distribution (mean and covariance matrix) from which new candidate solutions are sampled, effectively searching for the global optimum.

• Data Split: The recorded logs are split into 75% for parameter identification (training) and 25% for validation.

• Repetitions: Each identification process is repeated 3 times to ensure consistent convergence and robustness of the identified parameters.

  This comprehensive methodology ensures that the extended friction models are accurately parameterized and effectively integrated into simulations, leading to improved simulation fidelity.

5. Experimental Setup

5.1. Datasets

The datasets for parameter identification consist of recorded trajectories from a pendulum test bench for four distinct servo actuators. Around 100 logs, each 6 seconds long, were recorded for each servo actuator. The trajectories used are:

1. Accelerated oscillations: Oscillations with increasing frequency.

2. Slow oscillations with smaller sub-oscillations: A combination of slow large and faster small oscillations.

3. Raising and lowering slowly: Slow vertical movement of the mass.

4. Lift and drop: Mass lifted and then released.

   These trajectories are chosen to excite various friction dynamics and actuator behaviors across different velocity and load regimes.

The specific configurations used for recording these trajectories are detailed in Table I:

The following are the results from Table I of the original paper:

Description of Datasets and Choice:

• Servo Actuators:

  • Dynamixel MX-64 [13] and MX-106 [14]: These are standard spur gear reducers, known for their widespread use in educational and research robotics. They use the manufacturer's voltage control law.
  • eRob80:50 and eRob80:100 [15]: These are eRob80 actuators with harmonic drives and reduction ratios of 1:50 and 1:100, respectively. Harmonic drives are known for different friction characteristics, including quadratic load-dependence. They use a custom current control law.

• Load Mass: Varying load masses (from 0.5kg to 19.6kg) are used to explore load-dependent friction effects.

• Pendulum Length: Different pendulum lengths (0.1m to 0.5m) vary the inertia and lever arm for the external gravity torque.

• Proportional Gain: Different proportional gains are used for the PID controller to test the models under both high-gain (precise) and low-gain (loose) control, the latter being common in RL applications where friction effects are more pronounced.

  These datasets are effective for validating the method because they provide a rich set of data points covering a wide range of operating conditions (velocities, loads, control efforts) and different gearbox technologies, which allows for a thorough characterization and identification of the complex friction parameters.

5.2. Evaluation Metrics

The primary evaluation metric used in this paper is the Mean Absolute Error (MAE).

1. Conceptual Definition: The Mean Absolute Error (MAE) measures the average magnitude of the errors between predictions (simulated values) and actual observations (measured values). It calculates the average of the absolute differences between each pair of simulated and measured data points. MAE is a direct measure of prediction accuracy and is robust to outliers compared to Mean Squared Error (MSE) because it does not square the errors. In this context, it quantifies how closely the simulated servo actuator trajectories match the real-world recorded trajectories. A lower MAE indicates a more accurate simulation.

2. Mathematical Formula: $ MAE = \frac{1}{N} \sum_{i=1}^{N} |y_{i, \text{simulated}} - y_{i, \text{measured}}| $

3. Symbol Explanation:

• $N$: The total number of data points or observations in the dataset.
• $i$: An index representing a specific data point.
• $y_{i, \text{simulated}}$: The value predicted or generated by the simulation for the $i$-th data point (e.g., the simulated position $\theta$ at time step $i$).
• $y_{i, \text{measured}}$: The actual, observed, or measured value for the $i$-th data point (e.g., the real recorded position $\theta$ at time step $i$).
• $|\cdot|$: Denotes the absolute value, ensuring that positive and negative errors contribute equally to the average magnitude of error.
• $\sum_{i=1}^{N}$: Represents the sum of all absolute differences from the first data point ($i=1$) to the last ($i=N$).
• $\frac{1}{N}$: Divides the sum of absolute differences by the total number of data points to obtain the average absolute error.

5.3. Baselines

The paper's method, which proposes extended friction models ($\mathcal{M}_2$ to $\mathcal{M}_6$), is primarily compared against the standard Coulomb-Viscous model ($\mathcal{M}_1$).

• Standard Coulomb-Viscous Model ($\mathcal{M}_1$): This model represents the baseline because it is the friction model classically used in physics simulators like MuJoCo [2] and Isaac Gym [3]. It simplifies friction into a constant static friction ($K_c$) that opposes motion initiation and a viscous friction ($K_v$) that is linearly proportional to velocity and opposes ongoing motion.

  • Why it's Representative: It is the current state-of-the-art in widely adopted physics engines for robotic simulation. Therefore, demonstrating improvements over this model directly showcases the practical value of the proposed extended models for sim-to-real transfer.

    The comparison against $\mathcal{M}_1$ directly addresses the paper's core motivation: that simplified friction models limit transferability from simulation to reality. By showing significant MAE reduction compared to $\mathcal{M}_1$, the authors validate the importance of considering more advanced friction effects.

6. Results & Analysis

6.1. Core Results Analysis

The experimental results are presented in two main parts: Identification (on pendulum test bench data) and Validation (on 2R manipulators).

6.1.1. Identification Results

The Mean Absolute Error (MAE) obtained on the validation logs after parameter identification for each friction model ($\mathcal{M}_1$ to $\mathcal{M}_6$) on the four servo actuators is presented in Fig. 5.

As can be seen from the results in Fig. 5:

该图像是一个条形图，显示了四种伺服电机模型在识别后的验证日志中得到的平均绝对误差（MAE）。不同颜色的条形代表不同的模型（M1至M6），对应的电机包括MX-64、MX-106等。图中可见M2模型在所有电机中表现最佳，MAE最低，强调了先进摩擦特性在伺服电机仿真中的重要性。

Fig. 5. MAE obtained on the validation logs after identification for each model on the four servo actuators.

• Dynamixel Servo Actuators (MX-64, MX-106):

  • There is a consistent improvement in MAE from model $\mathcal{M}_1$ through $\mathcal{M}_5$. This suggests that incorporating Stribeck effects ($\mathcal{M}_2$), load-dependence ($\mathcal{M}_3$), Stribeck load-dependence ($\mathcal{M}_4$), and directional effects ($\mathcal{M}_5$) progressively enhances the accuracy for these actuators.
  • The M6 model (Quadratic) shows no further improvement for Dynamixel actuators, indicating that quadratic load-dependent friction is not a significant factor for their spur gear reducers.
  • For the MX-64, the MAE is reduced by a factor of 2.93 (from $\mathcal{M}_1$ to the best model, likely $\mathcal{M}_5$).
  • For the MX-106, the MAE is reduced by a factor of 2.02 (from $\mathcal{M}_1$ to the best model, likely $\mathcal{M}_5$).

• eRob80 Servo Actuators (eRob80:50, eRob80:100):

  • For the eRob80:100, the best performance is achieved with the \mathcal{M}_6`model`, showing `gradual improvements` from $\mathcal{M}_1$ to $\mathcal{M}_6$. This strongly supports the hypothesis that `harmonic drives` (used in eRob80) exhibit `quadratic friction effects`. The `MAE` is reduced by a factor of 2.34. * For the `eRob80:50`, there is `no clear improvement beyond the`\mathcal{M}_3`model`. This suggests that for this specific `harmonic drive` variant, `Stribeck effects` and `load-dependence` are important, but `directional` and `quadratic effects` (introduced in $\mathcal{M}_4$, $\mathcal{M}_5$, $\mathcal{M}_6$) do not significantly improve accuracy. The `MAE` is reduced by a factor of 1.51. * **Overall:** The results highlight that the `best model` for each `servo actuator` (`M5` for Dynamixel, `M6` for eRob80:100, `M3` for eRob80:50) `significantly improves the MAE` compared to the baseline `Coulomb-Viscous model` ($\mathcal{M}_1$). This strongly validates the effectiveness of the `extended friction models` for accurately representing `servo actuator dynamics` on the `pendulum test bench`. ### 6.1.2. Validation on 2R Arms To demonstrate the improvements on more complex systems, the models are validated on two `2R manipulators`. The desired test paths are presented in Fig. 6.  *该图像是一个示意图，展示了2R机械臂上记录和模拟的路径，包括圆形（A.）、正方形（B.）、方波（C.）和三角波（D.）。* Fig. 6. Paths recorded and simulated on the 2R arms: Circle (A.), Square $\mathbf { ( B . ) }$ , Square wave (C.), Triangular wave $\mathbf { ( D . ) }$ . The first arm uses `Dynamixel MX-106` and `MX-64` actuators, and the second uses `eRob80:100` and `eRob80:50` actuators. Trajectories are recorded and simulated using both `high gains (HG)` for precise control and `low gains (LG)` for loose control (typical in `RL`). Fig. 1 in the abstract and Section I provides an example log for an `MX-106` on a `triangular wave` with `low gains`, visually demonstrating the reduced error with model $\mathcal{M}_4$ compared to $\mathcal{M}_1$. The `MAE` for each model on the `2R arm trajectories` is presented in Fig. 7. As can be seen from the results in Fig. 7:  *该图像是图表，展示了在Dynamixel和eRob RR臂上不同模型的关节平均绝对误差（MAE）。图中分别展示了使用低增益（LG）和高增益（HG）控制器记录的圆形、方形、方波和三角波轨迹的MAE。不同模型的MAE通过柱状图进行对比，突出模型在仿真中的表现差异。* Fig. 7. MAE obtained on the 2R arms trajectories for each model. Each trajectory is recorded using Low Gains (LG) and High Gains (HG) for the controller. * **Dynamixel 2R Arm:** * The \mathcal{M}_4model (Stribeck load-dependent) outperforms others across all trajectory and gain configurations.
  • It achieves an MAE over twice as low as the Coulomb-Viscous model ($\mathcal{M}_1$).
  • Although $\mathcal{M}_5$ performed better during pendulum identification for Dynamixel, it does not improve results on the 2R arm. The authors suggest this might be due to overfitting during the identification phase, where the additional parameters of $\mathcal{M}_5$ might have captured noise specific to the pendulum setup rather than generalizable friction characteristics. This highlights the importance of validation on more complex systems.

• eRob80 2R Arm:

  • The \mathcal{M}_6`model` (Quadratic) is the `most accurate`, achieving an `MAE more than twice as low as the Coulomb-Viscous model` ($\mathcal{M}_1$). * This result is consistent with the `identification phase` for `eRob80:100`, further supporting the significance of `quadratic friction effects` in `harmonic drives`. * **Impact of Gains:** The improvements are evident for both `Low Gains (LG)` and `High Gains (HG)`, but often more pronounced for `LG`, where `friction dynamics` play a more dominant role in the overall system behavior. These validation results strongly confirm that the `extended friction models` provide `significant improvements in accuracy` when simulating complex `robotic systems` like `2R manipulators`, thereby enhancing the `realism` and `reliability` of `robotic simulations`. ## 6.2. Data Presentation (Tables) The paper presents one table (Table I) which was transcribed and analyzed in Section 5.1. ## 6.3. Ablation Studies / Parameter Analysis The paper implicitly conducts an `ablation study` by evaluating a hierarchy of `friction models` from $\mathcal{M}_1$ (Coulomb-Viscous) to $\mathcal{M}_6$ (Quadratic, Directional, Stribeck, Load-dependent). Each successive model adds more `friction effects`, essentially "ablating in" or "adding" complexity. * **Effectiveness of Components (Friction Models):** * **$\mathcal{M}_2$ (Stribeck):** Shows clear improvements over $\mathcal{M}_1$ for all actuators, indicating the `Stribeck effect` (velocity-dependent static friction) is generally important. * **$\mathcal{M}_3$ (Load-dependent):** Further improves accuracy, demonstrating the significance of `load-dependent friction`. For `eRob80:50`, improvements largely plateaued after $\mathcal{M}_3$, suggesting that `load-dependence` is a key effect for this specific actuator, and further complexity might not be necessary or could lead to overfitting. * **$\mathcal{M}_4$ (Stribeck Load-dependent):** Combines the previous effects and provides strong performance, especially for `Dynamixel` actuators on the `2R arm validation`, indicating that `Stribeck effects` also apply to `load-dependent friction`. * **$\mathcal{M}_5$ (Directional):** While showing some improvement during identification for `Dynamixel`, it did not generalize to the `2R arm`, hinting at potential `overfitting` or less pronounced `directional effects` in the 2R arm's typical operating range. For `eRob80`, it continued to show some benefits. * **$\mathcal{M}_6$ (Quadratic):** Proves crucial for `eRob80:100`, both in identification and `2R arm validation`, confirming the presence of `quadratic load-dependent friction` in `harmonic drives`. * **Hyper-parameter Impact (Proportional Gains):** * The `MAE` results for `2R arms` (Fig. 7) are presented for both `Low Gains (LG)` and `High Gains (HG)`. * The improvements from `extended friction models` are consistently observed for both `LG` and `HG`. * Often, the `MAE` is generally higher for `LG` scenarios, and the relative improvement offered by the `extended models` can be more significant. This is because with `low gains`, the controller exerts less force to correct errors, making the `intrinsic dynamics` of the actuator (including `friction`) more dominant and noticeable in the system's behavior. In `high-gain` scenarios, the controller can `mask` some of the `friction effects` by actively compensating for them. This confirms the paper's motivation regarding `RL applications` which often use `low gains`. The `drive/backdrive diagrams` in Fig. 4 for the `eRob80:100` actuator with different models further illustrate the impact of these components visually. They show how each model progressively refines the boundaries of the `static area` and the `drive/backdrive curves`, demonstrating the model's ability to capture the observed `load-dependent`, `Stribeck`, `directional`, and `quadratic effects`. The lines with lower opacity denoting the effect of velocity (1 rad/s per step) further show how the `Stribeck effect` (reduction of friction with velocity) influences the shape of these curves. # 7. Conclusion & Reflections ## 7.1. Conclusion Summary This work successfully presents and validates a methodology for simulating `servo actuators` using `extended friction models`. By moving beyond the simplistic `Coulomb-Viscous model`, the authors demonstrate that incorporating advanced `friction effects` such as `Stribeck`, `load-dependence`, `directional efficiency`, and `quadratic load effects` significantly improves the `accuracy` of `servo actuator simulations`. The proposed method includes a comprehensive analysis of various `friction models`, a robust `parameter identification method` using a `pendulum test bench` and `genetic algorithms`, and guidelines for `integrating these models into physics engines`. The validation on four distinct `servo actuators` and `2R manipulators` shows substantial reductions in `Mean Absolute Error`, often by more than a factor of two, compared to the standard model. These findings underscore the critical importance of high-fidelity `friction modeling` for enhancing `realism` and `reliability` in `robotic simulations`, particularly to bridge the `sim-to-real gap` for `Reinforcement Learning applications`. ## 7.2. Limitations & Future Work **Limitations identified by the authors:** * **Temperature Effects:** The current models do not account for the `temperature of servo actuators`. `Friction characteristics` are known to be `temperature-dependent` [12], which could introduce discrepancies. Addressing this would require implementing a `thermal model` and controlling/measuring temperature during data logging. * **Radial Forces:** `Radial forces` exerted on the actuator components are neglected to simplify the `experimental setup`. These forces can influence `friction`. * **Dwell Time:** The `dwell time` [24], which is the duration two surfaces remain stationary relative to each other before sliding, is not considered. This effect could be modeled by introducing a `delay` in the `friction response`. **Future work suggested by the authors:** * **Implementation in Physics Engines:** Focus on the full `implementation of the proposed models in a physics engine` (beyond simply updating static and viscous parameters) to simulate more accurately complex systems like `humanoids`. * **Improved Sim-to-Real Transfer for RL:** The ultimate goal is to use these improved models to `enhance the transferability of reinforcement learning policies from simulation to real-world`. ## 7.3. Personal Insights & Critique **Personal Insights & Applications:** This paper provides highly valuable insights for anyone working on `robotics simulation`, especially in the context of `Reinforcement Learning`. The systematic approach to modeling and identifying `extended friction effects` offers a clear pathway to improving `simulation fidelity`. * **Transferability:** The work directly addresses the `sim-to-real gap`, which is a major bottleneck in `RL`. More accurate `actuator models` mean that policies learned in simulation are more likely to perform well on real hardware, saving significant time and resources. * **Design & Diagnostics:** The `physics-based models` provide `interpretable parameters`. These parameters could be used not just for simulation, but also for `diagnosing issues` in real actuators or informing `actuator design` by understanding which `friction components` are most dominant. * **Generalizability:** The validation on different types of actuators (spur gear vs. harmonic drive) and complex `2R manipulators` suggests the methodology is generalizable across various robotic platforms. * **Value for Low-Gain Control:** The emphasis on `low-gain control` scenarios (common in `RL`) highlights a specific, critical area where simplified models fail most acutely, making this work particularly relevant. **Potential Issues, Unverified Assumptions, or Areas for Improvement:** * **Computational Cost:** While not explicitly discussed, `extended friction models` with more parameters and complex non-linear terms (like exponentials and quadratics) will inherently increase the `computational cost` of simulation per time step. For `real-time RL training` that relies on extremely fast simulations (e.g., `Isaac Gym`), this trade-off between `accuracy` and `speed` needs careful consideration. The current approach of updating parameters `on-the-fly` in existing `physics engines` might mitigate this to some extent, but a full, custom implementation could be more computationally intensive. * **Identification Complexity:** Identifying up to 11 parameters per actuator using a `genetic algorithm` (`CMA-ES`) can be computationally intensive and sensitive to the quality and diversity of the `recorded trajectories`. While the paper states consistent convergence, the practical effort and time required for such an extensive identification process for every new actuator could be substantial. * **External Torque Approximation in Physics Engines:** The approximation of using the `previous value of`\tau_e to compute the current friction budget in physics engines is stated as valid due to soft handling of constraints. While reasonable for many cases, in scenarios with very stiff contacts or sudden impacts, this approximation might break down and lead to inaccuracies or instabilities.

• Model Selection: The paper shows that the "best" model varies by actuator type (e.g., $\mathcal{M}_4$ for Dynamixel, $\mathcal{M}_6$ for eRob80:100, $\mathcal{M}_3$ for eRob80:50). This implies that a universal "best" model might not exist, and model selection would be an additional step in deployment. A more automated or adaptive model selection mechanism could be beneficial.

• Beyond Rotational Joints: The work focuses on rotational servo actuators and pendulum test benches. While the principles are likely transferable, extending these models to linear actuators or other joint types would require specific adaptations and validations.

  Overall, this paper makes a significant contribution to robotic simulation fidelity. The identified limitations and future work suggest exciting avenues for further research, particularly in optimizing computational efficiency and exploring even more complex friction phenomena for the next generation of highly realistic robotic simulations.