1. Bibliographic Information

1.1. Title

HUMAN ACTIVITY RECOGNITION AND OPTIMIZATION OF BIPED EXOSKELETONS THROUGH ARTIFICIAL INTELLIGENCE: AN INTEGRATED APPROACH

1.2. Authors

• Mihaela Rusanovschi (Corresponding author, ORCID: 0000-0002-2447-5997, mihaela.rusanovschi@iis.utm.md)

• Galina Marusic (ORCID: 0000-0002-2984-2055)

  Affiliation: Technical University of Moldova, 168 Stefan cel Mare Blvd., Chisinau, Republic of Moldova

1.3. Journal/Conference

The paper is published in the Journal of Engineering Science. The specific issue is March, 2025, Vol. XXXII (1), pp.1-79. The journal appears to be a peer-reviewed publication in the field of engineering.

1.4. Publication Year

2025 (Received: 03.02.2025, Accepted: 03.24.2025)

1.5. Abstract

This paper presents an integrated approach combining inertial sensor-based human activity recognition (HAR) with artificial intelligence (AI) techniques, specifically reinforcement learning (RL), to optimize bipedal exoskeletons. The core motivation is to enhance the adaptability and energy efficiency of exoskeletons in practical applications. The study hypothesizes that this integration can lead to personalized and efficient control strategies. The research develops a robust HAR system to classify activities such as normal walking, stair climbing, and sitting/standing. This system involves preprocessing accelerometer and gyroscope data through segmentation and feature extraction, followed by supervised classification using Support Vector Machines (SVM) and Random Forest algorithms. Concurrently, RL optimization is performed in simulated environments like Webots to improve exoskeleton control. Preliminary results demonstrate a 92% accuracy in HAR and a 15% reduction in metabolic cost through RL, which also improves exoskeleton stability and user comfort. This innovative, integrated approach aims to minimize manual adjustments in exoskeleton design, with promising applications in rehabilitation and physical augmentation.

1.6. Original Source Link

/files/papers/690214ed84ecf5fffe471893/paper.pdf Publication Status: Officially published in the Journal of Engineering Science in 2025.

2. Executive Summary

2.1. Background & Motivation

The core problem addressed by this paper is the limited adaptability and energy inefficiency of traditional bipedal exoskeletons to diverse user needs and environmental conditions. While exoskeletons have advanced significantly for medical, industrial, and military applications, their development faces challenges such as extensive testing on human subjects, complex manually established control laws, and a lack of personalized response.

Prior research often focuses on Human Activity Recognition (HAR) for monitoring purposes or Reinforcement Learning (RL) for control in isolation, without effectively integrating real-time activity data into adaptive control strategies. This creates a gap where exoskeletons struggle to dynamically understand user intentions and adjust their assistance accordingly, leading to suboptimal performance, higher metabolic cost, and reduced user comfort. The paper's entry point is the recognition that combining HAR to identify user activities with RL to adapt control policies could create a synergistic framework, reducing reliance on manual adjustments and improving overall exoskeleton performance.

2.2. Main Contributions / Findings

The paper makes several significant contributions:

• Integrated HAR-RL Framework: It proposes and validates an innovative, integrated approach that combines inertial sensor-based HAR with RL for bipedal exoskeleton optimization, addressing a critical gap in existing research.
• Robust HAR System Development: The research develops a Human Activity Recognition (HAR) system capable of classifying five distinct activities (normal walking, climbing stairs, descending stairs, sitting down, and rising from a chair) with a high accuracy of 92% using Support Vector Machines (SVM) and Random Forest algorithms on accelerometer and gyroscope data.
• RL-based Exoskeleton Optimization: It demonstrates the effectiveness of Reinforcement Learning (RL) (specifically Proximal Policy Optimization - PPO) in simulated environments (Webots and OpenSim) to optimize exoskeleton control, leading to a 15% reduction in metabolic cost compared to traditional PID controllers.
• Enhanced Exoskeleton Performance: The integrated HAR-RL system significantly improves exoskeleton stability (e.g., 10% increase for stair climbing) and user comfort (e.g., 16% decrease in quadriceps muscle force, 18% reduction in ankle impact).
• Reduced Adaptation Time: The integration of HAR predictions enabled dynamic adjustment of exoskeleton control, reducing the adaptation time to activity changes from 1.2 seconds (without HAR) to 0.5 seconds, highlighting the system's responsiveness.
• Paving the Way for Scalable Applications: By minimizing manual adjustments and enhancing energy efficiency, the research contributes to exoskeleton design with promising applications in rehabilitation and physical augmentation, making exoskeletons more practical and accessible.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To fully understand this paper, a foundational grasp of several key concepts is essential:

• Bipedal Exoskeletons: These are wearable robotic devices designed to enhance human physical capabilities or assist in mobility. "Bipedal" refers to their two-legged structure, mimicking human locomotion. They typically consist of a frame, motors (actuators), sensors, and a control system, worn by a user to provide assistive torque or force to joints. Their applications range from assisting individuals with mobility impairments (rehabilitation) to augmenting strength for industrial or military tasks.

• Artificial Intelligence (AI): A broad field of computer science that enables machines to perform tasks typically requiring human intelligence, such as learning, problem-solving, decision-making, perception, and understanding language. In this paper, AI techniques are used to analyze human movement and optimize robotic control.

• Human Activity Recognition (HAR): A subfield of AI that focuses on identifying and classifying human actions or movements from sensor data. The goal is to automatically determine what a person is doing (e.g., walking, running, sitting, climbing stairs). Inertial sensors are commonly used for HAR.

• Inertial Sensors: Electronic devices that measure and report a body's velocity, orientation, and gravitational forces.

  • Accelerometer: A sensor that measures non-gravitational acceleration, which is the rate of change of velocity of an object in its own reference frame. It typically measures acceleration along three perpendicular axes (X, Y, Z).
  • Gyroscope: A sensor that measures angular velocity, which is the rate of rotation around a particular axis. Like accelerometers, they commonly measure rotation around three axes.
  • Together, accelerometers and gyroscopes are often combined in Inertial Measurement Units (IMUs) to provide comprehensive motion data.

• Reinforcement Learning (RL): A paradigm of machine learning where an agent learns to make decisions by performing actions in an environment to maximize a cumulative reward.

  • Agent: The learner or decision-maker (e.g., the exoskeleton's control system).
  • Environment: The world with which the agent interacts (e.g., the simulated physical world where the exoskeleton operates).
  • State: A snapshot of the environment at a given time (e.g., exoskeleton joint angles, sensor readings).
  • Action: A decision or output from the agent that affects the environment (e.g., torques applied by exoskeleton actuators).
  • Reward: A scalar feedback signal from the environment indicating the desirability of an agent's actions (e.g., a high reward for stable, energy-efficient movement).
  • Policy: A strategy that maps states to actions, determining the agent's behavior. The goal of RL is to learn an optimal policy.

• Supervised Learning: A type of machine learning where an algorithm learns from a labeled dataset (input-output pairs). The model learns a mapping from inputs to outputs and can then predict outputs for new, unseen inputs. HAR systems often use supervised learning for classification.

• Support Vector Machines (SVM): A powerful supervised learning algorithm used for classification and regression. SVMs work by finding an optimal hyperplane that best separates different classes in the feature space.

  • Hyperplane: A decision boundary that separates data points of different classes. In a 2D space, it's a line; in 3D, it's a plane; and in higher dimensions, it's a hyperplane.
  • Kernel Trick: A technique used by SVMs to handle non-linearly separable data by implicitly mapping the input features into a higher-dimensional space where a linear separation is possible, without explicitly calculating the coordinates in that space. The Radial Basis Function (RBF) kernel is a common choice for this.

• Random Forest: An ensemble learning method for classification and regression that operates by constructing a multitude of decision trees at training time. For classification tasks, the output of the Random Forest is the class selected by most trees (majority vote).

  • Ensemble Learning: The process of combining multiple machine learning models (often called "weak learners") to achieve better predictive performance than a single model.
  • Decision Tree: A flowchart-like structure where each internal node represents a "test" on an attribute, each branch represents the outcome of the test, and each leaf node represents a class label.
  • Bootstrap Aggregating (Bagging): A technique where multiple subsets of the original training data are created by sampling with replacement. Each subset is then used to train a separate model.

• Proximal Policy Optimization (PPO): A popular Reinforcement Learning algorithm that balances ease of implementation, sample efficiency, and good performance. It's an actor-critic method that aims to update the policy in a stable manner by taking multiple small steps, ensuring that the new policy does not deviate too much from the old one.

  • Actor-Critic: An RL architecture where two neural networks work together: an "actor" network learns the policy (what action to take), and a "critic" network estimates the value function (how good an action is).

• Metabolic Cost: In the context of exoskeletons, this refers to the physiological energy expenditure of the human user. Reducing metabolic cost means the user expends less energy to perform a task, leading to less fatigue and increased endurance. It's often measured in terms of oxygen consumption or Joules per kilogram of body mass.

• Zero Moment Point (ZMP): A concept in robotics and biomechanics used to analyze the stability of dynamic bipedal locomotion. It represents the point on the ground about which the net moment of all forces (gravitational and inertial) acting on the robot/human equals zero. If the ZMP stays within the support polygon (the area defined by the feet on the ground), the robot/human is stable.

3.2. Previous Works

The paper contextualizes its work by citing several key developments and identifying gaps in existing research:

• RL for Metabolic Cost Reduction: Studies like [6] (presumably specifically cited) demonstrate that RL can significantly reduce metabolic energy consumption in exoskeleton-assisted locomotion, reporting reductions of up to 20%. This highlights the potential of RL for optimizing exoskeleton efficiency.
• High-Accuracy HAR Systems: Research such as [7] shows that HAR systems, particularly those using supervised learning, can achieve accuracies of over 90% for basic tasks. This establishes HAR as a mature field for activity detection.
• Divergent Research Paths: The authors point out a critical gap: much research focuses either exclusively on HAR for activity monitoring [8] or solely on RL for control without leveraging real-time activity data [9]. This divergence means exoskeletons often lack the ability to truly understand user intent and adapt dynamically.
• Lack of Standardized Metrics: The paper also references [10], which highlights the challenge of comparing exoskeleton performance across different studies due to a lack of standardized performance metrics. This complicates the assessment of novel control strategies.
• Need for Personalized Control: Recent work, including [11], emphasizes the importance of personalized control strategies because human biomechanics vary significantly between individuals. This underscores the need for adaptive systems that can tailor assistance to each user.

3.3. Technological Evolution

The field of exoskeletons has evolved from rigid, pre-programmed devices to more intelligent, adaptive systems. Early exoskeletons relied on manually tuned control laws and extensive physical testing, which was time-consuming, costly, and limited their adaptability. The integration of Artificial Intelligence has been a significant step forward. HAR emerged as a way for exoskeletons to "understand" what a user is doing, moving beyond simple predefined motion patterns. Concurrently, Reinforcement Learning has shown promise in learning complex, optimal control policies in simulated environments, reducing the need for exhaustive real-world experimentation.

This paper's work fits within the current technological trajectory of increasingly intelligent and autonomous exoskeletons. It addresses the current limitation where HAR and RL often exist as separate components, rather than synergistically informing each other.

3.4. Differentiation Analysis

Compared to the main methods in related work, the core innovation and differentiation of this paper's approach lie in its integrated framework of Human Activity Recognition (HAR) and Reinforcement Learning (RL).

• Traditional Exoskeletons: Rely on manually established control laws or PID controllers. These are often rigid, require extensive tuning, and lack adaptability to varying user intentions or environmental changes. This paper's approach replaces or augments these with AI-driven adaptive policies.

• HAR-only Systems: While successful in identifying activities, they often provide only "monitoring" capabilities [8]. They detect an activity but don't inherently feed that information into a real-time, adaptive control system for the exoskeleton. This paper uses HAR predictions as direct input to guide RL's reward function and state space, making exoskeleton control responsive to detected activities.

• RL-only Systems: Some RL approaches optimize exoskeleton control [9], but they might do so without explicit, high-level HAR input. They learn from generic sensor data, but might not explicitly "know" if the user intends to climb stairs versus walk, which could lead to less optimal or slower adaptation for specific tasks. This paper's method uses HAR to explicitly inform the RL agent about the current activity, allowing for more targeted and efficient policy adjustments (e.g., prioritizing stability for stair climbing).

  In essence, the paper differentiates itself by creating a feedback loop where HAR provides context and intent, which then dynamically shapes the RL's optimization goals, leading to faster adaptation, improved efficiency, and enhanced comfort tailored to the specific activity being performed. This is a significant step towards truly personalized and intelligent exoskeleton assistance.

4. Methodology

This section describes the integrated human activity recognition (HAR) system and the reinforcement learning (RL) optimization process for bipedal exoskeletons. The methodology is divided into two main parts: HAR based on inertial sensor data and RL optimization in simulated environments.

4.1. Principles

The core idea behind this integrated approach is to create an exoskeleton control system that can intelligently adapt to a user's current activity and optimize its assistance accordingly. This is achieved by first accurately identifying the user's activity using Human Activity Recognition (HAR). Once an activity (e.g., walking, stair climbing) is recognized, this information is fed into a Reinforcement Learning (RL) framework. The RL agent then uses this activity context to dynamically adjust its control policy, focusing on relevant optimization goals (e.g., prioritizing stability for stair climbing or speed for normal walking). This combination aims to provide personalized, energy-efficient, and responsive assistance, minimizing the need for manual adjustments and improving overall user experience.

4.2. Core Methodology In-depth (Layer by Layer)

4.2.1. Human Activity Recognition (HAR)

The HAR system aims to classify human activities using data from inertial sensors.

Data Acquisition

Data is collected from integrated inertial sensors, specifically a triaxial accelerometer and a triaxial gyroscope.

• Accelerometer: Measures linear acceleration along three axes (x, y, z), denoted as $acc\_x$, $acc\_y$, $acc\_z$.
• Gyroscope: Measures angular velocity along three axes (x, y, z), denoted as $gyro\_x$, $gyro\_y$, $gyro\_z$. The data is collected from human subjects performing five distinct activities: normal walking, climbing stairs, descending stairs, sitting down, and rising from a chair.
• Sampling Rate: $50 \mathsf { H z }$ (meaning 50 data points are collected per second).
• Recording Length: Approximately 30 seconds for each activity.
• Format: Continuous time series stored in CSV (Comma Separated Values) format.

Preprocessing

Raw sensor data undergoes several preprocessing steps to make it suitable for classification.

1. Segmentation: The continuous time series data is divided into smaller, overlapping segments called time windows. This is crucial because activities are continuous, and windows help create discrete samples for classification.

   • Window Size: $N = 128$ samples. At a $50 \mathsf { H z }$ sampling rate, this corresponds to $128 \text{ samples} / 50 \text{ samples/second} = 2.56$ seconds per window.
   • Overlap: 50% overlap between consecutive windows. This helps capture transitions between activities and ensures that no important information is lost at window boundaries.
   • Each window $W = \{ \omega _ { 1 } , \omega _ { 2 } , \dots , \omega _ { N } \}$ contains the sensor values for a particular channel (e.g., $acc\_x$).

2. Feature Extraction: From each segmented window, statistical features are calculated. These features reduce the dimensionality of the raw data while extracting relevant characteristics that distinguish different activities. The paper specifies four features:

   • Mean ($\mu$): Represents the average value of the sensor signal within a window. $ \mu = \frac { 1 } { N } \sum _ { i = 1 } ^ { N } \omega _ { i } $ Where:
     • $N$: The number of samples in the time window (128 in this case).
     • $\omega_i$: The $i$-th sensor value within the window $W$.
     • $\mu$: The mean value of the window.
   • Standard Deviation ($\sigma$): Measures the amount of variation or dispersion of the sensor signal values from the mean. A higher standard deviation indicates greater variability. $ \sigma = \sqrt { \frac { 1 } { N - 1 } \sum _ { i = 1 } ^ { N } ( \omega _ { i } - \mu ) ^ { 2 } } $ Where:
     • $N$: The number of samples in the time window.
     • $\omega_i$: The $i$-th sensor value within the window $W$.
     • $\mu$: The mean value of the window.
     • $\sigma$: The standard deviation of the window.
   • Root Mean Square (RMS): Represents the quadratic mean of the sensor signal values, often used to quantify the magnitude of a varying quantity. It is particularly useful for signals that oscillate around zero. $ RMS = \sqrt { \frac { 1 } { N } \sum _ { i = 1 } ^ { N } \omega _ { i } ^ { 2 } } $ Where:
     • $N$: The number of samples in the time window.
     • $\omega_i$: The $i$-th sensor value within the window $W$.
     • RMS: The Root Mean Square value of the window.
   • Signal Magnitude Area (SMA): A feature commonly used in HAR to quantify the overall magnitude of the acceleration signal over a period. It is the sum of the absolute values of the acceleration components. $ SMA = \frac { 1 } { N } \sum _ { i = 1 } ^ { N } ( | acc _ { x } ( i ) | + | acc _ { y } ( i ) | + | acc _ { z } ( i ) | ) $ Where:
     • $N$: The number of samples in the time window.
     • $acc_x(i)$, $acc_y(i)$, $acc_z(i)$: The $i$-th accelerometer readings for the x, y, and z axes, respectively, within the window.
     • SMA: The Signal Magnitude Area for the accelerometer data within the window. These four features are calculated for each sensor channel (3 accelerometer channels + 3 gyroscope channels). This results in a feature vector $X = [ f _ { 1 } , f _ { 2 } , \dots , f _ { k } ]$ per window, where $k$ is the total number of features. If 4 features are extracted for each of the 6 channels (3 accelerometer, 3 gyroscope), then $k = 4 \times 6 = 24$.

3. Normalization: The extracted features are standardized using Z-score transformation. This process scales the features so they have a mean of 0 and a standard deviation of 1, which helps prevent features with larger numerical ranges from dominating the classification process. $ x _ { scaled } = \frac { x - \mu _ { t r a i n } } { \sigma _ { t r a i n } } $ Where:

   • $x$: The original feature value.
   • $x_{scaled}$: The normalized feature value.
   • $\mu_{train}$: The mean of that specific feature calculated only from the training dataset.
   • $\sigma_{train}$: The standard deviation of that specific feature calculated only from the training dataset. This ensures that the test set features are scaled consistently with the training set, avoiding data leakage.

Classification

Two supervised classification models are employed to identify activities:

1. Support Vector Machines (SVM): SVM aims to find an optimal hyperplane that best separates different activity classes in the feature space.

   • Objective: The model minimizes the objective function $\frac { 1 } { 2 } \left| \left| \omega \right| \right| ^ { 2 }$ subject to the constraints $y _ { i } ( \omega \cdot x _ { i } + b ) \geq 1$. Where:
     • $\omega$: The normal vector to the hyperplane. Minimizing $||\omega||^2$ maximizes the margin between classes.
     • $x_i$: The feature vector of the $i$-th training sample.
     • $y_i$: The class label for the $i$-th sample (typically -1 or 1 for binary classification, extended for multi-class).
     • $b$: The bias term (offset) of the hyperplane.
     • $\omega \cdot x_i + b = 0$: Represents the hyperplane equation. The constraints ensure that all samples are correctly classified and lie outside the margin.
   • Kernel: For non-linear separability (when a straight line or plane cannot separate the data), a Radial Basis Function (RBF) kernel is used. This kernel implicitly maps the data into a higher-dimensional space where a linear separation might be possible. $ K _ { ( x _ { i } , x _ { j } ) } = \exp ( - \gamma | \big | x _ { i } - x _ { j } \big | | ^ { 2 } ) $ Where:
     • $K_{(x_i, x_j)}$: The kernel function output, representing the similarity between two feature vectors $x_i$ and $x_j$.
     • $\gamma$: A hyperparameter that defines how much influence a single training example has. A small $\gamma$ means a large radius of influence, and a large $\gamma$ means a small radius of influence. It is adjusted via cross-validation.

2. Random Forest: Random Forest is an ensemble method that aggregates the predictions of multiple decision trees.

   • Training: An ensemble of $T = 100$ decision trees is trained. Each tree is trained on a bootstrap sample (random sampling with replacement) of the original data. Additionally, at each node split, only a random subset of features is considered, which decorrelates the trees.
   • Prediction: The final prediction for a new sample is determined by majority vote among all the individual trees. $ y _ { pred } = mode ( { y _ { 1 } , y _ { 2 } , \dots , y _ { T } } ) $ Where:
     • $y_{pred}$: The final predicted class label.
     • $mode(\dots)$: The statistical mode (the most frequent class) among the predictions of the $T$ individual trees.
     • $y_1, \dots, y_T$: The class predictions from each of the $T$ decision trees.

Training and Testing

• Library: The implementation uses the scikit-learn library (version 1.2.2) in Python 3.9.
• Data Split: The dataset is divided into 70% for training and 30% for testing.
• Stratification: The train_test_split function is used with stratification, meaning that the proportion of each activity class is maintained in both the training and testing sets. This prevents an uneven distribution of classes that could bias evaluation.

Statistical Evaluation

The performance of the HAR models is assessed using standard classification metrics derived from the confusion matrix.

• Accuracy: The proportion of correctly classified samples out of the total number of samples. $ Accuracy = \frac { \sum TP _ { k } } { N } $ Where:
  • $TP_k$: The number of true positives for class $k$ (samples correctly identified as class $k$).
  • $N$: The total number of samples across all classes.
• Precision: For a given class $k$, it is the ratio of correctly predicted positive observations to the total predicted positive observations. It measures the quality of positive predictions. $ Precision _ { k } = \frac { TP _ { k } } { TP _ { k } + FP _ { k } } $ Where:
  • $TP_k$: True positives for class $k$.
  • $FP_k$: False positives for class $k$ (samples incorrectly identified as class $k$).
• Recall (Sensitivity): For a given class $k$, it is the ratio of correctly predicted positive observations to all observations in the actual class. It measures the ability of the model to find all the positive samples. $ Recall _ { k } = \frac { TP _ { k } } { TP _ { k } + FN _ { k } } $ Where:
  • $TP_k$: True positives for class $k$.
  • $FN_k$: False negatives for class $k$ (samples from class $k$ that were incorrectly classified as another class).
• F1 Score: The harmonic mean of Precision and Recall. It provides a single metric that balances both Precision and Recall, particularly useful when there is an uneven class distribution. $ F1 _ { k } = 2 \cdot \frac { Precision _ { k } \cdot Recall _ { k } } { Precision _ { k } + Recall _ { k } } $ Where:
  • $Precision_k$: The precision for class $k$.
  • $Recall_k$: The recall for class $k$.
• K-fold Cross-validation: A technique to estimate the robustness of the models. The dataset is divided into $k=5$ equal folds. The model is trained on k-1 folds and tested on the remaining fold, and this process is repeated $k$ times, with each fold serving as the test set once. The average performance across all folds provides a more reliable estimate of generalization capability.

4.2.2. Optimization of Exoskeletons through Reinforcement Learning (RL)

The RL component focuses on optimizing exoskeleton control policies in simulated environments.

Simulation Environment

To safely and efficiently train RL policies, the research utilizes two simulation environments:

• Webots: An open-source robotic simulator (version 2023a). It incorporates an ODE (Open Dynamics Engine) physics engine to realistically model the exoskeleton's dynamics. The exoskeleton model within Webots includes hip, knee, and ankle joints, each equipped with simulated electric actuators that apply torques.
• OpenSim: Biomechanical software (version 4.4) used to simulate the human-exoskeleton interaction. This software allows for modeling muscle forces and joint angles based on a standard human musculoskeletal skeleton, providing insights into the physiological impact of exoskeleton assistance.

RL Configuration

The Proximal Policy Optimization (PPO) algorithm is chosen for its stability and effectiveness in continuous action spaces.

• State Space ($S$): The state vector is the information the RL agent uses to make decisions. It contains:
  • Joint angles (of the hip, knee, and ankle joints).
  • IMU data (acceleration and angular velocity from exoskeleton or user, if integrated).
  • Interaction forces at the human-exoskeleton interface (how the human and exoskeleton push/pull on each other).
• Action Space ($A$): The action vector represents the control outputs from the RL agent, which are the torques applied by the actuators at the three exoskeleton joints.
  • $\tau_{hip}$: Torque applied at the hip joint.
  • $\tau_{knee}$: Torque applied at the knee joint.
  • $\tau_{ankle}$: Torque applied at the ankle joint.
• Reward Function: The reward function guides the RL agent to learn desired behaviors. It is designed to maximize forward speed, minimize metabolic cost, and maintain stability. $ R = \omega _ { 1 } \cdot v _ { forward } - \omega _ { 2 } \cdot E _ { metabolic } + \omega _ { 3 } \cdot S _ { stability } $ Where:
  • $R$: The scalar reward value.
  • $v_{forward}$: The forward speed of the exoskeleton and user, measured in $m \cdot s ^ { -1 }$. The RL agent is rewarded for higher forward speed.
  • $E_{metabolic}$: The estimated metabolic cost (energy expenditure) of the user, measured in $J \cdot k g ^ { -1 }$. This term is subtracted because the goal is to minimize metabolic cost.
  • $S_{stability}$: The stability margin based on the position of the Zero Moment Point (ZMP) within the support base. Higher values indicate greater stability.
  • $\omega_1, \omega_2, \omega_3$: Weighting parameters that adjust the relative importance of speed, metabolic cost, and stability in the reward function. These were adjusted empirically (e.g., 1.0, 0.5, 0.8).

Training

• Library: Training is performed using the Stable-Baselines3 library (version 1.6.0) in Python.
• Architecture: An actor-critic neural network is used, comprising 2 hidden layers, each with 64 neurons. The actor network learns the policy (mapping states to actions), and the critic network estimates the value function (how good a state or action is).
• Training Steps: The RL policy is trained for 1 million training steps, indicating a substantial amount of interaction with the simulated environment to learn robust control strategies.

HAR-RL Integration

The crucial step that links the two main components of the research.

• HAR predictions (i.e., the detected activity label like "normal walking" or "stair climbing") are used to dynamically adjust the reward function of the RL agent.
• Mechanism: This integration is simulated by passing the HAR labels as an additional input to the RL's state space. This allows the RL agent to be aware of the user's current activity.
• Dynamic Adjustment Example: If the HAR system detects "stair climbing," the reward function might be adjusted to place a higher weight on stability (e.g., increasing $\omega_3$). If "normal walking" is detected, the reward function might prioritize forward speed and metabolic efficiency (e.g., increasing $\omega_1$ and decreasing $\omega_2$). This ensures the exoskeleton provides context-aware assistance.

5. Experimental Setup

5.1. Datasets

The paper primarily relies on custom-collected data for HAR and simulated environments for RL optimization.

• For Human Activity Recognition (HAR):

  • Source: Data was collected from human subjects performing five distinct activities.
  • Characteristics: It consists of triaxial accelerometer and triaxial gyroscope data.
    • Activities: normal walking, climbing stairs, descending stairs, sitting down, and rising from a chair.
    • Sampling Rate: $50 \mathsf { H z }$.
    • Recording Length: Approximately 30 seconds per recording.
    • Format: Continuous time series data stored in CSV format.
  • Choice Justification: This custom dataset allows for specific control over the types of activities relevant to exoskeleton use and the sensor placement. The chosen activities represent common movements that an exoskeleton would need to assist.
  • Data Sample: The paper does not provide a concrete example of a raw data sample (e.g., a short snippet of CSV data), but describes its format as continuous time series of $acc_x$, $acc_y$, $acc_z$, $gyro_x$, $gyro_y$, $gyro_z$ values.

• For Reinforcement Learning (RL) Optimization:

  • Source: Simulated environments: Webots (version 2023a) and OpenSim (version 4.4).
  • Characteristics:
    • Webots: Models exoskeleton dynamics (hip, knee, ankle joints, electric actuators) with an ODE physics engine.
    • OpenSim: Simulates human-exoskeleton interaction, including muscle forces and joint angles based on a standard human musculoskeletal skeleton.
  • Choice Justification: Simulation environments are chosen to reduce the risks and costs associated with real-world experiments, allowing for extensive training of RL policies before deployment on physical hardware. They provide a controlled and repeatable environment for testing different control strategies and assessing their impact on metabolic cost, stability, and muscle effort.

5.2. Evaluation Metrics

The paper uses a comprehensive set of metrics to evaluate both the HAR system and the RL optimization, as well as their integration.

• For Human Activity Recognition (HAR): These metrics are derived from the confusion matrix and are standard for classification tasks. They are explained in detail in Section 4.2.1.

  • Accuracy: The overall proportion of correctly classified instances.
  • Precision ($Precision_k$): The proportion of true positive predictions among all positive predictions for a specific class $k$.
  • Recall ($Recall_k$): The proportion of true positive predictions among all actual positive instances for a specific class $k$.
  • F1 Score (F1_k): The harmonic mean of Precision and Recall for a specific class $k$.

• For Reinforcement Learning (RL) Optimization and HAR-RL Integration:

  • Speed ($v_{forward}$): The forward velocity of the exoskeleton and user.
    • Conceptual Definition: Quantifies how quickly the exoskeleton can move the user in a straight line. Higher speed is generally desirable for efficient locomotion.
    • Mathematical Formula: Measured in $m \cdot s^{-1}$.
    • Symbol Explanation: $m$: meters, $s$: seconds.
  • Metabolic Cost ($E_{metabolic}$): The estimated energy expenditure of the human user.
    • Conceptual Definition: Represents the physiological energy consumed by the user during movement. A primary goal of exoskeletons is to reduce this cost, thus easing user effort and prolonging endurance.
    • Mathematical Formula: Measured in $J \cdot k g^{-1}$ (Joules per kilogram). While the paper provides the unit, it does not explicitly provide the calculation formula. A common approach involves estimating metabolic power based on joint torques and velocities, or using models of muscle activity.
    • Symbol Explanation: $J$: Joules (unit of energy), kg: kilograms (unit of mass).
  • Stability ($S_{stability}$): A measure of the exoskeleton's balance, often quantified by the position of the Zero Moment Point (ZMP).
    • Conceptual Definition: Indicates how well the exoskeleton (and user) maintains balance during dynamic movements. A higher stability value implies a lower risk of falling. The paper mentions it's based on ZMP position within the support base.
    • Mathematical Formula: Measured in cm. The specific internal formula for $S_{stability}$ is not provided, but it would typically be inversely related to the deviation of the ZMP from the center of the support polygon.
    • Symbol Explanation: cm: centimeters (unit of length).
  • Muscle Force (e.g., quadriceps muscle force):
    • Conceptual Definition: The force generated by specific muscles, indicating the effort exerted by the user. Reducing muscle force implies less physical strain on the user.
    • Mathematical Formula: Measured in Newtons ($N$).
    • Symbol Explanation: $N$: Newtons (unit of force).
  • Ankle Impact:
    • Conceptual Definition: The peak force experienced at the ankle joint, particularly during events like foot strike. Reducing impact forces can improve user comfort and reduce the risk of injury.
    • Mathematical Formula: Measured in Newtons ($N$).
    • Symbol Explanation: $N$: Newtons (unit of force).
  • Adaptation Time: The time it takes for the exoskeleton control system to adjust its behavior in response to a detected change in activity.
    • Conceptual Definition: A measure of responsiveness. Faster adaptation time means the exoskeleton can quickly provide appropriate assistance when the user changes activity.
    • Mathematical Formula: Measured in seconds ($s$).
    • Symbol Explanation: $s$: seconds.

5.3. Baselines

The proposed methods are compared against established techniques to demonstrate their effectiveness.

• For Human Activity Recognition (HAR):

  • Support Vector Machines (SVM): One of the two primary classification algorithms used and evaluated against Random Forest.
  • Random Forest: The other primary classification algorithm used and evaluated against SVM. Both SVM and Random Forest serve as internal baselines for each other within the HAR component, allowing for a comparison of their performance on the specific dataset.

• For Reinforcement Learning (RL) Optimization:

  • Traditional PID Controller: This is a widely used and well-established control system often employed in robotics, including exoskeletons, as a baseline for comparison with RL-based control. A PID (Proportional-Integral-Derivative) controller calculates an "error" value as the difference between a desired setpoint and a measured process variable and applies a correction based on proportional, integral, and derivative terms. It represents a common, non-adaptive control strategy.

6. Results & Analysis

This section presents and interprets the experimental results for human activity recognition (HAR) and exoskeleton optimization using reinforcement learning (RL), including the performance of their integration.

6.1. Core Results Analysis

6.1.1. HAR System Performance

The HAR system demonstrated strong performance in classifying human activities.

• The SVM model with an RBF kernel achieved an overall accuracy of 92% on the test set.
• The Random Forest algorithm achieved a comparable accuracy of 91%. This indicates that both supervised learning models are effective for this HAR task.

The detailed performance by class, focusing on SVM, is presented in the table below:

The following are the results from [Table 1] of the original paper:

Analysis of Table 1:

• Normal walking showed the highest accuracy (indicated by high Precision, Recall, and F1 Score of 0.95, 0.94, and 0.94 respectively). This is attributed to its consistent and often periodic sensor signals.

• Activities like sitting down and rising up from a chair showed slightly lower Recall values (0.88 and 0.87, respectively). This suggests minor confusion between these two activities, likely due to similar movement dynamics (e.g., vertical motion, joint flexion/extension) during certain phases.

• Cross-validation ($k=5$) confirmed the robustness of the models, with a standard deviation of accuracy of $\pm 2\%$, implying good generalization to unseen data.

  The confusion matrix for the SVM model is presented, illustrating an idealized classification.

  该图像是论文中用于展示基于支持向量机（SVM）的人体活动识别（HAR）分类结果的混淆矩阵，显示各类活动的分类准确率，类别包括爬楼、下楼、起身、坐下和行走。

Figure 1. Confusion matrix for HAR classification with SVM.

Analysis of Figure 1: The confusion matrix (Figure 1) is presented as an idealized scenario with values of 1 on the diagonal, implying perfect classification for all activities. In a real-world scenario, the minor confusion between "sitting down" and "rising up from a chair" mentioned in the text would manifest as non-zero values off the diagonal in the corresponding cells. For instance, some "sitting down" instances might be misclassified as "rising up from a chair," and vice-versa. The authors suggest that additional features, such as signal energy or entropy, could potentially improve the distinction between these similar activities.

6.1.2. Channel Analysis and Feature Importance

• Accelerometer data contributed more significantly to classification than gyroscope data.
• The SMA (Signal Magnitude Area) feature calculated from accelerometer data was highly correlated with dynamic activities. For normal walking, the SMA was $12.5 \pm 1.2 m \cdot s ^ { -2 }$.
• For gyroscope data, the RMS (Root Mean Square) was $1.8 \pm 0.3 rad \cdot s ^ { -1 }$. This suggests accelerometers are better at capturing the linear motion and impact of activities, while gyroscopes capture rotational aspects.

6.1.3. Effect of Window Size

Additional tests explored the impact of segmentation window size on HAR performance and computational cost.

The following are the results from [Table 2] of the original paper:

Analysis of Table 2:

• A smaller window size ($N=64$ samples, 1.28 seconds) resulted in slightly lower accuracy (89%) but reduced processing time by 30%. This represents a trade-off between accuracy and computational efficiency.
• Increasing the window size to $N=256$ samples (5.12 seconds) slightly improved accuracy to 93% but came with a higher computational cost (0.35s processing time).
• The chosen $N=128$ (2.56 seconds) provided a good balance with 92% accuracy and a reasonable processing time of 0.22s.

6.1.4. SVM Model Training Evolution

The training process of the SVM model was monitored for accuracy and loss over 20 epochs.

该图像是图表，展示了论文中SVM模型训练过程中训练准确率和验证准确率随训练周期（Epoch）变化的趋势，反映模型性能的提升。

Figure 2. Evolution of SVM Model Accuracy During Training.

Analysis of Figure 2: Figure 2 shows the evolution of accuracy for the SVM model. The blue line (training accuracy) rapidly increased, reaching 0.9 after 5 epochs and stabilizing at 0.95. The orange line (validation accuracy) leveled off at 0.92. The discrepancy between training and validation accuracy (0.95 vs. 0.92) suggests a slight overfitting of the model to the training dataset. While not severe, this indicates that the model performed marginally better on data it had seen during training than on unseen validation data.

该图像是图表，展示了论文中SVM模型训练过程的损失变化，横轴为训练轮次（Epoch），纵轴为损失值（Loss），包括训练损失和验证损失，显示随着训练进行，损失逐渐下降，表明模型性能提升。

Figure 3. Evolution of SVM Model Loss During Training.

Analysis of Figure 3: Figure 3 displays the evolution of loss for the SVM model during training. Both the training loss (blue line) and validation loss (orange line) consistently decreased from an initial value of 1.5 to a final value of 0.2. This consistent reduction in loss for both sets confirms the model's convergence, indicating that the SVM successfully learned to minimize classification errors over the training period.

6.1.5. RL Optimization Performance

The Reinforcement Learning (RL) policy, trained using PPO in Webots, demonstrated significant improvements compared to a traditional PID controller for simulated normal walking.

The following are the results from [Table 3] of the original paper:

Analysis of Table 3:

• Metabolic Cost Reduction: The RL policy reduced the estimated metabolic cost by 15% (from $6.0 \pm 0.4 J \cdot kg^{-1}$ for PID to $5.1 \pm 0.3 J \cdot kg^{-1}$ for RL). This is a key finding, demonstrating the energy efficiency benefits of RL.
• Speed Improvement: RL also achieved a slightly higher forward speed ($1.2 \pm 0.1 m \cdot s^{-1}$) compared to PID ($1.1 \pm 0.1 m \cdot s^{-1}$).
• Stability Enhancement: RL improved stability (measured as stability margin) to $4.5 \pm 0.2 cm$, better than PID's $4.0 \pm 0.3 cm$.

6.1.6. HAR-RL Integration Performance

The integration of HAR predictions with RL control allowed for dynamic control adjustment based on the recognized activity.

• Stair Climbing: For "stair climbing," the RL policy (informed by HAR) increased stability by 10% (reaching $S_{stability} = 4.9 \pm 0.2 cm$), which is crucial for reducing the risk of falls during this challenging activity.

• Normal Walking: For "normal walking," the RL policy increased velocity by 8% (achieving $v_{forward} = 1.3 \pm 0.1 m \cdot s^{-1}$), optimizing for energy efficiency during sustained locomotion.

• Variable Terrain: On variable terrains (e.g., $5^\circ$ slopes), RL demonstrated superior stability compared to PID, reducing the ZMP deviation by 12% (from 3.5 cm to 3.1 cm). This highlights the adaptability of the RL approach.

• Reduced Muscle Effort: Tests conducted in OpenSim validated a reduction in muscle effort. The average quadriceps muscle force decreased from $250 \pm 20 N$ (with PID) to $210 \pm 15 N$ (with RL), representing a 16% decrease. This directly translates to reduced user fatigue.

• Improved Comfort: During "stair descending," RL reduced ankle impact by 18% (from 300 N to 246 N), indicating improved user comfort during impact-heavy movements.

  Crucially, the integration of HAR-RL significantly reduced the exoskeleton's adaptation time to activity changes from 1.2 seconds (without HAR) to 0.5 seconds. This represents a substantial improvement in responsiveness.

The following are the results from [Table 4] of the original paper:

Analysis of Table 4: This table further clarifies the benefits of HAR-RL integration over RL alone for specific activities.

• For Normal walking: HAR-RL slightly increased speed (1.3 vs $1.2 m \cdot s^{-1}$), further reduced metabolic cost (4.9 vs $5.1 J \cdot kg^{-1}$), and marginally improved stability (4.6 vs 4.5 cm) compared to RL without HAR context.
• For Climbed the stairs: HAR-RL also showed improvements across the board, increasing speed (0.9 vs $0.8 m \cdot s^{-1}$), reducing metabolic cost (6.2 vs $6.5 J \cdot kg^{-1}$), and notably improving stability (4.9 vs 4.7 cm) compared to RL without HAR context. These results confirm that explicitly informing the RL agent about the current activity via HAR leads to more refined and optimized exoskeleton assistance tailored to the task.

6.2. Ablation Studies / Parameter Analysis

The paper implicitly conducts an ablation study on the HAR component by evaluating the effect of window size on performance (Table 2). This shows a trade-off:

• Smaller windows (e.g., 64 samples) offer faster processing but lower accuracy.
• Larger windows (e.g., 256 samples) provide slightly higher accuracy but increase computational cost. The choice of 128 samples (2.56s) represents an optimized hyper-parameter setting that balances accuracy (92%) and processing time (0.22s).

While not a full ablation study in the sense of removing HAR entirely to compare with RL alone, the comparison of RL with and without HAR predictions (Table 4) serves a similar purpose, demonstrating the added value of the HAR component to the RL system. The weights ($\omega_1, \omega_2, \omega_3$) in the reward function are mentioned as being adjusted empirically, indicating some parameter tuning was performed to achieve the reported RL performance.

7. Conclusion & Reflections

7.1. Conclusion Summary

This study successfully demonstrated the efficacy of an integrated framework combining Human Activity Recognition (HAR) with Reinforcement Learning (RL) for optimizing the control of bipedal exoskeletons. The HAR system achieved a high accuracy of 92% in classifying five common human activities using inertial sensor data, allowing for precise identification of user movements. Concurrently, the RL optimization, implemented in simulated environments, led to a significant 15% reduction in metabolic cost compared to traditional PID controllers, while also enhancing exoskeleton stability by 10% in dynamic scenarios like stair climbing. The key contribution lies in the seamless HAR-RL integration, which enabled rapid adaptation to activity changes, reducing the response time from 1.2 seconds to 0.5 seconds. This adaptive control minimized user effort (e.g., 16% decrease in quadriceps strength) and improved comfort, offering substantial implications for medical rehabilitation and physical augmentation. The research moves towards reducing reliance on manual adjustments, paving the way for more adaptable and efficient exoskeleton systems.

7.2. Limitations & Future Work

The authors acknowledge several limitations:

• HAR Dependence on Training Data: The HAR system's performance is highly dependent on the training data, which, being collected from a limited number of subjects, may introduce bias and limit its generalizability to a wider population.

• Complexity of RL Simulations: The RL simulations in Webots and OpenSim do not fully reproduce complex real-world conditions, such as irregular surfaces or unpredictable human movements. This is a common challenge in robotics, known as the sim-to-real gap.

• Real-time Processing Computational Resources: Processing HAR data in real-time can demand significant computational resources, which might limit its direct applicability on wearable devices with constrained processing power.

  Based on these limitations, the authors suggest the following future research directions:

• Expanding the Dataset: To improve HAR robustness and generalizability, future work should focus on collecting data from a larger and more diverse group of subjects.

• Real-world Testing: Validating the integrated HAR-RL system under real-world conditions, beyond simulations, is crucial to address the sim-to-real gap and confirm its practical effectiveness.

• Deep Learning for HAR: Exploring deep learning architectures (e.g., convolutional neural networks - CNNs or recurrent neural networks - RNNs) for HAR could potentially improve accuracy and robustness, especially for differentiating nuanced activities like sitting down and rising up from a chair.

7.3. Personal Insights & Critique

This paper presents a highly relevant and promising approach to developing more intelligent and user-adaptive exoskeletons. The integration of HAR and RL is a logical and powerful synergy, addressing the critical need for exoskeletons to understand user intent and respond dynamically. The demonstrated reductions in metabolic cost and adaptation time are significant practical improvements.

Inspirations:

• The concept of a context-aware RL agent is particularly inspiring. By feeding HAR labels as an explicit part of the RL state, the agent is not just reacting to raw sensor data but is informed about the high-level task. This approach could be transferred to other human-robot interaction domains where robot autonomy needs to be guided by human intent, such as collaborative industrial robots or assistive home robots.
• The detailed analysis of HAR performance with different window sizes highlights the importance of signal processing parameters in AI systems, a crucial consideration often overlooked in favor of purely model-centric optimizations.

Potential Issues/Critique:

• Sim-to-Real Gap: While simulations are invaluable for RL training, the sim-to-real gap remains a major challenge. The metabolic cost and stability metrics are derived from simulations (Webots, OpenSim), and their direct translation to real human physiology and safety in physical exoskeletons requires rigorous validation. Factors like skin-exoskeleton interface friction, sensor noise in real-world scenarios, and unpredictable human perturbations are difficult to fully capture in simulations.

• Generalizability of HAR: The HAR dataset is custom-collected. Without details on the number of subjects, age, gender, or any physical limitations, the generalizability of the 92% accuracy is uncertain. Different individuals have varying biomechanics and activity execution styles, which could challenge the learned HAR model.

• Idealized Confusion Matrix: Presenting an "idealized classification" in the confusion matrix (Figure 1) rather than the actual one slightly diminishes the transparency of the HAR results. While the text discusses confusions, visualizing them would have provided stronger evidence for the stated limitations.

• Computational Cost for Wearable Devices: The paper mentions the computational cost as a limitation for wearable devices. Future work should explicitly consider edge AI solutions, model quantization, or pruning techniques to ensure that these AI algorithms can run efficiently on power-constrained, real-time exoskeleton controllers.

• Empirical Weight Adjustment: The reward function weights ($\omega_1, \omega_2, \omega_3$) were adjusted empirically. While common, this can be a tedious process. Future research could explore meta-learning or autoML techniques to optimize these hyperparameters more systematically.

  Overall, this paper provides a solid foundation for adaptive exoskeleton control. Overcoming the sim-to-real gap and ensuring the generalizability and real-time efficiency of the HAR component will be crucial steps in translating this promising research into widely applicable and impactful technologies.