1. Bibliographic Information

1.1. Title

Privacy-Preserving Action Recognition via Motion Difference Quantization

1.2. Authors

Sudhakar Kumawat and Hajime Nagahara from Osaka University, Japan.

1.3. Journal/Conference

The paper was published on arXiv (a preprint server) at 2022-08-04T05:03:27.000Z. While arXiv is a preprint server and not a peer-reviewed journal or conference, it is a highly influential platform for rapid dissemination of research in fields like computer science and physics. Papers published on arXiv are often subsequently submitted to and accepted by major conferences (e.g., CVPR, ICCV, ECCV, NeurIPS) or journals, which are known for their rigorous peer-review processes and significant impact in the computer vision and machine learning communities.

1.4. Publication Year

2022

1.5. Abstract

This paper introduces a novel, robust privacy-preserving encoder named BDQ (Blur, Difference, and Quantization) for human action recognition. The core problem addressed is the need for smart computer vision systems to understand surroundings for assistance without capturing sensitive personal information. BDQ operates in three sequential stages: first, a Blur module smoothens image edges; second, a Difference module computes pixel-wise intensity subtraction between consecutive frames to highlight motion and suppress high-level privacy attributes; and third, a Quantization module further removes low-level privacy attributes from the motion difference frames. The parameters of BDQ are optimized using an end-to-end adversarial training framework. This framework aims to maximize action recognition performance while simultaneously inhibiting the learning of privacy attributes by an adversary. Experimental results on three benchmark datasets (SBU, KTH, and IPN) demonstrate that BDQ achieves state-of-the-art trade-offs between action recognition accuracy and privacy preservation. Furthermore, the paper shows that BDQ's performance is comparable to that of DVS sensor-based event cameras, which are inherently privacy-preserving. The code is made publicly available.

1.6. Original Source Link

Official Source Link: https://arxiv.org/abs/2208.02459 PDF Link: https://arxiv.org/pdf/2208.02459v1.pdf Publication Status: This paper is a preprint available on arXiv.

2. Executive Summary

2.1. Background & Motivation

The ubiquitous deployment of smart computer vision (CV) systems in personal spaces, such as homes and public areas, has raised significant privacy and security concerns. While these systems offer benefits by understanding their surroundings for various tasks (e.g., elderly care, security, smart home control), they also inherently capture rich visual data that can contain highly sensitive personal information, including identity, gender, and specific activities. This creates a critical tension: users desire the utility of these systems but demand strong protection against the inadvertent or malicious capture and use of their private data.

The core problem the paper aims to solve is how to enable CV systems, specifically for human action recognition, to function effectively without compromising visual privacy. Existing solutions often either overly degrade image quality (impacting task performance) or are not robust enough against sophisticated adversaries. The paper identifies a gap in solutions that can offer a strong privacy-utility trade-off – maintaining high accuracy for the target task while robustly inhibiting the extraction of privacy attributes, all within a cost-effective and low space-time complexity framework suitable for consumer devices. The paper's innovative idea is to leverage motion-based feature extraction and quantization, combined with adversarial learning, to create a simple yet robust encoder that specifically addresses these challenges for action recognition.

2.2. Main Contributions / Findings

The paper makes several key contributions:

• Novel Privacy-Preserving Encoder (BDQ): It proposes a simple, robust, and modular privacy-preserving encoder called BDQ (Blur, Difference, Quantization) specifically designed for human action recognition. This encoder effectively allows crucial spatio-temporal cues necessary for action recognition to pass through, while simultaneously preserving privacy attributes.
• State-of-the-Art Trade-off: The BDQ encoder achieves a state-of-the-art trade-off between action recognition accuracy and privacy preservation on three benchmark datasets: SBU, KTH, and IPN. It outperforms previous methods like downsampling (Ryoo et al.) and UNet-like encoders (Wu et al.) by a significant margin.
• Low Space-Time Complexity: BDQ operates with significantly fewer parameters and lower computational cost compared to existing adversarial privacy-preserving models (e.g., Wu et al.'s UNet-like encoder), making it highly suitable for implementation on resource-constrained edge devices and consumer cameras.
• Parity with DVS Sensors: The paper demonstrates that the privacy-utility trade-off achieved by BDQ is comparable to that of Dynamic Vision Sensor (DVS)-based event cameras, which are considered inherently privacy-preserving due to their event-driven nature. This suggests BDQ can emulate the privacy benefits of specialized hardware using standard camera inputs.
• Extensive Analysis: The work includes a comprehensive analysis of BDQ, covering:

  • An ablation study to evaluate the contribution of each BDQ module (Blur, Difference, Quantization).

  • Demonstrations of BDQ's robustness against various adversarial models trying to learn or reconstruct privacy attributes.

  • Evidence of BDQ's ability to provide generalized spatio-temporal features usable by various action recognition networks.

  • A subjective evaluation (user study) to assess visual privacy perception by humans.

  • Discussion on the feasibility of hardware implementation (in supplementary material, but mentioned in summary).

    In essence, the paper provides a practical, efficient, and highly effective software-based solution for deploying privacy-aware action recognition systems, addressing a critical need in smart computer vision applications.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To fully understand this paper, a novice reader should be familiar with the following concepts:

• Computer Vision (CV) Systems: These are systems that enable computers to "see" and interpret digital images or videos. They involve tasks like object detection, image classification, human pose estimation, and action recognition.
• Privacy-Preserving AI/CV: This is a subfield focused on designing AI and CV systems that can perform their intended tasks while minimizing the exposure of sensitive personal information (e.g., identity, gender, specific actions) from the input data. This often involves techniques to transform or obfuscate the data before processing.
• Action Recognition: This is a specific computer vision task where the goal is to identify and classify human actions or activities from video sequences. For example, recognizing "walking," "running," or "waving." This task often requires processing both spatial (what's in the frame) and temporal (how things change over time) information.
• Adversarial Training: Inspired by Generative Adversarial Networks (GANs), adversarial training involves two competing neural networks:
  • A generator (or, in this paper, an encoder) that tries to produce data (or transformed data) that fools another network.
  • A discriminator (or, in this paper, a privacy attribute prediction model, also referred to as an adversary) that tries to distinguish between real and fake data, or in this context, tries to extract sensitive information from the transformed data. The two networks are trained simultaneously in a minimax game. In privacy-preserving contexts, the encoder aims to transform data to remove privacy attributes while retaining utility, and the adversary tries to learn the privacy attributes from the transformed data. The encoder is optimized to "fool" the adversary into not being able to extract privacy.
• Convolutional Neural Networks (CNNs): These are a class of deep neural networks widely used for analyzing visual imagery. They use convolutional layers that apply learnable filters to input data, detecting features like edges, textures, and patterns.
  • 2D CNNs: Process single images (e.g., ResNet-50 for image classification).
  • 3D CNNs: Extend 2D CNNs by adding a temporal dimension to the filters, allowing them to process video sequences and capture spatio-temporal features crucial for action recognition (e.g., 3D ResNet-50).
• Cross-Entropy Loss ($\mathcal{XE}$): A common loss function used in classification tasks. It measures the performance of a classification model whose output is a probability value between 0 and 1. It increases as the predicted probability diverges from the actual label. $ \mathcal{XE}(y, \hat{y}) = - \sum_{i=1}^{C} y_i \log(\hat{y}_i) $ Where $y$ is the one-hot encoded true label (1 for the correct class, 0 otherwise), $\hat{y}$ is the predicted probability distribution over $C$ classes.
• Entropy Function ($\mathcal{E}$): In information theory, entropy measures the uncertainty or unpredictability of a random variable. A high entropy value indicates high uncertainty. In this paper, it's used to encourage the privacy attribute prediction model ($P$) to be uncertain about privacy attributes from the BDQ output, meaning the BDQ encoder successfully suppressed these attributes. $ \mathcal{E}(P(E(V))) = - \sum_{i=1}^{C} p_i \log(p_i) $ Where $p_i$ is the probability assigned by the privacy model P to class $i$, given the BDQ output E(V).
• Gaussian Kernel ($G_{\sigma}$): A mathematical function used in image processing for blurring. It's a 2D bell-shaped curve that, when convolved with an image, blurs it by averaging pixel intensities based on a Gaussian distribution. The standard deviation $\sigma$ controls the degree of blurring (larger $\sigma$ means more blur). $ G_{\sigma}(x, y) = \frac{1}{2\pi\sigma^2} \exp\left(-\frac{x^2 + y^2}{2\sigma^2}\right) $ Where x, y are the distances from the kernel's center.
• Quantization: The process of mapping a continuous range of values to a finite set of discrete values. In image processing, this can reduce the number of distinct intensity levels, simplifying the image and removing fine details. This reduction of detail can contribute to privacy preservation.
• Heaviside Function ($\mathcal{U}$): A step function that outputs 0 for negative input and 1 for positive input. It's non-differentiable, making it problematic for gradient-based optimization in neural networks. $ \mathcal{U}(x) = \begin{cases} 0 & \text{if } x < 0 \ 1 & \text{if } x \ge 0 \end{cases} $
• Sigmoid Function ($\sigma(\cdot)$): A differentiable S-shaped function often used in neural networks to introduce non-linearity and to squish values into a range, typically (0, 1). It's used here as a differentiable approximation of the Heaviside function. $ \sigma(x) = \frac{1}{1 + e^{-x}} $
• DVS Sensors / Event Cameras: These are novel types of cameras that do not capture traditional frames. Instead, they asynchronously report events (pixel intensity changes) only when a pixel's brightness changes beyond a certain threshold. This event-driven nature makes them inherently privacy-preserving because they do not capture static scenes or fine-grained visual details, only outlines of moving objects.

3.2. Previous Works

Previous research in privacy-preserving computer vision, particularly for action recognition, can be broadly categorized:

• Early Hand-Crafted Methods: These approaches relied on explicit visual transformations like blurring, down-sampling, pixelation, or face/object replacement [2, 7, 23]. While intuitive, they require significant domain knowledge and often struggle to balance privacy with task accuracy. For example, simple down-sampling (as discussed in the paper's Figure 1) might preserve privacy but can destroy critical information needed for action recognition.
• Data-Driven Approaches using DNNs:
  • Adversarial Training for Privacy: Modern frameworks often leverage Deep Neural Networks (DNNs) and adversarial training [28]. The core idea, as adopted by this paper, is to train an encoder to inhibit sensitive attributes against an adversarial DNN (or privacy prediction model) whose goal is to extract those attributes, while simultaneously allowing attributes essential for the target CV task (e.g., action recognition).
    • General CV Tasks: Examples include privacy-preserving face detection [33], pose estimation [14, 32], and fall detection [3]. Pittaluga et al. [24] and Huang et al. [16] explored adversarial training for learning privacy-preserving encodings in general.
    • Action Recognition:
      • Low-Resolution Videos: Early works explored learning actions from low-resolution videos [9, 30, 29]. Ryoo et al. [30, 29] specifically learned image transformations to down-sample frames for action recognition, demonstrating that some tasks can be performed with significantly less visual detail. The paper uses Ryoo et al. [30] as a baseline.
      • Video Face Anonymization: Ren et al. [27] proposed adversarial training for video face anonymization, modifying videos to remove sensitive information while trying to maximize action recognition and using a discriminator to extract sensitive information.
      • UNet-like Encoders: Wu et al. [37, 36] proposed and compared multiple adversarial training frameworks for optimizing a UNet-like encoder (e.g., based on [18]). This encoder acts as a 2D conv-based frame-level filter to allow spatio-temporal attributes for action recognition while inhibiting privacy attributes against an ensemble of DNN adversaries. The paper uses Wu et al. [36] as a key baseline. A drawback noted by the authors is the need for an ensemble of adversaries for strong protection.
      • Self-Supervised Approaches: Concurrent to this work, Dave et al. [10] proposed a self-supervised framework for training a UNet-based privacy-preserving encoder.
• Optical Imaging Systems for Privacy: Some research explores hardware-level solutions to hide sensitive attributes at the point of capture. Examples include specialized camera systems that perform blurring and k-same face de-identification optically [25] or systems using coded aperture masks to enhance privacy [35, 6]. Wang et al. [35] proposed a lens-free coded aperture camera for privacy-preserving action recognition.

3.3. Technological Evolution

The evolution of privacy-preserving vision systems has progressed from simplistic, rule-based visual obfuscation to sophisticated, data-driven learning methods:

1. Early 2000s - Hand-crafted/Rule-based: Methods like simple blurring, pixelation, or face-swapping were applied post-capture. These were easy to implement but often led to a poor privacy-utility trade-off and were not robust against reconstruction.

2. Mid-2010s - Low-Resolution and Feature-based: Focus shifted to extracting information from inherently low-resolution data or specific, limited features (e.g., body pose keypoints) to avoid capturing full visual detail. This marked a step towards designing systems that capture less sensitive data upfront.

3. Late 2010s - Adversarial Learning: The rise of Generative Adversarial Networks (GANs) provided a powerful paradigm for privacy. Adversarial training allowed encoders to learn to transform data in a way that fooled privacy discriminators while preserving task-relevant information. This enabled more dynamic and context-aware privacy protection.

4. Early 2020s - Hardware-Software Co-Design and Specialized Sensors: Current trends include exploring optical solutions (e.g., coded apertures, specialized sensors like DVS cameras) that inherently limit information capture at the hardware level, as well as refining software encoders to be more efficient and robust.

   This paper's work fits within the early 2020s landscape, representing a refined data-driven adversarial learning approach that prioritizes efficiency and aims to achieve the privacy benefits of specialized hardware (DVS sensors) using standard camera inputs through intelligent software design.

3.4. Differentiation Analysis

Compared to the main methods in related work, the BDQ encoder's core differences and innovations are:

• Modular and Interpretable Design: Unlike complex UNet-like architectures (e.g., Wu et al. [36], Dave et al. [10]) that perform transformations in a single, opaque network, BDQ employs a simple, sequential three-module design (Blur, Difference, Quantization). Each module has a clear, intuitive role in contributing to privacy preservation and action recognition, making the design more interpretable and potentially easier to debug or adapt.

• Emphasis on Motion Features: The Difference module explicitly focuses on extracting motion features by computing pixel-wise intensity subtraction between consecutive frames. This is a deliberate design choice that enhances action recognition (as many state-of-the-art methods use similar temporal differencing) while naturally suppressing static, high-level privacy cues. This differentiates it from methods that might treat all pixels equally.

• Combined Multi-level Privacy Suppression: BDQ systematically addresses privacy at different levels:

  • Blur module: Smoothens edges, suppressing obvious spatial privacy features.
  • Difference module: Suppresses high-level spatial privacy cues by focusing on motion.
  • Quantization module: Removes low-level spatial privacy attributes by discretizing intensity values. This layered approach provides a more robust and comprehensive privacy-preserving mechanism.

• High Efficiency and Low Complexity: A significant advantage of BDQ is its extremely low space-time complexity. As shown in the results, it uses orders of magnitude fewer parameters and FLOPs than UNet-based encoders (e.g., Wu et al. [36]). This makes BDQ highly practical for deployment on edge devices with limited computational and memory resources, which is a crucial factor for widespread adoption of privacy-preserving systems.

• Direct Comparison to DVS Sensors: The paper explicitly benchmarks BDQ against DVS sensor-based event cameras, a specialized hardware solution known for its inherent privacy. Demonstrating comparable privacy-utility trade-off highlights BDQ's effectiveness as a software-based alternative or complement, making advanced privacy features accessible without requiring specialized hardware.

• Retention of Spatio-temporal Cues: The design of BDQ aims to preserve both spatial and temporal resolutions, which is critical for good action recognition performance. This contrasts with simple downsampling methods (Ryoo et al. [30]) that often destroy spatial resolution, leading to a significant drop in action accuracy (as illustrated in Figure 1).

  In summary, BDQ offers a more efficient, modular, and motion-centric approach to privacy-preserving action recognition, achieving superior privacy-utility trade-offs with significantly reduced computational overhead compared to prior adversarial learning methods.

4. Methodology

4.1. Principles

The core principle behind the BDQ encoder is to design a transformation pipeline that selectively filters out visual information deemed sensitive (privacy attributes) while retaining features critical for the target task (action recognition). This is achieved through a modular, sequential application of three fundamental image processing operations: Blur, Difference, and Quantization. The intuition is that privacy-sensitive information (like identity, facial features) often resides in fine spatial details and static scene content, whereas action-related information is heavily tied to motion and broader spatio-temporal patterns.

The theoretical basis for optimizing this pipeline lies in adversarial training. By framing the problem as a three-player non-zero sum game, the BDQ encoder learns to produce an output that simultaneously maximizes the action recognition model's performance and maximizes the uncertainty for a privacy attribute prediction model. This adversarial setup ensures that the learned parameters of BDQ strike an optimal balance, creating a privacy-utility trade-off that is data-driven and robust against potential adversaries. The use of differentiable approximations for operations like quantization allows the entire pipeline to be optimized end-to-end using standard gradient-based methods.

4.2. Core Methodology In-depth (Layer by Layer)

The BDQ encoder consists of three sequentially applied modules: Blur, Difference, and Quantization. Its parameters are optimized using a custom adversarial training framework.

The following figure (Figure 2 from the original paper) shows the system architecture:

该图像是示意图，展示了BDQ编码器架构及其用于隐私保护的人类动作识别的对抗训练框架。图中涉及的 $r o l l ( 1 , d i m = 1 )$ 操作用于沿时间维度平移帧数据。

Fig. 2: The BDQ encoder architecture and adversarial training framework for privacy-preserving action recognition. $r o l l ( 1 , d i m = 1 )$ operation shifts frames by one along the temporal dimension.

4.2.1. Blur Module

• Purpose: The Blur module's primary goal is to smoothen the edges in the input frames. This is crucial for two reasons:
  1. It helps suppress obvious privacy features that might be present at sharp spatial edges.
  2. It prepares the input for the subsequent Difference module by reducing noise and potentially misleading fine details, ensuring that the motion differences are more robust.
• Implementation: Given an input video $V = \{ v _ { i } | i = 1 , 2 , . . , t \}$, where $v_i$ is the $i$-th frame and $t$ is the total number of frames, the blurred frame B _ { v _ { i } } is obtained by convolving the input frame $v_i$ with a 2D Gaussian kernel $G _ { \sigma }$. The mathematical formulation for the Gaussian kernel is: $ G _ { \sigma } = \frac { 1 } { 2 \pi \sigma ^ { 2 } } \exp \left( - \frac { x ^ { 2 } + y ^ { 2 } } { 2 \sigma ^ { 2 } } \right) $ Where:
  • (x, y) represents the coordinates relative to the center of the kernel.
  • $\sigma$ (sigma) is the standard deviation of the Gaussian distribution. It is a learnable parameter during the adversarial training process and controls the degree of blurring. A larger $\sigma$ results in more blurring.
  • $2\pi\sigma^2$ is a normalization constant ensuring the kernel sums to 1. The convolution operation is denoted as $B _ { v _ { i } } = G _ { \sigma } * v _ { i }$.
• Details: The window size of the Gaussian kernel is fixed at $5 \times 5$. This small window size is chosen to stabilize training and prevent the loss of essential spatial features for action recognition.

4.2.2. Difference Module

• Purpose: This module operates on two consecutive blurred frames from the video. It performs pixel-wise intensity subtraction to highlight motion features and further suppress high-level spatial privacy cues.
• Implementation: Given two consecutive blurred frames, $B_{v_i}$ and $B_{v_j}$, the output of the Difference module, $D(B_{v_i}, B_{v_j})$, is simply their pixel-wise difference: $ D ( B _ { v _ { i } } , B _ { v _ { j } } ) = B _ { v _ { i } } - B _ { v _ { j } } $ Where:
  • $B_{v_i}$ is the $i$-th blurred frame.
  • $B_{v_j}$ is the $j$-th blurred frame, typically the frame immediately preceding $B_{v_i}$ (i.e., $j = i-1$).
• Role in Privacy & Utility:
  • Action Recognition: By highlighting intensity changes between frames, this module effectively captures temporal motion cues. Many state-of-the-art action recognition methods leverage similar temporal differencing in feature space to improve performance. This brings out the direction of motion.
  • Privacy Preservation: By focusing only on changes, static background elements and high-level spatial attributes (like detailed facial features or clothing patterns) that remain constant between frames are largely suppressed. This contributes significantly to privacy.
• Details: This module has no learnable parameters. For online applications, it requires storing the previous frame's blurred version for subtraction.

4.2.3. Quantization Module

• Purpose: Even after Blur and Difference, some low-level spatial privacy cues might persist. The Quantization module addresses this by applying a pixel-wise quantization function to the motion difference frames, effectively removing these remaining fine details.
• Implementation: The module discretizes the continuous input values (motion difference frame pixels) into a finite set of $N$ values. The original, non-differentiable formulation for quantization is: $ y = \sum _ { n = 1 } ^ { N - 1 } \mathcal { U } ( x - b _ { i } ) $ Where:
  • $y$ is the discrete output value (the quantized pixel intensity).
  • $x$ is the continuous input value (a pixel intensity from the motion difference frame).
  • $b _ { i } = \{ 0 . 5 , 1 . 5 , 2 . 5 , \ldots , N - 1 . 5 \}$ are the quantization bin boundaries or thresholds. The paper specifies $N = 2^k$, where $k$ is the number of bits. The authors fix the number of $b_i$ values to be 15.
  • $\mathcal { U } ( \cdot )$ is the Heaviside function, which outputs 1 if its argument is non-negative, and 0 otherwise. The sum counts how many thresholds $x$ exceeds, effectively mapping $x$ to a discrete integer bin.
• Differentiable Approximation: Since the Heaviside function is non-differentiable (gradients are zero almost everywhere), it's unsuitable for backpropagation. Following common practices [38, 33], the Heaviside function is replaced with a differentiable sigmoid function $\sigma ( \cdot )$: $ \sum _ { n = 1 } ^ { N - 1 } \sigma ( H ( x - b _ { i } ) ) $ Where:
  • $H$ is a scalar hardness term. This parameter controls the steepness of the sigmoid function. A higher $H$ makes the sigmoid approximate the Heaviside function more closely (steeper transition). The paper sets $H=5$ for all datasets.
  • The parameters learned in this module are the $b_i$ values, which define the quantization intervals.
• Details: The input to the Quantization module is first normalized to have values between 0 and 15, matching the range determined by the 15 fixed bin boundaries. The output is allowed to be non-integer, as no hardware constraints enforce integer outputs.

4.2.4. Training BDQ Encoder

The training framework is a three-player non-zero sum game involving three components:

1. Encoder ($E$): The BDQ encoder, which contains the learnable parameters ($\sigma$ from Blur, and $b_i$ from Quantization).

2. Target Task Predictor ($T$): A 3D CNN (specifically, a 3D ResNet-50) responsible for predicting the target action attribute from the BDQ output.

3. Privacy Attribute Predictor ($P$): A 2D CNN (specifically, a 2D ResNet-50) responsible for predicting privacy attributes from the BDQ output. This acts as the adversary.

   The goal of $E$ is to maximize the likelihood of action attributes (as measured by $T$) while simultaneously maximizing the uncertainty in privacy attributes (as measured by $P$). This is achieved through an iterative, two-step training procedure:

Step 1: Train Encoder ($E$) and Target Task Predictor ($T$) In this step, the parameters of the Privacy Attribute Predictor ($P$) are fixed. The BDQ encoder ($E$) and the Target Task Predictor ($T$) are trained together. The loss function is designed to increase action recognition accuracy while making it harder for $P$ to predict privacy attributes (i.e., maximizing the entropy of $P$'s output).

$ \mathcal { L } ( V , \theta _ { E } , \theta _ { T } ) = \mathcal { X } \mathcal { E } ( T ( E ( V ) ) , L _ { a c t i o n } ) - \alpha \mathcal { E } ( P ( E ( V ) ) ) $

Where:

• $V$ represents the input video.
• $\theta _ { E }$ denotes the learnable parameters of the BDQ encoder ($E$).
• $\theta _ { T }$ denotes the learnable parameters of the Target Task Predictor ($T$).
• $\mathcal { X } \mathcal { E } ( \cdot , \cdot )$ is the cross-entropy loss function. This term aims to minimize the error in predicting action labels, thus improving action recognition accuracy. T(E(V)) is the action prediction from the Target Task Predictor given the BDQ output. $L_{action}$ is the ground-truth action label.
• $\mathcal { E } ( \cdot )$ is the entropy function. This term aims to maximize the uncertainty of the Privacy Attribute Predictor ($P$) when it tries to predict privacy attributes from the BDQ output E(V). A higher entropy means $P$ is more uncertain, implying $E$ has successfully suppressed privacy attributes.
• $\alpha$ is the adversarial weight. This is a hyperparameter that controls the trade-off between prioritizing action recognition accuracy (first term) and maximizing privacy protection (second term). A higher $\alpha$ puts more emphasis on privacy preservation. The values of $\alpha$ are set to 2, 1, and 8 for SBU, KTH, and IPN datasets, respectively.

Step 2: Train Privacy Attribute Predictor ($P$) In this step, the parameters of the BDQ encoder ($E$) and the Target Task Predictor ($T$) are fixed. Only the Privacy Attribute Predictor ($P$) is trained. The loss function for $P$ is designed to maximize its ability to correctly predict privacy attributes from the BDQ output. This makes $P$ a stronger adversary against $E$.

$ \mathcal { L } ( V , \theta _ { P } ) = \mathcal { X } \mathcal { E } ( P ( E ( V ) ) , L _ { p r i v a c y } ) $

Where:

• $\theta _ { P }$ denotes the learnable parameters of the Privacy Attribute Predictor ($P$).
• $L_{privacy}$ is the ground-truth privacy label. This step essentially trains $P$ to become a better adversary, which in turn forces $E$ to learn more effective privacy-preserving transformations in the subsequent Step 1. These two steps are iterated until a satisfactory privacy-utility trade-off is achieved.

5. Experimental Setup

5.1. Datasets

The experiments were conducted on three benchmark datasets commonly used in action recognition research, each providing distinct scenarios for both action and privacy attribute prediction:

• SBU Kinect Interaction Dataset [39]:

  • Source & Characteristics: This dataset features two-person interactions recorded at 15 frames per second (fps). It includes eight types of interactions: approaching, departing, pushing, kicking, punching, exchanging objects, hugging, and shaking hands. The dataset is organized into 21 sets, each corresponding to a pair of actors performing all eight interactions. To streamline, these 21 sets were combined into 13 distinct actor-pair sets (e.g., if actor 1 and actor 2 appear in multiple sets with swapped roles, they are grouped into one actor-pair class).
  • Task & Privacy:
    • Target Task (Action Recognition): Classifying the video into one of the eight interaction types.
    • Privacy Label Prediction Task: Recognizing the specific actor-pair among the 13 distinct actor-pairs.
  • Why chosen: It's a standard dataset for two-person interaction, making it suitable for evaluating both interaction recognition and the privacy of individual participants.

• KTH Dataset [31]:

  • Source & Characteristics: A video-based action recognition dataset recorded at 25 fps. It comprises 25 distinct actors, each performing six basic actions: walk, jog, run, box, hand-wave, and hand-clap. The videos capture these actions under diverse settings, including outdoor, outdoor with scale variations, outdoor with different clothing, and indoor environments.
  • Task & Privacy:
    • Target Task (Action Recognition): Classifying the video into one of the six action classes.
    • Privacy Label Prediction Task: Recognizing the identity of the actor among the 25 distinct actor identities.
  • Why chosen: A classic dataset for single-person action recognition, offering a clear distinction between action and identity, suitable for evaluating identity-based privacy.

• IPN Hand Gesture Dataset [4]:

  • Source & Characteristics: This dataset focuses on video-based hand gestures, recorded at 30 fps. It includes 50 actors performing 13 common hand gestures relevant to touch-less screen interaction (e.g., pointing with one finger, clicking, throwing gestures, zooming).

  • Task & Privacy:

    • Target Task (Action Recognition): Classifying the video into one of the 13 hand gesture classes.
    • Privacy Label Prediction Task: Recognizing the gender (male/female, 2 classes) of the actors.

  • Why chosen: This dataset allows evaluation of fine-grained motor actions (gestures) and a different type of privacy attribute (gender), providing diversity beyond identity.

    These datasets were chosen because they are well-established benchmarks, allowing for direct comparison with previous works. They also provide varied target tasks (two-person interaction, full-body actions, hand gestures) and privacy attributes (actor-pair, individual identity, gender), enabling a comprehensive evaluation of the BDQ encoder's versatility and robustness.

5.2. Evaluation Metrics

The paper primarily uses Accuracy as the evaluation metric for both the target action recognition task and the privacy label prediction task.

• Accuracy:

  1. Conceptual Definition: Accuracy is a fundamental metric in classification that measures the proportion of total predictions that were correct. It provides a general assessment of how well a model performs across all classes. In the context of this paper, it is used to quantify how well the action recognition model identifies actions and how well the privacy model identifies privacy attributes (or, conversely, how poorly it identifies them, which implies good privacy preservation by the encoder).

  2. Mathematical Formula: $ \text{Accuracy} = \frac{\text{Number of Correct Predictions}}{\text{Total Number of Predictions}} $

  3. Symbol Explanation:

     • Number of Correct Predictions: The count of instances where the model's predicted class label perfectly matches the true (ground-truth) class label.

     • Total Number of Predictions: The total number of instances (e.g., video clips or frames) for which the model made a prediction.

       In the paper, for action recognition, clip-1 crop-1 accuracy is reported, meaning the accuracy is calculated based on a single video clip and a single spatial crop from that clip. For privacy prediction, the softmax outputs from the 2D ResNet-50 model are averaged over $t$ frames before calculating the average accuracy.

5.3. Baselines

The paper compares its BDQ method against two prominent previous works in privacy-preserving action recognition, along with a conceptual comparison to DVS sensor-based event cameras:

• Ryoo et al. [30]:

  • Method: This approach uses a down-sampling module as the degradation encoder. It learns image transformations to down-sample high-resolution action videos into multiple low-resolution videos. These transformations (sub-pixel translation, scaling, rotation, affine transformations) are optimized for action recognition.
  • Representativeness: It represents early works that focused on privacy through reduced spatial resolution. It's a straightforward and intuitive method to limit visual information.
  • Comparison in Paper: The paper evaluates different low spatial resolutions ($112 \times 112$, $56 \times 56$, $28 \times 28$, $14 \times 14$, $7 \times 7$, $4 \times 4$) and observes that while privacy prediction accuracy drops with increased downsampling, action recognition accuracy often drops at an even faster, detrimental rate (see Figure 3, row 1).

• Wu et al. [36]:

  • Method: This work uses a UNet-like network [18] as the degradation encoder. This encoder acts as a 2D convolution-based frame-level filter that transforms each frame into a feature map of the same shape. The encoder is trained using adversarial training (similar to the current paper's framework) to allow spatio-temporal features for action recognition while inhibiting privacy attributes against an ensemble of DNN adversaries.
  • Representativeness: This is a state-of-the-art data-driven adversarial learning approach for privacy-preserving action recognition, which directly addresses the privacy-utility trade-off using deep neural networks. Its UNet-like encoder is a common architecture for image-to-image transformations.
  • Comparison in Paper: The paper adopts Wu et al.'s adversarial training methodology (with some modifications, like resetting the privacy model periodically) for fair comparison. It finds that Wu et al. [36] performs significantly better than Ryoo et al. [30] but is still surpassed by BDQ in terms of the privacy-utility trade-off (see Figure 3, row 1). Crucially, BDQ also offers significantly lower space-time complexity (see Table 1).

• DVS Sensor-based Event Cameras:

  • Method: These are specialized hardware cameras that do not capture traditional frames. Instead, they detect and output events (pixel intensity changes) asynchronously. This event-driven nature naturally preserves privacy by only showing motion outlines and not static scenes or fine textures.

  • Representativeness: DVS sensors are considered an inherently privacy-preserving technology at the hardware level. Comparing BDQ (a software solution) to DVS provides a strong benchmark for the effectiveness of BDQ's privacy preservation capabilities.

  • Comparison in Paper: The paper converts videos to synthetic events (using [11]) to simulate DVS output and then evaluates action and privacy accuracy under different event thresholds. It concludes that BDQ achieves a trade-off comparable to that of DVS sensors (see Figure 8).

    These baselines were chosen to represent a spectrum of approaches: simple resolution reduction, complex DNN-based adversarial learning, and specialized hardware. This allows the paper to comprehensively demonstrate the advantages of BDQ across different dimensions of performance and practicality.

5.4. Implementation

5.4.1. Adversarial Training

The adversarial training framework comprises three main components:

• BDQ Encoder ($E$): The proposed Blur, Difference, Quantization modules.
• Action Recognition Model ($T$): A 3D ResNet-50 network. 3D CNNs are suitable for video data as they process both spatial and temporal information.
• Privacy Attribute Prediction Model ($P$): A 2D ResNet-50 network. 2D CNNs are used as the privacy model operates on individual frames (or difference frames) from the BDQ output.

Initialization:

• The 3D ResNet-50 for $T$ is initialized with Kinetics-400 pre-trained weights. Kinetics-400 is a large dataset for human action recognition, providing good general spatio-temporal features.
• The 2D ResNet-50 for $P$ is initialized with ImageNet pre-trained weights. ImageNet is a large image classification dataset, providing strong general image features useful for privacy attribute prediction.

Input Sequence Generation:

• For training, consecutive frames are densely sampled from the input video to form an input sequence.
  • SBU dataset: $t=16$ frames.
  • KTH and IPN datasets: $t=32$ frames.

Data Augmentation:

• Spatial Data Augmentation: For each input sequence, a random spatial position and a scale are chosen to perform multi-scale cropping. The scale is picked from the set $\{ 1 , { \frac { 1 } { 2 ^ { 1 / 4 } } } , { \frac { 1 } { 2 ^ { 3 / 4 } } } , { \frac { 1 } { 2 } } \}$. This helps the model generalize to variations in object size and position.
• Output Size: The final output of this augmentation process is an input sequence with a size of $224 \times 224$ pixels per frame.

Training Flow:

1. The augmented input sequence is passed through the BDQ encoder.
2. The BDQ output is then simultaneously fed to the 3D ResNet-50 (for action recognition) and the 2D ResNet-50 (for privacy prediction).
3. The optimization of parameters for $E$, $T$, and $P$ follows the adversarial training framework described in Section 3.2.
   • The adversarial weight $\alpha$ is critical for balancing action utility and privacy. It is set to:
     • $\alpha = 2$ for SBU
     • $\alpha = 1$ for KTH
     • $\alpha = 8$ for IPN
   • The scalar hardness term $H$ for the Quantization module is set to 5 for all datasets, controlling the steepness of the sigmoid approximation.

Optimization Details:

• Epochs: The adversarial training is performed for 50 epochs.
• Optimizer: SGD (Stochastic Gradient Descent).
• Learning Rate (lr): 0.001.
• Scheduler: Cosine annealing scheduler, which gradually decreases the learning rate following a cosine curve, often leading to better convergence.
• Batch Size: 16.

5.4.2. Validation

After the BDQ encoder ($E$) has been trained, its parameters are frozen for validation. Separate instances of the 3D ResNet-50 and 2D ResNet-50 models are then instantiated and trained for their respective tasks using the BDQ output as input.

Validation Model Initialization:

• A new 3D ResNet-50 (for action recognition) is initialized with Kinetics-400 pre-trained weights.
• A new 2D ResNet-50 (for privacy label prediction) is initialized with ImageNet pre-trained weights.

Training for Validation:

• These newly initialized models are trained using the output of the frozen BDQ encoder on the training set videos.
• Both networks are trained for 50 epochs using SGD optimizer, $lr = 0.001$, cosine annealing scheduler, and batch size 16.

Inference for Validation:

• For each input video, consecutive $t$ frames are sampled without any random shift.
  • SBU: $t=16$ frames.
  • KTH and IPN: $t=32$ frames.
• Each frame in the sequence is center cropped (without scaling) to a square region of $224 \times 224$.

Reporting Metrics:

• Action Recognition: The clip-1 crop-1 accuracy is reported using the generated sequence on the 3D ResNet model. This means that for each video, one clip is taken, one central crop is applied, and the accuracy is computed.
• Privacy Prediction: The softmax outputs from the 2D ResNet-50 model are averaged over all $t$ frames in the sequence, and then the average accuracy is reported. This gives a more robust estimate of the privacy model's ability to identify sensitive attributes from the full sequence.

6. Results & Analysis

6.1. Core Results Analysis

The paper's core results demonstrate BDQ's superior performance in achieving a privacy-utility trade-off compared to previous methods. This is primarily visualized in the trade-off curves where higher action recognition accuracy and lower privacy prediction accuracy are desired.

The following figure (Figure 3 from the original paper) presents the performance trade-off and learned quantization steps:

该图像是图表，展示了在三个数据集（SBU、KTH和IPN）上的性能权衡（第一行）和学习到的量化步骤（第二行）。不同的方法在动作、身份、手势和性别准确率上的表现被比较，BDQ方法在准确性上表现出色。

Fig. 3: Performance trade-off (row 1) and learned quantization steps (row 2) on the three datasets: SBU, KTH, and IPN.

Comparison with Baselines (Figure 3, row 1):

• Ryoo et al. [30] (Downsampling): The points for Ryoo et al. show that as the downsampling rate increases (indicated by larger markers, leading to lower spatial resolutions like $4 \times 4$), the privacy label prediction accuracy generally drops. However, the action recognition accuracy drops even more significantly. This indicates a poor trade-off, as a minimal gain in privacy comes at a high cost to task performance. This supports the paper's initial claim that downsampling can be detrimental for action recognition (Figure 1).

• Wu et al. [36] (UNet-like Encoder): This method demonstrates a much better trade-off than Ryoo et al. across all datasets. It can achieve lower privacy prediction accuracies while maintaining higher action recognition performance. This highlights the effectiveness of data-driven adversarial training over simple hand-crafted degradations.

• BDQ Encoder (Proposed): BDQ consistently surpasses both Ryoo et al. and Wu et al. by a significant margin across all three datasets (SBU, KTH, IPN). Its trade-off curve is closer to the ideal top-left corner (high action accuracy, low privacy accuracy) than any other method. This validates the effectiveness of BDQ's modular design and its adversarial training approach in finding a better balance between utility and privacy.

Space-Time Complexity (Table 1): The following are the results from Table 1 of the original paper:

This table provides a critical justification for BDQ's practical applicability.

• Parameters: BDQ has only 16 parameters, which are the $\sigma$ for Blur and the $b_i$ values for Quantization. In stark contrast, Wu et al.'s UNet-like encoder has 1.3 Million parameters.

• Size: BDQ's model size is 3.4 KB, while Wu et al.'s is 3.8 MB.

• FLOPs (Floating Point Operations): BDQ requires 120.4 Million FLOPs, whereas Wu et al. requires 166.4 Billion FLOPs.

  The dramatic difference in complexity indicates that BDQ is orders of magnitude more efficient in terms of model size and computational cost. This makes BDQ highly suitable for deployment on resource-constrained edge devices (e.g., smart cameras, smartphones) where memory, computation, and power budgets are limited, a key goal outlined in the introduction.

Learned Quantization Steps (Figure 3, row 2): This row visualizes the learned quantization interval values for each dataset. These plots show how the BDQ encoder, through adversarial training, adjusts the quantization bin boundaries ($b_i$) to optimize the privacy-utility trade-off. The specific patterns of these learned values reflect the data distribution and the chosen adversarial weights ($\alpha$) for each dataset, contributing to the observed performance.

6.2. Ablation Studies / Parameter Analysis

The paper conducts an ablation study to understand the contribution of each module (Blur, Difference, Quantization) within the BDQ encoder and analyzes the effect of the adversarial parameter $\alpha$.

The following figure (Figure 4 from the original paper) shows the results of the ablation study and the effect of the adversarial parameter $\alpha$:

该图像是图表，左侧展示了消融研究的结果，其中大点表示更高的 eta 值，右侧则呈现了对抗参数 eta 在量化步骤上的影响。

Fig. 4: Left- Results of the ablation study. Here, a bigger $\\cdot$ corresponds to a higher value of $\\alpha$ . RightEffect of the adversarial parameter $\\alpha$ on the quantization steps.

Ablation Study on BDQ Components (Figure 4, Left): This plot shows the privacy-utility trade-off for various combinations of BDQ modules on the SBU dataset, using a fixed adversarial weight $\alpha=2$.

• Individual Modules ('B', 'D', 'Q'):
  • 'B' (Blur only), 'Q' (Quantization only), and 'B+Q' (Blur + Quantization) combinations show very little effect in preserving privacy. Their performance is close to the 'Original Video' baseline, meaning the privacy prediction accuracy remains high. This suggests that these modules alone are insufficient for strong privacy protection.
  • 'D' (Difference only): Interestingly, the Difference module alone achieves the highest action recognition accuracy among all combinations (including 'Original Video'). This highlights its crucial role in extracting effective temporal features for action recognition. However, it still offers minimal privacy protection, with privacy accuracy remaining high.
• Combinations for Privacy:
  • $'D+Q'$ (Difference + Quantization): A significant drop in privacy accuracy is observed when Difference and Quantization are used together. This indicates that Quantization effectively removes the low-level privacy attributes that might still be present in the motion difference frames.
  • $'B+D+Q'$ (Full BDQ): When all three modules are combined, the privacy accuracy drops drastically, achieving the best privacy protection while maintaining a strong action recognition performance. This confirms that all three modules play a synergistic role, with Blur further suppressing initial spatial privacy cues, Difference extracting motion and suppressing high-level static cues, and Quantization removing residual low-level details.

Effect of Adversarial Parameter $\alpha$ (Figure 4, Right, and associated text): The adversarial weight $\alpha$ controls the balance between action recognition and privacy preservation during adversarial training.

• $\alpha = 0$ (No Adversarial Training): When $\alpha$ is zero, the model only optimizes for action recognition and does not consider privacy. The results show very little drop in privacy accuracy, implying that without explicit adversarial pressure, the encoder does not learn to suppress privacy attributes.
• Increasing $\alpha$: As $\alpha$ increases, both action recognition accuracy and privacy prediction accuracy begin to fall. This is an expected trade-off: pushing harder for privacy necessarily means some degradation in utility. The figure shows that action recognition accuracy falls more sharply than privacy accuracy initially, indicating the sensitivity of action performance to aggressive privacy measures.
• Effect on Quantization Steps: The right plot in Figure 4 illustrates how the learned quantization values change with increasing $\alpha$. A higher\alphaleads to increased quantization (fewer, coarser bins, or more aggressive binning), which in turn causes both action and privacy accuracies to drop. This directly links the hyperparameter $\alpha$ to the learned behavior of the Quantization module and its impact on the privacy-utility trade-off.

6.3. Strong Privacy Protection

A crucial aspect of any privacy-preserving model is its robustness against unseen adversaries. The paper evaluates BDQ's ability to protect privacy against a diverse set of state-of-the-art image classification networks.

The following figure (Figure 5 from the original paper) shows the actor-pair accuracy on various image classification networks:

该图像是一个柱状图，展示了不同图像分类网络在演员配对准确率上的表现。图中的蓝色柱子代表原始视频的准确率，而橙色柱子表示经过降质处理的视频的准确率。对比显示，在大多数网络中，原始视频的准确率显著高于降质视频。

Fig. 5: Actor-pair accuracy on various image classification networks.

• Experiment Setup: The pre-trained BDQ encoder (from Section 4) is used to generate degraded videos. Ten different state-of-the-art image classification networks (e.g., ResNet-50, MobileNet-v3, etc.) are then trained to predict actor-pair labels from these degraded videos on the SBU dataset. For comparison, these networks are also trained on the original videos to establish baselines. All networks are initialized with ImageNet pre-trained weights.
• Results (Figure 5):
  • The blue bars (Original Video) show high actor-pair accuracy for all networks, indicating that privacy attributes are easily identifiable from raw video.
  • The orange bars (Degraded Video - BDQ output) consistently show significantly lower actor-pair accuracy across all ten adversary networks. This demonstrates that BDQ effectively protects privacy information against a wide range of powerful, pre-trained CNN architectures.
  • The best performing adversary (ResNet-50) achieved only 34.18% accuracy on BDQ output, while the worst (MobileNet-v3) achieved 25.46%. This is much lower than the accuracies on original video, indicating robust protection.
  • Crucially, BDQ was only exposed to ResNet-50 during its own adversarial training. Its consistent performance against other unseen adversary architectures (e.g., VGG, Inception, DenseNet) highlights its generalization capability in privacy protection.

6.4. Generalized Spatio-temporal Features

Beyond privacy protection, a good privacy-preserving encoder must allow the target task (action recognition) to be performed effectively by various downstream models.

The following figure (Figure 6 from the original paper) shows the action recognition accuracy on various action recognition networks:

该图像是图表，展示了不同动作识别网络在原始视频和降级视频上的动作识别准确率。纵轴表示准确率（%），横轴为不同的3D网络模型，包括3D ResNet50、3D ResNet101、3D ResNext101、3D MobileNetv2和3D ShuffleNetv2。

Fig. 6: Action recognition accuracy on various action recognition networks.

• Experiment Setup: Similar to the privacy robustness test, the pre-trained BDQ encoder generates degraded videos. Five different 3D CNNs (e.g., 3D ResNet50, 3D ResNext-101, 3D MobileNetv2, 3D ShuffleNet-v2) are then separately trained to predict action classes from these BDQ outputs on the SBU dataset. Baselines are established by training these networks on original videos. All networks are initialized with Kinetics-400 pre-trained weights.
• Results (Figure 6):
  • The orange bars (Degraded Video - BDQ output) show that all 3D CNNs achieve high action recognition accuracy when trained on BDQ's output.
  • The best performing model (3D ResNext-101) achieved 85.1% accuracy, with the worst (3D ShuffleNet-v2) achieving 81.91%.
  • Importantly, these accuracies are only marginally lower than their corresponding baselines trained on original videos (blue bars). This demonstrates that BDQ successfully retains sufficient spatio-temporal information for various action recognition networks to perform well, proving its utility and generalizability for the target task.

6.5. Robustness to Reconstruction Attack

A sophisticated attacker might try to reverse the privacy-preserving transformation to reconstruct the original sensitive information. The paper investigates BDQ's robustness against such reconstruction attacks.

The following figure (Figure 7 from the original paper) visualizes reconstruction results:

该图像是插图，展示了不同参数 $\alpha$ 值下的重构结果与原始帧的对比。图像分为两行，第一行展示了原始帧及使用 BDQ 模块（$\alpha = 0$）的效果，第二行则展示了 Rec. BDQ 模块在 $\alpha = 0$ 和 $\alpha = 2$ 下的重构效果。该图旨在证明 BDQ 编码器在隐私保护与动作识别之间的有效权衡。

Fig. 7: Visualization of reconstruction results for $\\alpha = 0$ and 2.

• Experiment Setup: A 3D UNet model [8] is trained for 200 epochs on the SBU dataset to act as a reconstruction network. Its input is degraded videos from the BDQ encoder, and its output is trained to match the original videos (ground truth).

  • Case 1: Untrained BDQ encoder ($\alpha = 0$): The 3D UNet is trained to reconstruct from BDQ output where the BDQ encoder was not trained adversarially for privacy (i.e., $\alpha = 0$). In this scenario, BDQ merely applies blur, difference, and quantization without privacy-centric optimization.
  • Case 2: Trained BDQ encoder ($\alpha = 2$): The 3D UNet is trained to reconstruct from BDQ output where the BDQ encoder was fully trained with $\alpha = 2$ (prioritizing privacy).

• Results (Figure 7):

  • Untrained BDQ ($\alpha = 0$, $Rec. BDQ, alpha=0$ column): When the input to the reconstruction network comes from an untrained BDQ encoder, the network can successfully reconstruct the original video with satisfactory accuracy. This means if BDQ is not adversarially trained for privacy, its transformations are largely reversible.
  • Trained BDQ ($\alpha = 2$, $Rec. BDQ, alpha=2$ column): However, when the input comes from a trained BDQ encoder (optimized for privacy with $\alpha = 2$), the reconstruction is significantly poor. The visual details are heavily lost, and the privacy information is still preserved. For example, facial features and fine textures are largely unrecoverable.

• Conclusion: This demonstrates BDQ's resistance to reconstruction attacks. The adversarial training process effectively scrambles or removes privacy-sensitive information in a way that is difficult for a reconstruction network to reverse, even with full knowledge of the BDQ encoder and a large paired training set.

6.6. Subjective Evaluation

Beyond quantitative metrics and algorithmic robustness, visual privacy also depends on human perception. The paper conducted a user study to assess whether the BDQ output provides visual privacy against the human visual system.

• Experiment Setup:
  • Dataset: Videos from the SBU dataset, processed by the BDQ encoder trained with $\alpha = 2$.
  • Task: Participants were shown a BDQ output video (featuring two interacting persons) and had to select the identities of both actors from seven cropped face options.
  • Participants: 26 individuals.
  • Questions: 60 questions in total.
• Baselines for Random Chance:
  • Selecting both actors correctly by chance: 4.76% (1/7 * 1/7, roughly).
  • Selecting at least one actor correctly by chance: 52.38%.
• Results:
  • Both Actors Correctly Recognized: Participants achieved 8.65% accuracy. This is higher than random chance (4.76%), suggesting humans can still infer some identity information, but it's very low and far from perfect recognition.
  • At Least One Actor Correctly Recognized: Participants achieved 65.64% accuracy. This is higher than random chance (52.38%), again indicating some level of human inference, but the overall difficulty is high given the degraded visual quality.
• Conclusion: While not perfect, the low accuracy for correctly identifying both actors suggests that the BDQ output offers a reasonable degree of visual privacy against human observers, making it difficult to confidently identify individuals. This subjective evaluation complements the objective adversarial robustness tests.

6.7. Comparison with Event Camera

Dynamic Vision Sensors (DVS) or event cameras are proposed as inherently privacy-preserving solutions due to their event-driven nature. The paper compares BDQ's performance trade-off with that of a DVS sensor.

The following figure (Figure 8 from the original paper) shows example event frames, event threshold, action recognition accuracy, and actor-pair recognition accuracy on SBU:

该图像是示意图，展示了在不同阈值下（th = 0.4, 0.8, 1.2, 1.6, 2.0, 2.4）的人体动作识别准确率。每个阈值下显示的准确率分别为93.54%、92.47%、90.32%、87.09%、86.02%和82.79%，而演员对识别准确率在同一阈值下为73.99%、58.33%、47.84%、46.23%、40.12%和34.87%。

Fig.8: Example event frames (Row 1), event threshold (Row 2), action recognition accuracy (Row 3) and actor-pair recognition accuracy (Row 4) on SBU.

• DVS Sensor Principle: Unlike traditional cameras, DVS sensors only report events (pixel intensity changes) when a pixel's brightness changes beyond a fixed threshold. This means static parts of a scene are not captured, only outlines of motion. This leads to inherent privacy but can make traditional frame-based processing challenging.
• Experiment Setup:
  • Conversion to Events: Videos from the SBU dataset are converted into synthetic events using the method from [11]. This allows simulating DVS output from standard video and controlling the pixel-level threshold.
  • Threshold Effect: A higher threshold means fewer events are registered (only large intensity changes), leading to sparser, more privacy-preserving output but potentially less information for action recognition. Lower thresholds capture more events.
  • Processing Events: The generated events are converted into event frames (using [11]) and then used to train a 3D ResNet-50 for action recognition and a 2D ResNet-50 for actor-pair recognition (privacy).
  • Initialization & Training: Identical to the BDQ validation setup (Section 4).
• Results (Figure 8):
  • Row 1 shows example event frames at different thresholds, illustrating how higher thresholds lead to sparser visual information.
  • Row 2 lists the event thresholds used.
  • Row 3 (Action Recognition Accuracy) and Row 4 (Actor-pair Recognition Accuracy) show the trade-off for DVS sensors. As the threshold increases:
    • Action recognition accuracy drops (e.g., from 93.54% at th=0.4 to 82.79% at th=2.4).
    • Actor-pair recognition accuracy also drops (e.g., from 73.99% at th=0.4 to 34.87% at th=2.4).
  • Comparison to BDQ: The paper specifically notes that the trade-off achieved by the DVS sensor at a threshold value of 2.4 is close to the trade-off achieved by the BDQ encoder with\alpha = 2$$ (which was ~86.5% action accuracy and ~34% privacy accuracy for SBU from Figure 3, row 1).
• Conclusion: This comparison is a strong finding. It suggests that BDQ, a software-based encoder operating on traditional video, can achieve a privacy-utility trade-off similar to that of DVS sensors, which are specialized hardware inherently designed for privacy. This positions BDQ as a viable and accessible alternative or complement to event cameras for privacy-preserving applications.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper successfully introduces BDQ, a novel, simple, and robust privacy-preserving encoder for human action recognition. BDQ's modular design, comprising Blur, Difference, and Quantization modules, effectively processes input video frames to retain essential spatio-temporal cues for action recognition while suppressing privacy-sensitive attributes. The parameters of BDQ are optimized through an end-to-end adversarial training framework, which intelligently balances the conflicting goals of maximizing action recognition accuracy and minimizing privacy leakage.

The experimental results on three benchmark datasets (SBU, KTH, and IPN) demonstrate that BDQ achieves a state-of-the-art privacy-utility trade-off, outperforming previous methods that rely on simple downsampling or more complex UNet-like adversarial encoders. A significant finding is BDQ's remarkably low space-time complexity, making it highly suitable for deployment on resource-constrained edge devices. Furthermore, the paper provides compelling evidence that BDQ's trade-off is comparable to that offered by DVS sensor-based event cameras, a specialized hardware solution for inherent privacy. Extensive analyses confirm BDQ's effectiveness through ablation studies, robustness against diverse adversaries (including reconstruction attacks), and a subjective user study on visual privacy.

7.2. Limitations & Future Work

The authors explicitly state one key limitation:

• Dependency on Motion: "Due to its design, our proposed framework for privacy preservation cannot work in cases when the subject or the camera does not move." This is a direct consequence of the Difference module, which relies on pixel-wise intensity subtraction between consecutive frames to highlight motion and suppress static features. If there is no motion, the Difference module would output near-zero frames, losing all information, including action cues. This would make it unsuitable for tasks involving static observations or very slow, subtle movements.

  While the paper doesn't explicitly detail future work beyond the supplementary mention of hardware implementation, potential directions could include:

• Addressing Static Scenes: Developing mechanisms to handle privacy in static or near-static scenes, potentially by integrating features from the Blur module more directly or by incorporating other privacy-preserving techniques (e.g., semantic segmentation for anonymization of specific regions) when motion is absent.

• Expanding Privacy Attributes: Exploring the preservation of other privacy attributes beyond identity, gender, or actor-pair, such as emotional state, specific objects in the scene, or more nuanced activity contexts.

• Hardware Implementation: As hinted in the supplementary material, exploring dedicated hardware implementations for the BDQ modules could further enhance efficiency and integrate privacy protection directly at the point of capture, similar to DVS sensors.

• Generalization to Other Tasks: Applying the BDQ principles to other privacy-preserving computer vision tasks beyond action recognition, such as pose estimation or object detection, where different utility-privacy trade-offs might be required.

• Dynamic $\alpha$ and $H$ Tuning: Investigating adaptive methods for tuning the adversarial weight $\alpha$ and the hardness term $H$ based on the specific application or user privacy preferences, rather than fixed values.

7.3. Personal Insights & Critique

This paper offers a highly insightful and practical approach to a critical problem. My personal insights and critique are as follows:

• Elegance in Simplicity: The modular BDQ design is its greatest strength. While other adversarial privacy-preserving models often rely on complex, large UNet-like encoders, BDQ achieves superior performance with a simple, interpretable sequence of operations. This simplicity directly translates to its remarkable efficiency (low parameters, low FLOPs), making it a genuinely viable solution for edge computing and consumer devices. This is a crucial practical advantage often overlooked by highly complex DNN solutions.

• Clever Use of Motion: The Difference module is particularly clever. By explicitly making motion the primary source of information, the system inherently discards much of the static, identity-revealing background and appearance details. This aligns well with action recognition, which is fundamentally about change over time, and demonstrates a deep understanding of the problem domain.

• Strong Benchmarking: The comparison against DVS sensors is a powerful statement. DVS cameras are specialized hardware, and demonstrating that a software solution can achieve comparable privacy-utility trade-offs makes BDQ highly compelling for broader adoption with existing camera infrastructure.

• Thorough Analysis: The extensive analysis, including ablation studies, robustness tests against multiple adversaries, reconstruction attacks, and a subjective user study, adds immense credibility to the claims. This comprehensive validation is a hallmark of rigorous academic research.

• Limitations & Future Opportunities: The stated limitation regarding static scenes is significant. While understandable given the Difference module's design, it does restrict the applicability of BDQ to purely dynamic scenarios. This opens up an interesting avenue for future research: how to gracefully handle transitions from motion to stillness, or to extract privacy-preserving information from static scenes (e.g., pose without identity). Perhaps a hybrid approach, where a different privacy-preserving module activates when motion is minimal, could be explored.

• Transparency of bdq parameters: The visualization of learned quantization steps (Figure 3, row 2 and Figure 4, right) provides valuable transparency into how the BDQ encoder makes its decisions, especially regarding the privacy-utility trade-off controlled by $\alpha$. This interpretability is a benefit over black-box DNN encoders.

• Transferability: The core idea of modular, motion-focused, and adversarially trained privacy preservation could potentially be transferred to other domains. For instance, in industrial settings for activity monitoring where worker identity is private but their actions (e.g., assembly steps) are critical. Or in elderly care, where falls need to be detected without identifying the individual in the home.

  Overall, this paper presents a highly impactful contribution, offering a pragmatic, efficient, and robust solution for privacy-preserving action recognition that pushes the boundaries of what's achievable with current technology.