1. Bibliographic Information

1.1. Title

Demystifying the Characteristics of High Bandwidth Memory for Real-Time Systems

1.2. Authors

Kazi Asifuzzaman*, Mohamed Abuelala†, Mohamed Hassan† and Francisco J Cazorla* *Barcelona Supercomputing Center, Spain †McMaster University, Canada

1.3. Journal/Conference

The paper indicates its venue as servers r or orks.I.9/ICCAD98.2021.63, which suggests it was likely presented at a conference in 2021, possibly related to computer-aided design or real-time systems, given the content. Without a more specific identifier for "ICCAD98," it's challenging to precisely comment on the venue's reputation, but "ICCAD" (International Conference on Computer-Aided Design) is a highly reputable conference in the field of electronic design automation. The presence of "2021" confirms its contemporary relevance at the time of publication.

1.4. Publication Year

2021

1.5. Abstract

The paper addresses the growing demand for memory performance in critical real-time systems across domains like automotive and avionics, driven by advanced software functionalities and artificial intelligence algorithms. Traditional DRAM struggles to meet these demands. High-Bandwidth Memory (HBM) offers a promising alternative with high bandwidth, low power, and high integration. However, its predictability and isolation properties for critical embedded systems are unclear. This work presents what the authors claim to be the first timing analysis of HBM for real-time systems. It reveals HBM's unique structural and timing characteristics compared to DRAM, demonstrating how these can be exploited to enhance time predictability, improve task isolation, and reduce worst-case memory latency.

1.6. Original Source Link

/files/papers/69003d27bd968e29d463b6a3/paper.pdf (This appears to be a link to a PDF file within a local or specific file system, indicating it was provided directly rather than a publicly accessible journal link. Its publication status is likely that of a conference paper as suggested by the venue information.)

2. Executive Summary

2.1. Background & Motivation

The core problem the paper aims to solve is the inability of traditional Dynamic Random Access Memory (DRAM) to meet the increasing memory performance requirements of modern critical real-time systems. These systems, found in domains like automotive and avionics, are increasingly incorporating complex software functionalities, including Artificial Intelligence (AI) algorithms, which process massive amounts of sensor data. This trend leads to unprecedented demands for high memory bandwidth and low latency.

This problem is important because in critical real-time systems, not only average performance but also predictability and worst-case performance are paramount. A system must guarantee that all tasks complete within their deadlines, which means understanding and bounding the worst-case execution time (WCET) of memory accesses. Existing DRAM technologies often suffer from highly variable access latencies and limited predictability, making them challenging for critical applications.

The paper's entry point is High-Bandwidth Memory (HBM), an emerging technology that offers significantly higher bandwidth, lower power consumption, and better integration compared to DRAM. However, despite its performance advantages, its suitability for real-time systems—specifically its predictability and isolation properties—remains largely unexplored. The innovative idea is to conduct the first dedicated timing analysis of HBM from a real-time systems perspective, aiming to demystify its characteristics and assess its potential benefits for critical embedded applications.

2.2. Main Contributions / Findings

The paper makes several primary contributions:

1. HBM Device Structure and Timing Analysis: It provides an in-depth analysis of HBM's unique device structure and how its functional and timing behavior differs from conventional DRAMs. This leads to the identification of specific HBM features that can offer latency guarantees not present in other DRAM-based memories. These insights are articulated as a set of observations.

2. Impact on Predictability and Isolation: The work analyzes how main HBM features can be leveraged to either increase timing isolation among tasks or decrease worst-case memory latency (WCL). It develops HBM-specific latency formulations and illustrative timing diagrams, suggesting that HBM is a promising memory protocol for real-time embedded systems due to its potential for improved predictability.

3. Empirical Comparison: The authors perform an empirical comparison between the latest HBM standard (HBM2) and DDR4 DRAM. This comparison uses a detailed cycle-accurate simulator (DRAMSim3 integrated with MacSim) and a wide set of representative and synthetic benchmarks to assess average performance, worst-case performance, and isolation properties.

4. Novel Timing Simulation Model: Recognizing limitations in existing simulators regarding HBM features, the paper develops and open-sources a C++ timing simulation model. This model is derived from JEDEC standards for HBM2 and DDR4, allowing for accurate assessment of all HBM features, including recently introduced ones like pseudo-channels, which are often not fully implemented in state-of-the-art memory simulators.

   The key conclusions and findings include:

• HBM's architectural features, such as wider connections, independent channels, pseudo-channels, reduced tCCD, dual command interface, and implicit precharge, offer significant advantages in improving timing predictability and isolation compared to traditional DRAM.
• Empirical results show that HBM2 consistently reduces the total number of read requests, provides lower worst-case per-request read latency, and offers better average-case performance compared to DDR4, particularly for memory-intensive applications.
• HBM allows for various degrees of isolation (stack, channel, pseudo-channel) that can be exploited for real-time systems, though achieving full isolation may involve a trade-off with peak bandwidth.
• The developed timing model confirms the benefits of individual HBM features in isolation, providing granular insights into their impact on latency and bandwidth. These findings collectively demonstrate that HBM is a viable and potentially superior memory technology for critical real-time systems that require stringent predictability guarantees.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand this paper, a foundational understanding of DRAM architecture, real-time systems concepts, and the basic structure of High-Bandwidth Memory (HBM) is essential.

3.1.1. Dynamic Random Access Memory (DRAM)

DRAM is the most common type of main memory in computers. It stores each bit of data in a separate capacitor within an integrated circuit, requiring periodic refresh cycles to maintain the data.

• Structure:
  • Channels: Independent data paths to the memory controller. Each channel can operate in parallel.
  • Ranks: A collection of DRAM chips that share a common command/address bus and are accessed simultaneously. In a typical DIMM (Dual In-line Memory Module), multiple ranks can exist.
  • Banks: Within each rank, memory is organized into multiple banks. These banks are largely independent, allowing for parallel operations to a certain extent. Each bank contains a 2-D matrix of memory cells (rows and columns).
  • Row Buffer: Each DRAM bank has a dedicated row buffer that temporarily stores the entire contents of the most recently accessed row. Accesses to data within the currently open row are faster.
• Commands: Memory operations are initiated via specific commands issued by the memory controller.
  • ACT (Activate): Fetches an entire row from the memory cells into the row buffer. This is a prerequisite for accessing data in that row.
  • CAS (Column Address Strobe): Used to perform a read (RD) or write (WR) operation on a specific column within the currently active row in the row buffer.
  • PRE (Precharge): Writes back the contents of the row buffer to the memory cells and closes the row, making the bank available for activating a different row. If the requested row is not the one currently in the row buffer, a PRE command must be issued before a new ACT command.
• Timing Constraints (JEDEC Standard): To ensure correct operation, the DRAM standard mandates specific delays (in clock cycles) between different commands. These constraints are critical for real-time analysis as they dictate minimum latencies.
  • tRCD (Activate to Read/Write Delay): Minimum time between an ACT command and a CAS command to the same bank.
  • tRL (Read Latency): Time between the RD command and the start of data transfer.
  • tWL (Write Latency): Time between the WR command and the start of data transfer.
  • tRP (Precharge to Activate Delay): Minimum time between a PRE command and a subsequent ACT command to the same bank.
  • tRAS (Row Active Time): Minimum time a row must remain active before a PRE command can be issued to that bank.
  • tWR (Write Recovery Time): Minimum time after a WR command finishes before a PRE command can be issued to the same bank.
  • tRTP (Read to Precharge Delay): Minimum time after a RD command finishes before a PRE command can be issued to the same bank.
  • tCCD (CAS to CAS Delay): Minimum time between two consecutive CAS commands to the same bank group. This is the column-to-column timing constraint, or minimum burst duration.
  • tRRD (Row to Row Delay): Minimum time between two ACT commands to different banks within the same bank group.
  • tFAW (Four-Activate Window): A constraint limiting the number of ACT commands that can be issued within a specific time window across banks in the same rank (e.g., no more than 4 ACT commands in a tFAW window).

3.1.2. Real-Time Systems

Systems where the correctness of computations depends not only on the logical results but also on the time at which the results are produced.

• Worst-Case Execution Time (WCET): The maximum possible time a task can take to execute on a given hardware platform, under any possible input and system state. Accurately bounding WCET is crucial for guaranteeing task deadlines.
• Predictability: The ability to accurately determine and bound the timing behavior of a system, particularly its worst-case timing. In memory systems, this means understanding the maximum possible delay for any memory access.
• Isolation: Ensuring that the execution and timing of one task or component do not unduly interfere with or delay other tasks or components, especially in shared resources like memory.

3.1.3. High-Bandwidth Memory (HBM)

HBM is a type of 3D-stacked synchronous DRAM (SDRAM) that offers significantly higher bandwidth than traditional flat (planar) DRAM by stacking multiple DRAM dies vertically on top of a base logic die.

• Stacks: Multiple DRAM core dies are stacked, connected by Through Silicon Vias (TSVs).
• Logic Die: The base layer of the stack, responsible for external communication and housing the memory controller interface.
• TSVs (Through Silicon Vias): Vertical electrical connections passing through the silicon wafer, enabling high-density, low-latency communication between stacked dies.
• Wider Interface: HBM employs a much wider data bus (e.g., 1024 bits external) compared to DRAM (e.g., 64 bits), which is a key source of its higher bandwidth.
• Channels: HBM is organized into multiple independent channels (e.g., 8 channels in a stack), each with its own command/address and data interface.
• Pseudo-Channels: A feature introduced in HBM2 where each channel can be further divided into two semi-independent sub-channels, offering finer-grained isolation.

3.2. Previous Works

The paper contextualizes its work by discussing existing research in HBM and DRAM memory predictability.

• HBM as an Emerging Standard: Early works (e.g., [22]) presented HBM as a technology providing superior bandwidth (>256GB/s) and lower power consumption. More recent studies (e.g., [4]) compare HBM and DDR for high-performance systems. Challenges like capacity scaling (stacking more dies) are explored in [18]. Power consumption comparisons [36] confirm HBM2's energy efficiency. The authors note that these works primarily focus on average-case performance or specific application improvements (e.g., CNNs on HBM-enabled GPUs [37]) rather than real-time predictability. Bingchao et al. [4] describe pseudo-channel and dual-command features of HBM, but conclude they don't significantly improve average-case performance, contrasting with this paper's focus on worst-case benefits.
• DRAM Memory Predictability: There's an extensive body of literature on handling memory contention in real-time systems. These efforts generally fall into two categories:
  • Software Solutions: Techniques to increase isolation among tasks in memory, such as bank partitioning among processors (e.g., PALLOC [38]) or controlling access counts (e.g., [39]).
  • Hardware Solutions (Memory Controllers): Designs for predictable DRAM controllers, often balancing predictability and performance (e.g., [8], [24], [28], [40]). Guo et al. [8] provide a comprehensive survey of predictable DRAM controllers.
• Limitations of DDR DRAMs for Real-Time: Hassan [9] identifies inherent limitations of DDR DRAMs in achieving reasonable predictability, citing highly variable access latencies and overly pessimistic bounds. This work proposes using Reduced Latency DRAM (RLDRAM) as an alternative.
• Memory Simulators: The paper mentions existing memory simulators like DRAMSim3 [10], RAMulator [33], and GEM5 [34], noting that they are primarily designed for high-performance systems and often overlook or abstract features crucial for worst-case timing analysis (e.g., pseudo-channel mode in HBM2). A recent DRAM simulator targeting real-time systems, McSim [35], currently does not support HBM.

3.3. Technological Evolution

The evolution of memory technology, particularly in the context of high-performance and real-time systems, has been driven by increasing demands for bandwidth and capacity.

• Traditional DRAM: For decades, planar DRAM (e.g., DDRx generations) has been the workhorse of main memory. While continuous improvements in clock speed and interface width have been made (e.g., DDR3, DDR4), they face fundamental limitations in scaling bandwidth due to physical constraints (e.g., pin count, signal integrity).

• GDDRx (Graphics Double Data Rate): A specialized form of DRAM designed for graphics cards, offering higher bandwidth than standard DDR by operating at very high data rates. However, this comes at a considerable power cost and is still limited by a relatively narrow interface.

• HBM (High-Bandwidth Memory): Represents a significant architectural shift. Instead of a planar layout, HBM uses 3D stacking of DRAM dies on a logic die, connected by TSVs. This allows for a much wider interface (e.g., 1024 bits) and multiple independent channels, drastically increasing bandwidth while reducing power consumption. It was initially prevalent in GPUs and accelerators. The evolution from HBM1 to HBM2 introduced features like pseudo-channels.

  This paper's work fits within this technological timeline by evaluating the suitability of the latest HBM technology (HBM2) for a domain (real-time embedded systems) where it hasn't been traditionally applied and where its unique properties for predictability and isolation are not yet fully understood.

3.4. Differentiation Analysis

Compared to prior work, this paper's core innovation and differentiation lie in its specific focus and comprehensive analysis:

• First Timing Analysis of HBM for Real-Time Systems: While HBM has been analyzed for high-performance computing and its average performance benefits are known, this paper is, to the authors' knowledge, the first to undertake a timing analysis of HBM specifically for real-time systems. This means focusing on predictability, worst-case latencies, and isolation properties, which are distinct from average-case performance metrics.

• Device-Centric Analysis: The paper focuses on the inherent architectural characteristics and functionalities of the HBM device itself, rather than specific memory controller implementations. This allows for general observations and insights independent of any particular scheduling technique, which can then inform the design of future predictable HBM memory controllers.

• Comprehensive Feature-Level Investigation: It goes beyond high-level comparisons by delving into specific HBM features (e.g., pseudo-channels, reduced tCCD, dual command interface, implicit precharge) and quantitatively assessing their isolated and combined impact on worst-case latency and isolation, often developing new formulations or timing diagrams to illustrate these effects.

• Custom Simulation Model: Recognizing the limitations of existing simulators that abstract away real-time critical HBM features, the authors develop and open-source a dedicated timing simulation model derived directly from JEDEC standards. This ensures accurate and detailed analysis of HBM's complex timing behavior, particularly for newer features like pseudo-channels not fully modeled elsewhere.

  In essence, while previous works looked at HBM for raw performance or at DRAM for predictability, this paper uniquely bridges these two areas by rigorously examining whether HBM's performance benefits can be translated into predictable performance for critical real-time applications.

4. Methodology

4.1. Principles

The core idea of the method used in this paper is to systematically identify and analyze the unique architectural and timing characteristics of High-Bandwidth Memory (HBM) that differentiate it from traditional DRAM, and then evaluate how these characteristics can be leveraged to improve predictability and reduce worst-case memory latency (WCL) in real-time systems. The intuition behind this is that HBM's design, driven by high-bandwidth demands, might inadvertently or inherently possess properties beneficial for real-time predictability, such as improved isolation and reduced command conflicts.

The theoretical basis relies on a deep understanding of memory device operation (DRAM and HBM JEDEC standards) and real-time systems theory (WCET analysis, timing predictability, isolation). By comparing the timing models and operational sequences of HBM and DRAM, the authors aim to quantitatively demonstrate HBM's advantages for critical embedded applications. This involves breaking down memory access latency into its constituent components and showing how HBM's features can reduce these components or their variability.

4.2. Core Methodology In-depth (Layer by Layer)

The methodology unfolds in several integrated steps: first, a detailed structural and functional analysis of HBM compared to DRAM, followed by a feature-by-feature theoretical examination of how HBM characteristics affect predictability, and finally, an empirical validation using simulation and a custom timing model.

4.2.1. HBM Structure and Features Analysis (Section II)

The paper begins by contrasting the organizational structures of DRAM and HBM.

• DRAM Background: Briefly reviews DRAM fundamentals: independent channels, ranks, banks, row buffers, and commands (ACT, CAS, PRE). It also highlights standard JEDEC timing constraints between these commands (e.g., tRCD, tRL, tWL, tRP, tRAS, tWR, tRTP, tCCD, tRRD, tFAW).

  The following figure (Figure 1 from the original paper) illustrates the organizational structures of DRAM (left) and HBM (right) devices:

  该图像是图1，展示了DRAM（左）和HBM（右）设备的组织结构示意图。左侧为多层DRAM芯片堆叠及其行列译码结构，右侧为多通道多银行HBM结构及其与核心的连接布局，突出两者在架构上的差异。

As seen in Figure 1, DRAM typically has a planar structure with multiple chips forming ranks within a channel, while HBM stacks multiple DRAM dies on a logic die.

The following figure (Figure 2 from the original paper) describes the most relevant timing constraints for DRAM commands:

该图像是图7，展示了HBM的双命令特性。图中通过时间序列展示了不同Rank和Bank的命令发出时序，突出C3命令在多个Bank间的分布以实现高效并发访问。

Figure 2 visually depicts the temporal relationships between DRAM commands, such as the delay between ACT and RD (tRCD) and RD to data start (tRL).

• HBM Device Organization: HBM stacks DRAM dies vertically on a base logic die, connecting them with Through Silicon Vias (TSVs). The logic die handles external communication. Each DRAM core die can accommodate multiple independent channels, connected to the logic die with wide TSV I/Os.

  • Observation 1: HBM offers wider connections (1024 bits) to the processing unit compared to DRAM (e.g., 64 bits).

  • Observation 2: A processor can be connected to several independent HBM stacks residing on the same silicon interposer.

  • Observation 3: HBM channels, even within the same core die, operate independently via private data and address/control signals. Each HBM channel uses a non-shared 128-bit TSV connection to the logic die. This implies higher parallelism and reduced inter-channel contention.

  • Observation 4: While HBM is based on DRAM banks, its organization differs from DRAM, lacking the concept of "ranks." Instead, it uses channels, pseudo-channels, (logical) banks, and half (physical) banks.

  • Observation 5: In DDR3/4, access granularity is typically 4, 8, or 16 bits per physical bank, leading to 64 bits for a DRAM rank. In contrast, each HBM bank supplies 128 bits per single access.

    The following figure (Figure 3 from the original paper) further illustrates the HBM organization:

    该图像是一张图表，展示了图8中HBM伪通道对tFAW约束的影响，分别对比了(a) DRAM，(b) 带伪通道的HBM，以及(c) 带伪通道且ACT命令为两周期的HBM。

Figure 3a shows multiple HBM stacks, 3b depicts per-channel connections, and 3c details the internal structure, including pseudo-channels and legacy mode arrangements.

• HBM's Core Memory Cells: Despite its unique structure, HBM uses conventional DRAM cells for storage. Thus, basic commands and associated timing constraints (like ACT, RD/WR, PRE) are dictated by JEDEC standards for both DRAM and HBM.
• Reduced Column-to-Column Timing (tCCD):
  • Observation 6: HBM has a smaller tCCD compared to DRAM. This is due to HBM supporting a burst length (BL) of up to 4, whereas DRAM typically uses $BL = 8$. For $BL = 8$, tCCD is constrained by $BL/2 = 4$ cycles. With $BL = 4$ (or 2), HBM can achieve $tCCD = 1$ or 2.
• Pseudo Channel Mode: A feature introduced in HBM2.
  • Observation 7: In HBM2, a single memory access (i.e., CAS command) provides 32 Bytes (B) of data using $BL = 4$. This is because each pseudo-channel has 64 I/Os, so $64 \times 4 = 256$ bits, or 32B.
  • Observation 8: Pseudo-channels are semi-independent. They share row/column command buses and clock inputs but can decode and execute commands independently, offering a degree of isolation.
• Dual Command Interface:
  • Observation 9: HBM has dedicated pins for column addresses separate from row address pins. This enables read/write commands to be issued concurrently with ACT/PRE commands, reducing bus conflicts.
• Implicit Precharge:
  • Observation 10: In pseudo-channel operation, HBM allows a subsequent ACT command to be issued to another row in the same bank without explicitly closing the previous row. The HBM device's internal circuitry handles an implicit PRE command to close the first row before activating the second. This can remove explicit PRE commands from the command bus.

4.2.2. HBM for Real-Time Systems (Section III)

This section delves into how the identified HBM features can be leveraged to improve isolation and reduce worst-case latency. The analysis isolates each feature's impact for clarity.

4.2.2.1. HBM Degrees of Isolation (Section III-A)

HBM offers multiple levels of isolation that can be exploited to reduce contention among tasks by mapping data/instructions to different memory partitions:

• Stack Isolation (Observation 2): Requests to different HBM stacks do not interfere with each other, as each stack operates independently. This is unique to HBM's 3D-stacked architecture.
• HBM (Logical) Bank Isolation: Similar to DRAM, requests from different tasks can be mapped to non-overlapping banks within an HBM channel.
• Half (Logical) Bank Isolation (Pseudo-Channels, Observation 8): Pseudo-channels allow requests to be sent to different half-logical banks. While they share some resources (command buses), their semi-independence provides a reduced degree of isolation, useful for mitigating inter-task contention.

4.2.2.2. Reconciling Isolation and Bandwidth Trade-offs (Section III-B)

Bank partitioning for isolation in DRAM systems often leads to reduced bandwidth because a single request might require multiple accesses. HBM's wider access granularity can mitigate this.

The following figure (Figure 4 from the original paper) illustrates the effect of bank partitioning:

该图像是图表，展示了图9中DDR4与HBM2在EEMBC基准测试（左图）及合成和MXM基准测试（右图）中的读请求数量对比，且注意两图的刻度不同。

Figure 4 shows that for a 64B cache line, DRAM (top) might need 4 accesses due to its narrower data bus (e.g., 16-bit), while HBM (bottom) can satisfy it in 2 accesses due to its wider 32B per-access granularity in pseudo-channel mode.

The paper presents two lemmas to quantify the Worst-Case Delay (WCD) due to interference under bank partitioning:

Lemma 1. Under bank partitioning where each core is assigned BC private banks, a request with a data size of $Y$ bytes targeting a DRAM with a data bus width of cw bits and a burst length of BL suffers a total WCD due to interference from other requestors that can be computed as shown in Equation 1. $ WCD_{DRAM}^{tot} = \frac{Y}{BL \times BC \times cw / 8} \times WCD^{Acc} $ Where:

• $WCD_{DRAM}^{tot}$: Total Worst-Case Delay for DRAM.

• $Y$: Data size of the request in bytes.

• BL: Burst Length (number of clock cycles data is transferred per CAS command).

• BC: Number of banks assigned to the requestor.

• cw: Data bus width in bits.

• $cw / 8$: Data bus width in bytes.

• $WCD^{Acc}$: Worst-Case interference delay suffered by a single access.

  Lemma 2. Under bank partitioning scheme where each core is assigned a single private bank, a request with a data size of $Y$ bytes targeting HBM suffers a total WCD due to interference from other requestors. $ WCD_{HBM}^{tot} = Y / 32 \times WCD^{Acc} $ Where:

• $WCD_{HBM}^{tot}$: Total Worst-Case Delay for HBM.

• $Y$: Data size of the request in bytes.

• 32: Represents the 32B provided per single HBM access (from Observation 7, in pseudo-channel mode, a single CAS command with $BL=4$ on a 64-bit pseudo-channel bus transfers $64 \text{ bits} \times 4 \text{ cycles} = 256 \text{ bits} = 32 \text{ Bytes}$).

• $WCD^{Acc}$: Worst-Case interference delay suffered by a single access.

  These lemmas highlight how HBM's larger access granularity (32B vs. typically 16B for DRAM with $cw=16$ and $BL=8$ yielding $16 \text{ bits} \times 8 \text{ cycles} / 8 \text{ bits/byte} = 16 \text{ Bytes}$) reduces the number of accesses required for a given data size, thereby potentially halving the WCD for a 64B cache line.

4.2.2.3. Reducing CAS Latency (Section III-C)

The CAS latency is a significant component of WCD. For a sequence of $N^{OP}$ row-open requests (multiple CAS commands to the same row), the total CAS latency is given by: $ L^{CAS} = (N^{OP} - 1) \times tCCD + tCL + tBUS $ Where:

• $L^{CAS}$: Total CAS latency.

• $N^{OP}$: Number of open-row requests (or CAS commands).

• tCCD: CAS to CAS Delay.

• tCL: CAS Latency (time from CAS command to data transfer start).

• tBUS: Time required to transfer the data ($BL/2$).

  HBM offers a significant advantage here due to its reduced tCCD (Observation 6), which can be 1 or 2 cycles compared to $\geq 4$ for DRAM. This directly reduces the total CAS latency.

The following figure (Figure 5 from the original paper) illustrates the effect of reduced tCCD:

该图像是图表，展示了论文中图10最差读请求延迟对比，比较了DDR4与HBM2内存对EEMBC基准（左图）及Synthetic和MXM基准（右图）的性能表现，HBM2普遍具有更低的最差读延迟。

Figure 5 shows that for three consecutive CAS commands, HBM with $tCCD=2$ (Figure 5b) completes the sequence in 21 cycles, whereas DRAM with $tCCD=4$ (Figure 5a) takes 31 cycles, assuming $tCL=15$.

4.2.2.4. Reducing Bus Conflicts (Section III-D)

• Implicit Precharge (Observation 10): By allowing an ACT command to be issued to a bank with an active row (with the device handling the PRE implicitly), HBM eliminates the need for the memory controller to explicitly issue PRE commands on the bus. This removes PRE bus conflict delays.

  The following figure (Figure 6 from the original paper) captures how the Implicit Precharge feature reduces memory access latency:

  该图像是论文中的图表，展示了HBM2中通道分区（Partition）与交织（Interleave）对EEMBC基准测试（左）及合成与MXM基准测试（右）的带宽（BW）影响，且两部分坐标刻度不同。

Figure 6 demonstrates that in a scenario where a new request targets a different row in an already active bank, DRAM's explicit PRE command might be delayed by bus conflicts (7 cycles in the example, from cycle 37 to 44). HBM's implicit PRE allows the controller to issue the next ACT directly after satisfying tRAS and tRP, leading to an earlier completion (cycle 85 vs. 92 for DRAM).

• Dual Command Bus (Observation 9): HBM's dedicated row and column address pins enable concurrent issuance of ACT/PRE commands and RD/WR commands. This significantly reduces command bus conflicts, where an ACT might otherwise delay a CAS, or vice-versa, depending on priority.

  The following figure (Figure 7 from the original paper) illustrates the Dual Command Interface:

  该图像是图表，展示了图12中DDR4与HBM2在EEMBC基准测试（左侧）及Synthetic和MXM基准测试（右侧）中的执行周期对比，注意两图的刻度不同。

Figure 7 shows a scenario where concurrent ACT commands to different banks and CAS commands to another bank would cause bus conflicts in DRAM (left), delaying CAS commands by one cycle. In HBM (right), the dual command interface allows these commands to be issued simultaneously, eliminating such delays.

4.2.2.5. Reducing ACT Latency (Section III-E)

• tFAW Constraint: DRAMs have a tFAW constraint (four-bank activation window) that limits the number of ACT commands within a certain time window (e.g., 4 ACTs in a tFAW window). This can cause significant idle gaps and delays. While rank interleaving in DRAM can circumvent tFAW, it introduces other delays like tRTR (read to read turnaround time between ranks).

• HBM's Pseudo-Channels (Observation 11): HBM lacks the concept of ranks (Observation 4). Instead, its pseudo-channels mean that ACT commands targeting different pseudo-channels do not have to conform to the tFAW constraint. There is also no tRTR-like constraint between pseudo-channels. This allows for clever interleaving of ACTs to mitigate the tFAW effect on WCL.

  The following figure (Figure 8 from the original paper) shows the effect of HBM's pseudo-channel on the tFAW constraint:

  该图像是图表，展示了Fig. 13中DDR4与HBM2在不同基准测试下的总片外内存时间。左侧为EEMBC基准，右侧为Synth.和MXM基准，注意两者纵轴尺度不同，HBM2的内存时间显著低于DDR4。

Figure 8a shows how DRAM's tFAW constraint causes an idle gap (cycles 53-62) for a sequence of 8 requests, finishing at cycle 84. Figure 8b illustrates how HBM, by leveraging pseudo-channels (and assuming single-cycle ACTs for this specific analysis point), can avoid this tFAW effect, finishing at cycle 76.

4.2.2.6. HBM Drawback: Two-cycle ACT commands (Section III-F)

One drawback of HBM is that ACT commands consume two cycles on the command bus, unlike DRAM's single-cycle ACTs. This affects all ACT-related timing constraints, as they are measured from the second cycle of the command. This adds an extra cycle per ACT command, potentially negating some benefits, as shown in Figure 8c. Even with pseudo-channels to mitigate tFAW, the two-cycle ACT can bring the completion time back to that of DRAM (e.g., cycle 84 in Figure 8c).

5. Experimental Setup

5.1. Datasets

The experiments use a combination of standard benchmarks and synthetic kernels to represent real-time applications with varying memory demands.

• EEMBC Autobench (EEMBC) [29]: This suite is used to model control-type applications. These are common in real-time embedded systems, often found in automotive, industrial, and general-purpose systems, and are typically less memory-intensive. These benchmarks help assess HBM's performance in scenarios where memory access might not be the primary bottleneck but predictability is still critical.

• Matrix Multiplication (MXM) Kernel: This is a common kernel for payload applications, which are often more memory-intensive. It is configured with different total memory footprints: 2MB, 4MB, and 8MB. MXM is highly relevant as it forms a core component of many AI algorithms, such as those used in object detection libraries (e.g., YOLOv3 [30]), and can account for a significant portion of their execution time (e.g., >65% for YOLO).

• Bandwidth (BW_read and BW_write) Micro-benchmarks from IsolBench [32]: These synthetic benchmarks are designed to stress the memory system by generating high read and write bandwidth demands. They are useful for evaluating the raw memory performance and throughput characteristics of HBM and DDR4.

• Latency Micro-benchmark from IsolBench [32]: This micro-benchmark is designed to measure memory access latency, often exhibiting random pointer-chasing-like patterns. It helps assess the worst-case and average-case latency behavior, particularly in scenarios with low data locality where high bandwidth might not offer significant benefits.

  These datasets were chosen to cover a spectrum of real-time application types, from control-intensive (EEMBC) to memory-intensive (MXM, BW_read/write) and latency-sensitive (Latency), allowing for a comprehensive validation of HBM's predictability and performance characteristics.

5.2. Evaluation Metrics

For every evaluation metric mentioned in the paper, a complete explanation is provided:

• Worst-Case Execution Time (WCET):

  1. Conceptual Definition: WCET represents the maximum possible time a task can take to execute on a given hardware platform under any possible input and system state. It is a critical metric in real-time systems to guarantee that tasks meet their deadlines. The paper focuses on memory-related components of WCET.
  2. Mathematical Formula: $ WCET = WCCT + WCL $
  3. Symbol Explanation:
     • WCET: Worst-Case Execution Time of a task.
     • WCCT: Worst-Case Computation Time, representing the time spent executing instructions on the processor, excluding memory access delays.
     • WCL: Worst-Case Memory Latency, representing the maximum time spent waiting for memory accesses.

• Worst-Case Memory Latency (WCL) per Request:

  1. Conceptual Definition: $WCL^{perReq}$ is the maximum latency observed for a single memory request during the execution of a task. This metric is fundamental to understanding the worst-case behavior of the memory subsystem.
  2. Mathematical Formula: $ WCL = WCL^{perReq} \times NumReqs $
  3. Symbol Explanation:
     • WCL: Total Worst-Case Memory Latency for a task.
     • $WCL^{perReq}$: Worst-Case Memory Latency suffered by a single request.
     • NumReqs: The worst-case total number of memory requests issued by the task.

• Total Number of Read Requests:

  1. Conceptual Definition: This metric quantifies the total count of memory read operations issued by a task to the off-chip memory. In modern architectures, write requests (due to cache evictions) typically do not stall the processor pipeline and are thus less critical for WCET analysis. Focusing on read requests helps isolate the impact of memory access patterns and cache behavior on performance.
  2. Mathematical Formula: (No explicit formula provided in the paper, as it's a direct count) $ \text{Total Read Requests} = \sum_{\text{all read ops}} 1 $
  3. Symbol Explanation:
     • The formula represents a simple summation of all read operations.

• Bandwidth Degradation (due to channel partitioning):

  1. Conceptual Definition: This metric quantifies the percentage reduction in available memory bandwidth when isolation techniques, such as channel partitioning, are applied. While isolation improves predictability, it often comes at the cost of peak performance by limiting the parallelism of memory access.
  2. Mathematical Formula: (No explicit formula provided in the paper, but standard calculation for percentage degradation) $ \text{Bandwidth Degradation (%)} = \left(1 - \frac{\text{BW}{\text{partitioned}}}{\text{BW}{\text{interleaved}}}\right) \times 100% $
  3. Symbol Explanation:
     • $\text{BW}_{\text{partitioned}}$: Bandwidth achieved when memory is partitioned for isolation.
     • $\text{BW}_{\text{interleaved}}$: Bandwidth achieved when memory accesses are interleaved across all channels for maximum performance.

• Execution Time (Cycles):

  1. Conceptual Definition: The total number of clock cycles required for a benchmark or application to complete its execution. This is a fundamental measure of overall performance.
  2. Mathematical Formula: (No explicit formula provided in the paper, as it's a direct measurement from the simulator) $ \text{Execution Time} = \text{End Cycle} - \text{Start Cycle} $
  3. Symbol Explanation:
     • $\text{End Cycle}$: The clock cycle at which the execution finishes.
     • $\text{Start Cycle}$: The clock cycle at which the execution begins.

• Total Off-Chip Memory Time:

  1. Conceptual Definition: The cumulative time spent by the processor waiting for or performing accesses to the off-chip main memory. This metric directly reflects the impact of memory system performance on overall application execution, isolating memory-related stalls.
  2. Mathematical Formula: (No explicit formula provided in the paper, as it's a cumulative measurement from the simulator) $ \text{Total Off-Chip Memory Time} = \sum_{\text{all memory accesses}} \text{latency}_{\text{access}} $
  3. Symbol Explanation:
     • $\text{latency}_{\text{access}}$: The time taken for each individual off-chip memory access.

5.3. Baselines

The paper's method (HBM2) is primarily compared against DDR4 DRAM.

• DDR4-2133: A standard DDR4 DRAM operating at 2133MHz is used as the reference predictable DDR technology. This choice is representative of commonly used high-performance DRAM today. The authors explicitly state they are unaware of worst-case analyses for GDDR5/6, justifying the use of DDR4 as the primary baseline for real-time predictability comparisons.

5.4. Simulation Environment

• CPU Simulator: MacSim [11], a detailed cycle-accurate processor simulator.
• Off-chip Memory Simulator: DRAMSim3 [10], integrated with MacSim. DRAMSim3 is a cycle-accurate, thermal-capable DRAM simulator.
• Custom Timing Simulation Model: A $C++$ simulation model developed by the authors, derived directly from the JEDEC standards for HBM2 and DDR4 [12], [13]. This model is open-sourced [14] and is crucial for accurately capturing HBM features (like pseudo-channels) that are not fully implemented or detailed in existing simulators like DRAMSim3.

5.5. Configuration

The following table (Table II from the original paper) lists the cache and memory configuration parameters used: The following are the results from Table II of the original paper:

5.5.1. Cache Configuration

• L1 Cache: 16KB size.
• Last-Level Cache (LLC): 256KB size.
• Associativity: 8 for L1, 16 for LLC.
• Cache Line Size:
  • DDR4: 64 Bytes (B).
  • HBM2: 512 Bytes (B).
  • The cache line size is chosen to match the off-chip memory transaction size for each memory type, ensuring that all bytes of a cache line are transferred in a single memory request to maximize performance. The paper confirms that using 512B cache lines for DDR4 (with 64B sectors) does not significantly alter their results.

5.5.2. Memory Structural Configuration

• DDR4-2133:
  • Single channel, 1 rank, 8 banks (organized as 2 bankgroups with 4 banks per group).
  • device_width: 16 bits.
  • BL (Burst Length): 8.
• HBM2:
  • A single HBM2 stack with four dies.
  • Each die contains two channels (total of 8 channels per stack).
  • Each channel has 16 banks (organized as 4 bankgroups with 4 banks per group).
  • device_width: 128 bits.
  • BL: 4.
• Operating Frequency: Both HBM2 and DDR4 devices run at the same frequency of 2133MHz.
• Timing Constraints: Details are listed in Table I (not transcribed here, but referenced in the paper, showing cycle values for parameters like tRCD, tRL, tRP, tRAS, tWR, tCCD, etc., for both HBM and DDR4).

6. Results & Analysis

6.1. Core Results Analysis

The experimental results consistently demonstrate HBM's superiority over DDR4 in terms of predictability and performance for real-time systems.

6.1.1. Total Number of Read Requests

The following figure (Figure 9 from the original paper) shows the read requests of DDR4 vs HBM2 for EEMBC BMs (left) and Synthetic and MXM BMs (right):

该图像是图表，展示了不同HBM特性的孤立和组合分析对平均请求延迟（左）和整体带宽（右）的影响，比较了DRAM与HBM单一及全部特性情况。

As seen in Figure 9, HBM generally leads to a significant reduction in the number of issued read requests compared to DDR4. For EEMBC benchmarks, the average reduction is $6.5 \times$, and for BW benchmarks, it's up to $8 \times$. This is primarily because HBM, with its wide 1024-bit interface, can transfer 512B per single transaction, whereas DDR4 only handles 64B. This larger transaction size allows applications with good data locality to achieve more cache hits and reduce off-chip memory accesses. Benchmarks like Latency and MXM show smaller reductions, attributed to their access patterns causing cache conflicts and thus limiting the benefit of larger cache lines.

6.1.2. Worst-Case Per-Request Read Latency

The following figure (Figure 10 from the original paper) shows the worst latency of read request DDR4 vs HBM2 for EEMBC BMs (left) and Synthetic and MXM BMs (right):

该图像是图2，展示了DRAM命令及其时序约束，包括tRL、tRTW、tWTR、tRCD、tWL、tRAS、tRC、tB、tWR和tRP的时间关系，直观说明了读写操作的时间流程。

Figure 10 illustrates that HBM consistently introduces lower Worst-Case Latency (WCL) for read requests compared to DDR4. HBM's WCL ranges from 291-353 cycles, while DDR4's is higher, ranging from 415-480 cycles. This behavior holds true for both control-type (EEMBC) and payload/synthetic (MXM, BW, Latency) benchmarks. This indicates that HBM provides reduced per-request WCL regardless of the application's memory access aggressiveness. The paper also validated that using 512B lines for DDR4 does not affect these WCL results.

6.1.3. Memory Isolation Opportunities

The following figure (Figure 11 from the original paper) shows the channel partitioning vs interleaving in HBM2 for EEMBC BMs (left) and Synthetic & MXM BMs (right):

该图像是图示性示意图，来源于文献中的图3，展示了HBM堆叠结构(a)、各通道数据和总线连接(b)、以及包含伪通道模式（通道6）和传统模式（通道7）中4个芯片堆栈银行排列的内部结构(c)。

Figure 11 explores the trade-off between isolation and performance in HBM2. While HBM offers various degrees of isolation (stack, channel, pseudo-channel partitioning), enforcing isolation by partitioning memory resources among requestors typically degrades performance. The results show a bandwidth degradation of 15-45% (35% on average) for EEMBC benchmarks and 67-87% (78% on average) for bandwidth-intensive synthetic benchmarks when using channel partitioning versus interleaving across all channels. This confirms the well-known performance-isolation trade-off, although HBM's wider access can mitigate its effects (as discussed in Section III-B). The paper notes that the ideal compromise point is use-case dependent.

6.1.4. Average-Case Performance

The following figure (Figure 12 from the original paper) shows the execution cycles of DDR4 vs HBM2 for EEMBC BMs (left) and Synthetic and MXM BMs (right):

该图像是一个示意图，展示了不同请求到达时序对 $WCD^{Acc}$ 和 $t_{BUS}$ 期间数据传输的影响，反映了内存访问的时间窗口和数据传输延迟。

As depicted in Figure 12, HBM2 generally shows better average-case performance (fewer execution cycles) than DDR4. For EEMBC benchmarks, which are not memory-intensive, HBM2 provides an average improvement of 12%. For memory-stressing synthetic benchmarks, the performance gap is much clearer, with improvements up to $4 \times$ for BW_write. Latency benchmark shows a modest 3% improvement for HBM2, and MXM shows 1.17x improvement. This confirms that HBM offers performance benefits, especially when memory is a bottleneck, but its impact varies with the memory utilization and locality patterns of the application.

6.1.5. Total Off-Chip Memory Time

The following figure (Figure 13 from the original paper) shows the total off-chip memory time for DDR4 & HBM2 for EEMBC BMs (left) and Synth. & MXM BMs (right):

该图像是图表，展示了图5中DRAM与HBM在tCCD（命令间隔周期）效果上的区别，突出展示HBM的tCCD减少现象，通过颜色区分不同命令周期的执行状态。

Figure 13 highlights the significant reduction in off-chip memory time achieved by HBM2 compared to DDR4. HBM shows, on average, a $5 \times$ less memory time for EEMBC benchmarks. This gap widens to $6 \times$ for EEMBC and MXM benchmarks, and up to $9 \times$ for the BW_write synthetic benchmark. This metric directly reflects the efficiency of the memory subsystem, confirming that HBM2 consistently and substantially improves memory performance. The extent to which this translates to overall application performance depends on how memory-bound the application is.

6.2. Data Presentation (Tables)

The following are the results from Table I of the original paper:

The following are the results from Table II of the original paper:

6.3. Ablation Studies / Parameter Analysis

The paper conducts a form of ablation study through its "Synthetic Experiments" (Section IV-D) using the custom C++ simulation model. This allows for an isolated analysis of individual HBM features and their combined effect.

The authors developed four specific tests to stress different HBM features:

1. Dual_CMD: Measures the impact of the dual command feature (Section II-D) by issuing concurrent streams of CAS commands to one bank and ACT commands to other banks.

2. Partition: Studies the impact of HBM's wide data bus (Section III-B) by assigning a single bank to an emulated core for 64B requests.

3. Reduced_tCCD: Tests the reduced tCCD feature (Section III-C) with a stream of open-row requests (mainly CAS commands) without interfering streams.

4. tFAW: Stresses the tFAW constraint (Section III-E) by issuing close-row requests (containing ACT commands) to different banks, assessing the pseudo-channel feature.

   For each test, two HBM models were used:

• HBM_one_feature: Only the specific feature under test is modeled as HBM-like, while all other parameters are identical to DRAM. This isolates the benefit of that single feature.

• HBM_all_features (or simply HBM): Models all HBM parameters and features.

  The following figure (Figure 14 from the original paper) shows the isolated and combined analysis of the impact of different HBM features on average request latency (left) and overall bandwidth (right):

  该图像是图表，展示了原始DRAM与带内部PRE（预充电）特性的高带宽内存（HBM）在访问延迟上的对比。图中以色块和时间周期详细表示了不同内存访问阶段，突出PRE特性对降低延迟的影响。

Figure 14 presents the results of these synthetic experiments. Key findings include:

• Dual_CMD and Reduced_tCCD: These features notably contribute to bandwidth improvement and reduced average request latency. Dual_CMD eliminates bus conflicts by allowing parallel ACT and CAS commands, improving sustained bandwidth. Reduced_tCCD enables faster consecutive CAS commands, increasing throughput.

• Bank Partition: This feature (due to HBM's 32B per-access capability) allows filling a 64B cache line in two CAS commands, significantly reducing execution time and considerably improving bandwidth compared to four CAS commands for DRAM.

• tFAW: HBM's mitigation of the tFAW constraint (via pseudo-channels) leads to increased bandwidth and reduced execution time.

  These isolated analyses, combined with the theoretical framework in Section III, provide strong insights into how each HBM feature contributes to better worst-case performance and predictability.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper undertakes the first timing analysis of High-Bandwidth Memory (HBM) for real-time systems, rigorously investigating its potential benefits in terms of timing predictability and isolation compared to traditional DRAM. The authors identify and thoroughly explain unique HBM features, such as wider connections, independent channels, pseudo-channels, reduced tCCD, dual command interface, and implicit precharge. Through a combination of theoretical analysis, HBM-specific latency formulations, and extensive empirical evaluations using detailed simulation (including a custom JEDEC-standard-derived model), the paper demonstrates that HBM2 consistently reduces worst-case memory latency, decreases the total number of read requests, and offers superior average-case performance. It highlights how HBM's architectural advantages can be exploited to enhance isolation among tasks, which is critical for mixed-criticality real-time systems. Overall, the work establishes a solid foundation for understanding and leveraging HBM in future predictable real-time embedded systems.

7.2. Limitations & Future Work

The authors acknowledge several limitations and suggest future research directions:

• Device-Centric Focus: The current analysis primarily focuses on the HBM device characteristics and functionalities. While this provides general insights, it is explicitly stated as a "first essential step opening the door towards designing predictable HBM memory controllers." Future work would need to address the design and analysis of the memory controller itself, which orchestrates accesses to the HBM device.
• Impact of HBM Controller: The paper notes that the value of $WCD^{Acc}$ (worst-case interference delay suffered by a single access) depends on the memory controller architecture. Their current analysis of the HBM device characteristics provides general observations not limited to a specific scheduling technique deployed by the memory controller. Therefore, fully realizing HBM's predictability benefits will require designing HBM memory controllers optimized for real-time constraints.
• Trade-off Exploration: While the paper explores the trade-off between isolation and bandwidth, it states that "Deciding the exact ideal compromise point of this trade-off is not the focus of the paper." Future work could delve deeper into optimal partitioning strategies for different real-time application mixes.
• Energy Consumption: While mentioning that HBM consumes significantly less energy than DDR4, the paper explicitly states that energy consumption, though important for embedded systems, is not the focus of this first work on time predictability. This suggests an area for future research.

7.3. Personal Insights & Critique

This paper offers a highly valuable and timely contribution to the real-time systems community. The systematic, feature-by-feature analysis of HBM, coupled with both theoretical formulations and empirical validation, is commendably rigorous. The development and open-sourcing of a custom timing simulation model to accurately capture HBM2's nuances, particularly pseudo-channels, address a critical gap in existing research tools and demonstrate a commitment to thoroughness.

One key inspiration drawn from this paper is the idea that new hardware architectures, while primarily designed for average-case performance (e.g., high bandwidth for GPUs), can possess inherent characteristics that significantly benefit worst-case predictability if analyzed correctly. It challenges the assumption that only specialized "real-time" hardware can provide the necessary guarantees. HBM's architectural shift (3D stacking, wider interfaces, independent channels) fundamentally changes how memory contention manifests, offering new avenues for isolation and latency reduction previously unavailable with planar DRAM.

The methodology of isolating each HBM feature's impact through synthetic experiments (Figure 14) is particularly insightful. It allows researchers to understand the specific mechanisms by which HBM improves predictability, rather than just observing aggregate performance gains. This granular understanding is crucial for designing future predictable memory controllers and memory management units that can fully leverage these features.

A potential area for improvement or further exploration, building on the paper's identified limitations, would be a detailed study on how to design a real-time HBM memory controller. The current work provides the "what" and "why" of HBM's benefits, but the "how" of implementing these in a controller (e.g., optimal scheduling, arbitration policies that maximize isolation while respecting real-time constraints) remains an open challenge. Additionally, while the paper uses standard benchmarks, exploring the behavior with more diverse and complex mixed-criticality workloads, including those involving dynamic memory access patterns, could further strengthen the findings. The single-core setup in the simulations, while simplifying the analysis of the memory device itself, doesn't fully capture multi-core contention scenarios, which is a major concern in real-time systems.

In summary, this paper provides a robust foundation for integrating HBM into real-time systems, opening up exciting possibilities for next-generation critical embedded applications that demand both high performance and stringent timing guarantees. Its methods and conclusions could potentially be transferred to the analysis of other emerging memory technologies (e.g., CXL-attached memory, non-volatile memories) from a real-time predictability perspective.