1. Bibliographic Information

1.1. Title

FogBus: A Blockchain-based Lightweight Framework for Edge and Fog Computing

1.2. Authors

• Shreshth Tuli (a,b)

• Redowan Mahmud (a,*)

• Shikhar Tuli (c)

• Rajkumar Buyya (a)

  Affiliations:

• (a) Cloud Computing and Distributed Systems (CLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Parkville, VIC 3010, Australia

• (b) Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India

• (c) Massachusetts Institute of Technology, Cambridge, MA 02139, USA

1.3. Journal/Conference

Published by Elsevier Inc. The paper was received on October 2, 2018, revised on March 13, 2019, accepted on April 12, 2019, and made available online on April 13, 2019. Elsevier is a highly reputable publisher of scientific, technical, and medical content, indicating a significant peer-review process and a strong standing within academic communities.

1.4. Publication Year

2019

1.5. Abstract

The paper proposes FogBus, a lightweight framework designed to integrate Edge, Fog, and Cloud infrastructures, addressing limitations in existing frameworks such as platform independence, security, resource management, and multi-application execution. FogBus offers platform-independent interfaces for IoT applications and computing instances, assisting developers, users (for running multiple applications), and service providers (for resource management). It incorporates Blockchain, authentication, and encryption for secure data operations. Due to its simplified, cross-platform software systems, it is easy to deploy, scalable, and cost-efficient. The effectiveness of FogBus is demonstrated through a real-world computing environment for Sleep Apnea analysis, integrating finger pulse oximeters as IoT devices with Smartphone-based gateway and Raspberry Pi-based Fog nodes. Experimental evaluations show that FogBus is comparatively lightweight and responsive, and its various settings can tune the computing environment according to situational demands.

1.6. Original Source Link

/files/papers/691b152480efabfa5c6ee4c3/paper.pdf (This link is a relative path; assuming it refers to an internal file system or a specific host that serves these files.) Publication status: Officially published by Elsevier Inc.

2. Executive Summary

2.1. Background & Motivation

The rapid growth of the Internet of Things (IoT) paradigm, with trillions of devices expected by 2025, leads to an increased demand for latency-sensitive and compute-intensive applications. Traditional Cloud-centric IoT models face significant challenges:

• High Latency: Cloud datacenters are geographically distant from IoT devices, causing communication delays that drastically affect the Quality of Service (QoS) for critical applications like remote patient monitoring or disaster detection.

• Network Congestion: The massive volume of data generated by IoT devices can overwhelm network infrastructure when simultaneously transferred to the Cloud.

• Heterogeneity and Resource Constraints: Fog and Edge computing emerged to address these issues by bringing computation closer to IoT data sources. However, Fog nodes are often resource-constrained and highly heterogeneous in terms of computing capabilities, operating systems, and communication standards, making seamless integration challenging.

• Security and Management: Existing frameworks for IoT-Fog-Cloud integration often lack platform independence, fail to support simultaneous execution of multiple applications, offer limited scope for customization, rely on centralized management (degrading QoS), and have narrow security focuses, increasing vulnerability.

  The core problem FogBus aims to solve is the need for a secure, platform-independent, lightweight, and efficient framework that can seamlessly integrate heterogeneous IoT, Fog, and Cloud infrastructures to support diverse and demanding IoT applications.

2.2. Main Contributions / Findings

The paper's primary contributions are:

• Proposed Lightweight Framework (FogBus): Introduction of FogBus for end-to-end IoT-Fog-Cloud integration, designed to harness both edge and remote resources based on application requirements. It emphasizes a lightweight design suitable for resource-constrained Fog nodes.

• Platform-Independent Architecture: Design of a system that offers platform-independent application execution and node-to-node interaction, effectively overcoming the heterogeneity of computing instances across operating systems and peer-to-peer communication standards.

• Enhanced Security with Blockchain: Integration of Blockchain technology to ensure data integrity during the transfer of confidential data, complemented by authentication and encryption techniques for data protection and privacy.

• Support for Diverse Applications and Management Policies: FogBus is designed to assist in implementing various concurrent/application programming models (e.g., SPMD, workflows, streams) and allows service providers to apply customized resource management and scheduling policies.

• Real-World Case Study and Evaluation: Development of a prototype application system for Sleep Apnea data analysis, demonstrating FogBus's effectiveness in a healthcare scenario. Comprehensive evaluation of FogBus's characteristics (lightness, responsiveness) against existing frameworks and the impact of its settings on system parameters (service latency, energy consumption, network usage).

  The key findings from the evaluation are:

• Lightweight and Responsive: FogBus demonstrates lower CPU and RAM usage, shorter system initiation times, and faster data retrieval compared to other existing frameworks.

• Efficient Resource Management: It achieves a comparatively lower QoS expectation miss rate by effectively harnessing both edge and remote resources (Cloud) to handle workloads.

• Tunable Performance: Different FogBus settings (e.g., With/Without Interval, With/Without Blockchain, Fog/Cloud Only/Integrated) allow for flexible tuning of the computing environment to balance performance, security, and resource consumption based on specific application demands and situational constraints.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand FogBus, a foundational grasp of the following concepts is essential:

• Internet of Things (IoT): A network of physical objects ("things") embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems over the internet.

  • Sensors: Devices that detect and respond to changes in the physical environment (e.g., temperature sensors, pulse oximeters).
  • Actuators: Devices that convert an electrical signal into a physical action, often triggered by IoT data analysis (e.g., smart light switches, robotic arms).
  • Cyber-Physical Systems (CPS): Systems that integrate computation, networking, and physical processes. IoT is a key enabler for CPS, allowing real-time interaction with the physical world.

• Cloud Computing: A model for delivering computing services—including servers, storage, databases, networking, software, analytics, and intelligence—over the Internet ("the cloud"). It typically involves large, centralized datacenters.

  • Infrastructure as a Service (IaaS): Provides virtualized computing resources over the internet.
  • Platform as a Service (PaaS): Provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app.
  • Software as a Service (SaaS): A method of software delivery that allows data to be accessed from any device with an internet connection and a web browser.

• Fog Computing / Edge Computing: Paradigms that extend Cloud computing to the edge of the network, bringing computation, storage, and networking closer to the IoT devices that generate data. This reduces latency and network congestion.

  • Edge Computing: Generally refers to computation performed at the "edge" of the network, often on the IoT device itself or a gateway very close to it.
  • Fog Computing: Often seen as a broader concept encompassing Edge computing, forming an intermediate layer between IoT devices and the Cloud. Fog nodes can be distributed across various locations, ranging from smart devices (e.g., Raspberry Pi, smartphones) to micro-datacenters.
  • Fog Gateway Nodes (FGNs): Entry points for IoT devices into the Fog environment, often responsible for data filtration, aggregation, and initial processing.
  • Fog Computational Nodes (FCNs): Fog nodes equipped with processing power, memory, and storage to perform more substantial computations.

• Blockchain: A decentralized, distributed, and immutable ledger that records transactions across many computers. Each "block" in the chain contains a timestamp and a hash of the previous block, linking them together. This structure makes it highly resistant to modification.

  • Hash Value: A fixed-size string of characters that uniquely identifies a block's data. Any change in the data results in a different hash value.
  • Proof-of-Work (PoW): A mechanism requiring computational effort to create new blocks, deterring malicious activities by making data tampering computationally expensive.
  • Digital Signature: A mathematical scheme for verifying the authenticity of digital messages or documents, ensuring data source integrity.
  • Data Integrity: Ensuring that data is accurate, consistent, and unaltered over its entire lifecycle.

• Platform Independence: The ability of software to run on different operating systems (OSs) or hardware architectures without modification. This is crucial in heterogeneous Fog environments.

  • Java Virtual Machine (JVM): A virtual machine that enables a computer to run Java programs as well as programs written in other languages that are compiled to Java bytecode. JVM is key to Java's platform independence.
  • HTTP (Hypertext Transfer Protocol) / RESTful APIs: Standardized protocols for communication over networks, enabling web-based interactions between diverse systems. RESTful APIs (Representational State Transfer Application Programming Interfaces) use HTTP methods to perform operations.

• Quality of Service (QoS): The overall performance of a service, particularly in telecommunications and computer networking. Key metrics include latency (delay), throughput (data rate), reliability (availability), and availability.

• SPMD (Single Program Multiple Data): A parallel computing programming model where multiple autonomous processors simultaneously execute the same program on different portions of data. This is relevant for distributed IoT application processing.

3.2. Previous Works

The paper categorizes existing frameworks into two types: application-specific prototypes and generalized PaaS models.

• Application-Specific Prototypes:

  • Rahmani et al. (2018): A prototype for IoT-enabled health-care with smart gateways for local storage and processing, using Cloud as a back-end. Limitation: Relies on OS-level security, lacks generalized platform independence.
  • Dubey et al. (2017): Another smart health-care prototype using Intel Edison and Raspberry Pi as Fog nodes. Employs role-based authentication. Limitation: Cloud partially adopted, narrow security.
  • Azimi et al. (2017): Hierarchical health-care framework using MAPE-K model for computations, with analytics split between Cloud and Fog. Provides data encryption. Limitation: No explicit mention of platform independence or robust resource management for diverse applications.
  • Gia et al. (2017): Low-cost health monitoring, where IoT devices pre-process data and Fog layer manages distributed databases for security. Limitation: Focus on IoT device capabilities may increase energy consumption, limited scope for multi-application support.
  • Akrivopoulos et al. (2017): Health data sharing and emergency notification. Spark IoT Platform Core in the Cloud manages operations, using encryption and authentication. Limitation: Cloud-centric management can lead to latency issues, limited Fog resource leverage.
  • Chen et al. (2017) & Craciunescu et al. (2015): Prototypes for smart city surveillance and gas-leak monitoring. Fog handles processing; Cloud stores data. Limitation: Lack explicit security features or heterogeneity management.
  • Hu et al. (2017): Face identification prototype. Cloud manages resources and offloads tasks to Fog. Limitation: Omissions on security, centralized Cloud management.

• Generalized PaaS Models:

  • Yangui et al. (2016): Cloud-centric PaaS for IoT-Fog-Cloud integration, automating application provisioning. Handles node heterogeneity, extends Cloud Foundry security. Limitation: Still Cloud-centric, potential for latency and congestion.
  • Bruneo et al. (2016) (Stack4Things): Fog-centric PaaS for multi-application deployment on IoT devices. Fog acts as a centralized programmable coordinator. Applies Cloud-based security. Limitation: Centralized Fog coordination can be a single point of failure or bottleneck, Cloud-based security might not be optimized for Fog.
  • Verba et al. (2017): Gateway architecture offering PaaS for Fog node and IoT device integration. Supports messaging with authentication, horizontal integration with Cloud. Limitation: Limited scope in security (only authentication), resource management policies not fully detailed.
  • Yi et al. (2015a) (Cloudlet-based PaaS): Requires resource-enriched Fog nodes and virtualization of every node. Uses existing authentication techniques. Limitation: High resource requirements for Fog nodes, virtualization overhead.
  • Vatanparvar et al. (2015): PaaS for electricity usage management. Deals with heterogeneity, encrypts data. Limitation: Specific application domain, general applicability might be limited.
  • Chang et al. (2017) (Indie Fog): Utilizes user's networking devices for IoT applications. Extends core services from Cloud to Fog. Supports Cluster of Fog nodes, registry service for security. Limitation: Cloud-dependent core services, registry might be a bottleneck, security limited to registry.
  • Mohamed et al. (2017): Service-oriented framework for smart-city services. Classifies services into core (resource management, security) and application-specific. Uses authentication and access control. Limitation: Security exploited from a narrow perspective, computing capabilities of edge/remote resources not fully leveraged.

3.3. Technological Evolution

The IoT paradigm initially relied on Cloud computing for data storage and processing. However, as IoT devices proliferated and generated vast amounts of data, and as applications demanded real-time, latency-sensitive responses, the Cloud-centric model proved inadequate due to inherent delays and network congestion. This led to the emergence of Fog and Edge computing in the early 2010s (e.g., Bonomi et al., 2012), pushing computation closer to the data sources.

Early Fog/Edge frameworks focused on specific applications (e.g., healthcare, smart cities) or offered basic PaaS capabilities, often facing limitations in handling heterogeneous devices, providing comprehensive security, or supporting multiple applications concurrently. Many were also Cloud-dependent or employed centralized management, reintroducing single points of failure and bottlenecks.

FogBus fits into this evolution by attempting to overcome these persistent challenges. It leverages advances in distributed systems, virtualization, and security technologies like Blockchain to provide a more robust, platform-independent, and lightweight solution for orchestrating IoT, Fog, and Cloud resources. It moves beyond specific application prototypes or partially generalized PaaS to offer a more comprehensive and flexible framework that explicitly addresses heterogeneity, security, and resource management in a decentralized manner, while maintaining a lightweight footprint suitable for resource-constrained Fog environments.

3.4. Differentiation Analysis

Compared to the main methods and frameworks in related work, FogBus introduces several core differences and innovations:

• Comprehensive Platform Independence: While some frameworks (e.g., Yangui et al., 2016, Bruneo et al., 2016, Vatanparvar et al., 2015) address heterogeneity to some extent, FogBus explicitly designs its System Services (using PHP for Service Interface and Java for Management Activity) to ensure OS and P2P communication-level heterogeneity is overcome across all hardware instruments. This includes IoT devices, FGNs, FCNs, and Cloud datacenters.

• Integrated and Layered Security: Unlike frameworks that exploit security from a "narrow perspective" (e.g., Dubey et al., 2017 with role-based authentication, Verba et al., 2017 with authentication, Chang et al., 2017 with a registry service), FogBus provides a multi-faceted security approach. It integrates:

  • Blockchain for Data Integrity: This is a significant differentiator. FogBus uses Blockchain to ensure data tampering prevention and source authentication, which is not commonly found in the surveyed frameworks.
  • Authentication and Encryption: Alongside Blockchain, it applies traditional authentication (user credentials, port knocking, privileged port authentication) and attribute-based encryption for sensitive data privacy.

• Lightweight Design with Decentralized Management: Many existing frameworks either push computation to IoT devices or resource-enriched Fog nodes (increasing cost and energy) or rely on centralized management (degrading QoS). FogBus is designed to be lightweight, as evidenced by its lower CPU and RAM usage. It promotes a decentralized bottom-up approach for resource management (e.g., broker nodes distribute tasks and coordinate activities, GCNs form clusters), reducing reliance on Cloud-centric or single-point-of-failure Fog-centric control.

• Multi-Application and Service Tuning Support: The paper criticizes existing frameworks for barely supporting simultaneous execution of multiple applications and offering "narrow scope to application developers and users to tune the framework." FogBus explicitly addresses this by facilitating multi-application execution, concurrent programming models (SPMD, workflows, streams), and offering "service tuning facility to both users and providers."

• End-to-End IoT-Fog-Cloud Integration: While others integrate parts, FogBus aims for a truly end-to-end integration, where resources can be dynamically provisioned from Fog to Cloud based on application needs and Fog load, ensuring QoS and resilience.

  In essence, FogBus differentiates itself by offering a more holistic, secure, and flexible solution that explicitly tackles the core challenges of heterogeneity, security, and resource management in IoT-Fog-Cloud environments through a lightweight, platform-independent, and decentralized architecture, with Blockchain as a key innovation for data integrity.

4. Methodology

4.1. Principles

The core idea behind FogBus is to create a robust, secure, and flexible framework that seamlessly integrates diverse hardware instruments (from IoT devices to Cloud datacenters) into a unified computing environment. It addresses the challenges of latency, network congestion, heterogeneity, and security in IoT-Fog-Cloud systems by employing platform-independent software components, a master-worker network topology, and Blockchain technology for data integrity. The framework is designed to be lightweight and responsive, enabling IoT applications to be executed closer to the data sources while leveraging Cloud resources when needed, thereby optimizing QoS and resource utilization. It aims to empower developers to build applications, users to run multiple applications, and service providers to manage resources efficiently and securely.

4.2. Core Methodology In-depth (Layer by Layer)

FogBus facilitates IoT-Fog-Cloud integration through three main architectural layers: Hardware Instruments, Software Components (System Services), and a Network Structure. The implementation details describe how these layers are realized using specific technologies.

4.2.1. Hardware Instruments

The foundation of FogBus is built upon heterogeneous hardware instruments:

• IoT devices: These are typically energy and resource-constrained devices equipped with sensors (to perceive external environments) and actuators (to convert commands into physical actions). They primarily act as data producers or consumers, though some may have limited computational capabilities for pre-processing raw data. FogBus connects IoT devices to proximate Fog Gateway Nodes (FGNs) via wireless or wired protocols like Zigbee, Bluetooth, and NFC. The sensing frequency and data format are tunable and device-specific.

• Fog Gateway Nodes (FGNs): These are the entry points for IoT devices into the distributed computing infrastructure. FGNs perform several critical functions:

  • Configuration: Assist IoT devices in getting configured within the integrated environment.
  • User Interface: Offer a front-end program for applications, allowing users to set authentication credentials, access back-end programs, convey service expectations, receive outcomes, manage IoT devices, and request resources.
  • Data Management: Operate data filtration, organize data into a general format, and aggregate data from multiple IoT sources. For large-scale processing, FGNs forward data to other computing instances.
  • Communication: Maintain rapid and dynamic communication with accessible Fog Computational Nodes (FCNs) using protocols such as Constrained Application Protocol (CoAP) or Simple Network Management Protocol (SNMP).

• Fog Computational Nodes (FCNs): FogBus can manage numerous heterogeneous FCNs, which are equipped with processing cores, memory, storage, and bandwidth to perform various FogBus operations. FCNs can take on three different roles:

  1. Broker nodes: These FCNs initiate data processing for IoT applications when an FGN connects to them. If the broker node cannot meet an application's resource requirements locally, it acts as a broker, communicating with other FCNs and Cloud datacenters to provision necessary resources. Its responsibilities include:
     • Distributing computational tasks across computing instances.
     • Seamlessly monitoring and coordinating activities.
     • Conducting load balancing.
     • Ensuring reliability through security features and fault-tolerant techniques like Blockchain and replication during interactions.
  2. General Computing Nodes (GCNs): These FCNs are used for general computing purposes and are accessible only via broker nodes, which act as a firewall for GCNs. Broker nodes explicitly manage the GCNs' resources and forward data along with executable back-end applications for processing. GCNs form clusters under the supervision of a specific broker node when executing distributed applications and can relate to multiple broker nodes. To handle concurrent commands from different broker nodes, Vector Clocks are used for synchronizing the system.
     • Vector Clocks: These are data structures used in distributed systems to determine the partial ordering of events. In FogBus, Vector Clocks help GCNs identify concurrent commands issued by different broker nodes. GCNs then arbitrarily order these commands and notify the corresponding broker nodes. For application-level consistency, a GCN executes at most one application at a time when carrying out a command.
  3. Repository nodes: These are FCNs dedicated to managing a distributed database for data sharing, replication, recovery, and secured storage. They provide interfaces for instant access and analysis of historical data, maintain meta-data of applications (model, runtime requirements, dependencies), and can preserve intermediate data during execution for anomaly-driven restart points. To ensure data-level consistency, repository nodes manage all data in a log-structured manner, driven by their updating timestamp and source.

• Cloud datacenters: When the Fog infrastructure becomes overloaded or when service requirements are latency-tolerant, FogBus extends resources from Cloud datacenters to execute back-end IoT applications. This expands the computing platform globally and, in association with Fog repository nodes, facilitates extensive data storage and distribution, making data access and processing location-independent.

4.2.2. Software Components

FogBus uses a set of interrelated software components, broadly classified into three System Services, to manage OS and P2P communication-level heterogeneity:

• Broker Service: Manages the functionalities of a broker node and initiates other software components.

  • Broker Security Manager:
    • Verifies user authentication credentials (in association with Credential Archive of Repository Service).
    • Generates public and private key value pairs for port knocking, privileged port authentication, and attribute-based encryption to secure communication.
    • Acts as the Blockchain interface to ensure data integrity. With the help of the Data Manager, it creates new blocks from received data. The hash values and proof-of-work for each block are sent to Credential Archive for distribution to other nodes, ensuring consistent verification.
  • Resource Manager:
    • Selects suitable resources by identifying application requirements (Application Catalogue of Repository Service) and perceiving resource status (Resource Monitor of Computing Service).
    • Uses Cloud Integrator for contextual data of Cloud-based instances (VMs, containers).
    • Provisions resources on FCNs and Cloud considering the heterogeneity of computing instances (processing capabilities).
    • Facilitates service providers to apply various resource provisioning policies.
    • Maintains a resource configuration file tracking FCN and Cloud instance addresses and deployed applications, shared with Cloud for failure recovery.
  • Data Manager:
    • Receives sensed and pre-processed data from IoT devices, aggregates data, and calibrates data receiving frequency.
    • Creates blocks and chains for integrity (with Broker Security Manager).
    • Forwards data to Application Executor (Computing Service) for processing.
    • Stores encrypted data in Data Container (Repository Service) for future use.
    • Sends subsequent data streams directly to allocated resources using the resource configuration file.
  • Cloud Integrator:
    • Handles all FogBus interactions with Cloud.
    • Notifies Cloud instance context to the framework.
    • Forwards storage and resource provisioning commands to the Cloud.
    • Offers interfaces for customized Cloud-integration scripts and allows third-party software (e.g., Aneka) to interact with multiple Cloud datacenters via APIs.

• Repository Service: Runs across Fog nodes to manage repository-related operations.

  • Credential Archive:
    • Preserves user authentication credentials.
    • Distributes security keys and details of each data block generated by Broker Service.
    • Provides Secure Socket Layer (SSL) and Transport Layer Security (TLS) certificates for Cloud integration.
    • Supports Data Container for encrypting/decrypting stored data.
    • Periodically updates its image on Cloud (via Cloud Dilator) for security attribute recovery.
  • Application Catalogue:
    • Maintains details about applications: operations, recommended system properties, execution/programming model, resource requirements, and task dependencies.
    • Can extend information from Cloud via Cloud Dilator.
    • Assists Resource Manager in provisioning resources and Application Executor in assembling applications.
  • Data Container:
    • Stores IoT data for long-term analysis, ensuring privacy through encryption.
    • Receives intermediate data from Application Executor for fault tolerance (restarting processing).
    • Allows customization and sharing of database schemas.
    • Maintains association with Cloud Dilator to grasp remote data and disperse local data through Cloud.
  • Cloud Dilator: Facilitates Repository Service components to interact with Cloud by using commands from Cloud Integrator.

• Computing Services: Responsible for controlling the operations of a GCN or when a broker node executes back-end applications itself.

  • Executor Security Manager:
    • Manages seamless and secure interactions of an FCN with others during computing operations.
    • Assisted by Credential Archive for security attributes.
    • Plays a significant role in verifying Blockchains (along with Credential Archive and Broker Security Manager).
  • Resource Monitor:
    • Monitors busy and idle status (e.g., CPU usage, memory occupation, network utilization, power consumption) of computing resources with Application Executor.
    • Based on this, Resource Manager provisions resources.
    • Tracks QoS performance, notifies Resource Manager on overload or failure, triggering actions like dynamic resource provisioning, application execution migration, and intermediate data storage (currently abstract for customization).
    • Conducts operations for system synchronization.
  • Application Executor:
    • Allocates resources for applications on FCNs based on Resource Manager's instructions.
    • Extends application executables from Application Catalogue for deployment.
    • Receives data from Data Manager for processing.
    • Periodically informs Resource Monitor of resource status.
    • Extracts intermediate data and stores it in Data Container on anomaly detection/prediction, ensuring fault tolerance.

4.2.3. Network Structure

The FogBus network structure (Figure 3 from the original paper) is designed for persistent, stable, secure, scalable, and fault-tolerant communication among hardware instruments.

该图像是FogBus框架的网络结构示意图，展示了云、路由器、物理设备以及各类节点（包括Broker节点、计算节点和存储库节点）之间的连接关系。此框架用于支持边缘和雾计算中的物联网应用集成。

Fig. 3. Network Structure of FogBus framework.

• Topology: Employs a master-worker topology. Broker nodes are masters, and other FCNs are workers.

  • Master (broker node): Receives data/user info from FGNs, discovers workers for processing/storage, manages worker functionalities during runtime, delivers service results to FGNs, and connects Fog to Cloud.
  • Workers (other FCNs): Communicate among themselves under master supervision for data sharing and overhead reduction.
  • All (masters, workers, FGNs) connect to a common WLAN managed by one or more routers.

• Scalability:

  • Service providers can scale up the number of active Fog nodes. An FCN can become a worker by making itself accessible to a master, which then configures necessary software.
  • Supports coexistence of multiple masters in a WLAN, providing FGNs diverse options for data stream dispatch.
  • Masters can share workers, ensuring data integrity and privacy as each master maintains its own chain of blocks and separate database on workers.
  • Software components at masters facilitate integration with multiple Cloud datacenters simultaneously.

• Reliability: (Figure 4 from the original paper illustrates this concept.)

  
  该图像是示意图，描绘了FogBus框架中用户与计算节点之间的关系。图中展示了从用户到主节点的转变，以及工作节点如何在条件转变下被转换为主节点，以确保系统的可靠性和灵活性。

  Fig. 4. Ensuring reliability in FogBus framework.

  • Multiple Masters: Implicitly eradicates single point of failure of the master-worker model.
  • Synchronization: Vector Clock-based synchronization and one-to-one interaction ensure explicit isolation and consistency of operations on different worker nodes.
  • Master Replication: Each master can replicate its image over one of its worker nodes. If a master fails, the worker gains master privileges, preventing network collapse. This relies on the platform-independent nature of FogBus software.
  • Worker Monitoring & Recovery: Masters periodically check worker status and store their intermediate data and configurations (including deployed applications). If a worker fails, masters share its information with other workers for immediate residual data processing. If all workers of a master are overloaded, workers of other masters are considered (with internal communication among masters). This ensures continuous computation.

• Security:

  • Authentication: FGN and FCN authentication is handled by WLAN routers.
  • Network-level Controls: Masters apply network-level access control and packet filtration to resist infrastructure compromise and eliminate malicious content.
  • Redundant Links: Multiple communication links exist to different Fog nodes, allowing routing path readjustment on network anomaly.
  • Cloud Security: Leverages Cloud-provided network security policies when interacting with Cloud infrastructures.

• Performance:

  • Dedicated Network Bandwidth: Utilizes bandwidth dedicatedly for a specific system, preventing degradation from external entities.
  • Scalability Efficiency: Adding FCNs does not require very high additional network resources.
  • Service Delivery: Facilitates easy deployment and faster service delivery.
  • Throughput: As a localized network, its throughput remains acceptable.
  • Resource Adjustment: Supports periodic adjustment of network resources for any frequency and volume of incoming traffic.
  • Cost Efficiency: Less dependence on external hardware for network management reduces capital expenditure (CapEx) and operational expenditure (OpEx).

4.2.4. Design and Implementation

The implementation ensures platform independence across diverse hardware by using compatible APIs, execution environments, scripting, and programming languages.

• System Services: Each System Service (Broker, Repository, Computing) is divided into a Service Interface and Management Activity.

  • Service Interface:
    • At masters: Receives data/user specifications from gateways, presents service results, allows providers to notify worker IP addresses.
    • At workers: Acts as a receptor, decodes application output to masters.
    • Implemented as web programs using PHP (an HTML-embedded server-side scripting language).
    • Uses HTTP protocol-based RESTful APIs for data exchange and information sharing among FCNs within the WLAN.
    • PHP and HTTP are cross-platform and widely supported by OSs (Unix, Windows, Linux, NetWare) and IoT devices, ensuring OS and P2P communication-level independence.
    • An Apache server is deployed in each FCN to run the web programs.
  • Management Activity:
    • At masters: Contains resource provisioning and load balancing policies, updates configuration files, generates and forwards commands to workers.
    • At workers: Performs resource allocation, monitoring, status sharing, stores data in relational databases, creates input files for back-end programs.
    • Developed in Java programming language. Compiled Java programs run on Java Virtual Machine (JVM), which is easily installed across various OSs, enabling wide platform functionality.
    • MySQL servers are installed in FCNs for database management.
  • Security: Service Interface and Management Activity jointly handle encryption and authentication for masters and workers, in addition to Blockchain.

• Blockchain:

  • Implemented in Java programming language.
  • Masters create blocks from received data.
  • The hash of each block is calculated using the SHA256 algorithm based on the data, hash of the previous block, timestamp, and a nonce value.
  • Masters generate random public/private key pairs dynamically per block to create unique digital signatures with the original data.
  • Blockchain details, digital signature attributes, and Base64 encoded data are shared with workers.
  • Workers use the master's public key to verify the data's legitimate source, rejecting data from other keys.
  • Workers verify each block and its hash by mining the nonce value to support the proof-of-work.
  • If a worker reports tampering or forgery, the Blockchain from the majority of the network is copied to that node.
  • Service Interface at masters displays latest hashes of the Blockchain copy at each worker, allowing users/providers to track data flow and suspicious activity.

• Cloud Plugin:

  • Activated only when users specify Cloud integration for data processing.
  • Deployed on the master.
  • Offers flexibility for customized or third-party Cloud Plugin services.
  • In the current version, Aneka (a PaaS framework for Cloud-based applications management) is used.
  • Aneka offers APIs for provisioning and scheduling resources in a heterogeneous Aneka-Cloud (public, private, hybrid). It supports programming models like Bag of tasks, Distributed threads, MapReduce, and Parameter sweep.
  • The Aneka-based Cloud plugin in FogBus specifies Cloud instance IP addresses.
  • It initiates task or thread models in Aneka-Cloud for single/multiple Cloud instances.
  • FogBus's built-in resource provisioning policy prioritizes Fog infrastructure. If Fog load exceeds a threshold, the application and input data stream are referred to Cloud.
  • Management Activity at the master stores data in a Cloud input file. The Aneka-based Cloud plugin parses this file every 500 milliseconds polling period, checks for pending data, forms tasks/threads, encapsulates data, and launches them to Aneka-Cloud instances. Blockchain is applied here too.

• Application: FogBus supports diverse IoT applications, structured into front-end and back-end programs.

  • Front-end program:
    • Runs in FGNs (e.g., Android, iOS, Windows, Tizen, WebOS, RTOS). Must be developed in a language supported by these platforms.
    • May require temporary data storage, using compatible database systems and schemas.
    • Handles data from IoT devices, supporting Bluetooth Low Energy (BLE) for energy-constrained devices.
    • Correlates with the Service Interface at masters for data/user info forwarding and service outcome reception. Its user interface should easily parse the master's Service Interface web programs.
  • Back-end program:
    • Executed in FCNs.
    • Preferably distributed and modular to leverage FCN capabilities.
    • Uses cross-platform programming languages like Java to overcome OS-level heterogeneity.
    • Scripts should specify points for storing intermediate data during execution for fault tolerance.
    • Must be able to extract input files and update output files at workers.

4.2.5. Formulas and Key Mechanisms (as described in paper)

The paper describes the conceptual application of SHA256 for Blockchain hashing and Vector Clocks for synchronization without providing explicit mathematical formulas for these standard concepts. My explanation remains faithful to this.

Blockchain Hashing (SHA256): The hash of each block is calculated by the masters using the SHA256 algorithm. The components contributing to this hash are:

• The data contained within the block.

• The hash of the previous block in the chain.

• The timestamp of when the data was received.

• A nonce value, which is a random number used in proof-of-work.

  Conceptually, the hash function $H$ operates as follows: $ H_{\text{current_block}} = \text{SHA256}(\text{Data}, H_{\text{previous_block}}, \text{Timestamp}, \text{Nonce}) $ This process ensures that any change to the data, previous hash, timestamp, or nonce would result in a different current block hash, invalidating the chain unless recomputed. The nonce is mined to find a hash that follows a specific pattern, fulfilling the proof-of-work requirement.

Vector Clocks for Synchronization: Vector Clocks are used by GCNs to handle concurrent commands issued by multiple broker nodes. A Vector Clock for a process (or GCN in this case) is an array of logical clocks, one for each process in the distributed system. When a GCN receives a command from a broker node, it updates its Vector Clock and compares it to the Vector Clock received with the command. This allows the GCN to:

• Identify if commands are concurrent (neither happened before the other).
• Establish a partial ordering of events.
• In FogBus, GCNs use this to arbitrarily order concurrent commands and notify the issuing broker nodes for consistency.

5. Experimental Setup

5.1. Datasets

The paper uses a real-world application for Sleep Apnea analysis as a case study. The data is collected from finger pulse oximeters which act as IoT devices.

• Source: Jumper JPD-500F Finger Pulse Oximeter.
• Characteristics: Collects peripheral capillary oxygen saturation (SpO2) and heart beat rate data. Each data signal length is 18 KB with a sensing frequency of 2 signals per second. The data is encoded in UTF-8.
• Domain: Healthcare, specifically for diagnosing and monitoring Sleep Apnea, a condition where breathing repeatedly stops and starts during sleep.
• Data Sample (Conceptual): The Android interface receives data values from the Oximeter which are then appended to a list. This list contains timestamp, heart beat rate, and blood oxygen level readings.
  • Example: A data entry might look like [Timestamp, HeartRate, SpO2], e.g., [1678886400, 72, 95].
• Purpose: This dataset is chosen because Sleep Apnea analysis is a latency-sensitive application where timely data processing is crucial for patient monitoring, making it an ideal candidate to demonstrate the benefits of Fog and Edge computing within FogBus. The SpO2 and heart rate data are used to calculate the Apnea-Hypopnea Index (AHI), which determines the intensity of Sleep Apnea.

5.2. Evaluation Metrics

The paper evaluates FogBus using several performance metrics, which are explained below.

• Load on Resources: Quantifies the computational burden a framework imposes on its underlying hardware.

  1. Conceptual Definition: Measures how much of the processing power and memory capacity of a Fog node (especially the core/master/broker node) is consumed by the framework's operations. A lower load indicates a more lightweight framework, crucial for resource-constrained environments.
  2. Mathematical Formula: Not explicitly provided in the paper as a formal formula but typically measured as a percentage.
     • CPU Usage: $ \text{CPU Usage (%)} = \frac{\text{CPU Time Used}}{\text{Total CPU Time}} \times 100 $
     • RAM Usage: $ \text{RAM Usage (%)} = \frac{\text{Memory Used}}{\text{Total Available Memory}} \times 100 $
  3. Symbol Explanation:
     • CPU Time Used: The amount of time the CPU spends executing non-idle threads or processes.
     • Total CPU Time: The total time the CPU was available for execution during the measurement period.
     • Memory Used: The amount of RAM currently consumed by processes.
     • Total Available Memory: The total physical RAM installed and available in the system.

• QoS Expectation Miss Rate: Measures the frequency with which the framework fails to meet predefined Quality of Service targets.

  1. Conceptual Definition: Indicates the efficiency of the framework in managing and harnessing its computing resources to deliver services within expected performance parameters (e.g., latency, throughput). A lower miss rate signifies higher reliability and better performance in meeting user expectations.
  2. Mathematical Formula: Not explicitly provided, but generally calculated as: $ \text{QoS Expectation Miss Rate} = \frac{\text{Number of Requests Failing QoS Target}}{\text{Total Number of Requests}} $
  3. Symbol Explanation:
     • Number of Requests Failing QoS Target: The count of service requests that did not meet their specified QoS criteria (e.g., exceeding a maximum latency threshold).
     • Total Number of Requests: The total count of service requests submitted to the framework.

• Time-based Attributes: Metrics related to the speed and responsiveness of the framework.

  1. Conceptual Definition:
     • System Initiation Time: The duration required for all components within a framework to interconnect and become fully operational from a stopped state.
     • Data Retrieving Time: The time taken to access specific data through the framework, often from storage or another node.
  2. Mathematical Formula: Not explicitly provided, typically measured in units of time (e.g., seconds).
     • System Initiation Time: $T_{\text{initiation}}$ (measured in seconds).
     • Data Retrieving Time: $T_{\text{retrieval}}$ (measured in seconds).
  3. Symbol Explanation:
     • $T_{\text{initiation}}$: Time elapsed from the start command to the point where the system is ready to accept requests.
     • $T_{\text{retrieval}}$: Time elapsed from the data access request to the point where the data is successfully retrieved.

• Service Latency: The total time taken from when a service request is initiated until the result is delivered.

  1. Conceptual Definition: A critical QoS metric that combines communication delay and processing time. It represents the overall responsiveness of the system to a user's request.
  2. Mathematical Formula: As defined in the paper, it is the summation of network propagation delay and task completion/application execution time. $ L_{\text{service}} = D_{\text{network}} + T_{\text{execution}} $
  3. Symbol Explanation:
     • $L_{\text{service}}$: Total service latency.
     • $D_{\text{network}}$: Network propagation delay (time for data to travel across the network).
     • $T_{\text{execution}}$: Task completion or application execution time (time for the computation to finish).

• Network Usage: Measures the amount of network bandwidth consumed by the framework.

  1. Conceptual Definition: Quantifies the volume of data transmitted and received by the framework over the network during its operations. Lower network usage is beneficial for reducing congestion and operational costs, especially in resource-constrained environments or where bandwidth is limited.
  2. Mathematical Formula: Not explicitly provided, typically measured in units of data rate (e.g., Mbps - megabits per second).
     • Network Usage: $U_{\text{network}}$ (measured in Mbps).
  3. Symbol Explanation:
     • $U_{\text{network}}$: The average or peak data transfer rate across the network interfaces of the involved computing nodes.

• Energy Consumption: The amount of electrical energy consumed by the computing infrastructure while performing operations.

  1. Conceptual Definition: Measures the power used over time. Lower energy consumption is vital for cost efficiency and environmental sustainability, particularly for edge devices which may have limited power sources.
  2. Mathematical Formula: Not explicitly provided, typically measured in Joules or Watt-hours.
     • Energy Consumption: $E_{\text{consumption}}$ (measured in Joules).
  3. Symbol Explanation:
     • $E_{\text{consumption}}$: The total energy consumed by the Fog nodes and Cloud VMs during the execution of applications.

5.3. Baselines

The paper compares FogBus against implementations of three existing frameworks by following their guidelines:

• Stack4Things (Bruneo et al., 2016): A Fog-centric PaaS framework for deploying and executing multiple applications over computationally sound IoT devices, where Fog infrastructure acts as a centralized programmable coordinator.

• Cloudlet-based PaaS (Yi et al., 2015a): A comprehensive PaaS framework for integrated environments, which requires resource-enriched Fog nodes and virtualization of all nodes (including IoT devices).

• Indie Fog (Chang et al., 2017): A PaaS framework that utilizes user's networking devices to execute IoT applications, extending core services and resource management instructions from Cloud datacenters to Fog nodes.

  These baselines are chosen because they represent prominent efforts in IoT-Fog-Cloud integration and PaaS models in the literature, allowing for a comparative evaluation of FogBus's lightweight nature, resource management capabilities, and responsiveness against contemporary solutions.

5.4. Experiment Parameters

The following are the results from Table 2 of the original paper:

The "Interval between creating sequential data processing requests seconds" parameter in the table has no value specified in the paper, which is an oversight in the provided table content. However, the experimental results discuss With Interval and Without Interval settings, implying that an interval value would be defined in specific experiment scenarios.

5.5. System Configuration (Sleep Apnea Analysis Prototype)

The prototype for Sleep Apnea analysis uses the following hardware and software configuration:

• IoT Device: Jumper JPD-500F Finger Pulse Oximeter (1.5V, Bluetooth Low Energy v4.2 (BLE), UTF-8 data encoding).

• Gateway (Smartphone): Oppo A73 CPH1725 running Android 7.1.1.

• Broker/Master Node: Dell Latitude D630 Laptop (Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz, 2GB DDR2 RAM, 32-bit Windows 7, Apache HTTP Server 2.4.34, Java SE Runtime Environment (JRE) 1.6, MySQL 5.6, .net 3.5, Aneka 3.1).

• Other FCN/Worker Node: Raspberry Pi 3 B+ (ARM Cortex-A53 quad-core SoC CPU @ 1.4GHz, 1GB LPDDR2 SDRAM, IEEE 802.11, 64-bit Raspbian Stretch, Apache HTTP Server 2.4.34, JRE 1.6, MySQL 5.6).

• Public Cloud: Microsoft Azure B1s Machine (1vCPU, 1GB RAM, 2GB SSD, Windows Server 2016, .NET 3.5, Aneka 3.1). The real implementation of this system model is shown in Figure 6 from the original paper.

  该图像是FogBus基于的睡眠呼吸暂停分析的实际实现。图中展示了两个显示器和一台笔记本电脑，连接了多个手指脉搏血氧仪作为物联网设备，以及底部的Raspberry Pi节点，构成了计算环境。

Fig. 6. Real-world implementation of FogBus-based Sleep Apnea analysis

5.6. Installed Package

The prototype is primarily Fog infrastructure-centric, but can offload to Azure VM via the Aneka-based Cloud Plugin if the Fog infrastructure is overloaded. The application package consists of an Android front-end and a data analytic back-end program.

• Android Interface at Smartphone Gateway (HealthKeeper):

  • An Android executable named HealthKeeper is installed on the Smartphone.

  • Developed using MIT App Inventor (an open-source platform).

  • Home screen (Figure 7a): Allows the operator to pair the Oximeter with the Smartphone via Bluetooth and enter the master's IP address.

  • Session screen (Figure 7b): Manages all interaction with the master, including automatic data transmission. Initializes an empty data list and timer when recording starts, appends each data value from the Oximeter. When recording stops, the list is sent to the master. Displays results pulled from the master.

    该图像是图7，展示了健康管理应用HealthKeeper的安卓界面。左侧为主页，用户可以输入主节点的IP地址并配对传感器；右侧为会话界面，显示了睡眠数据，包括AHI（呼吸暂停-低通气指数）和最低氧气水平等信息，并提供了记录和分析功能。

  Fig. 7. (a) Home and (b) Session Screen of the android interface.

• Data Analytic at Worker Computing Nodes:

  • Encapsulates two open-source Java programs.
  • Stored in the repository worker and forwarded to computing workers for installation based on master commands.
  • Takes input data as a file, parsing the first, second, and third columns as timestamp, heart beat rate, and blood oxygen level respectively.
  • Uses a Boolean variable to track dips in oxygen level (below 88%) and a counter variable for Apnea-Hypopnea Index (AHI).
  • AHI based cases for Sleep Apnea analysis:
    • AHI < 5: Normal
    • 5 <= AHI < 15: Mild
    • 15 <= AHI < 30: Moderate
    • AHI >= 30: Severe
  • Additional information stored: minimum oxygen level, minimum/maximum heart rate, average heart rate, average rise/fall of heart rates.
  • Filters heart beat pattern during oxygen dips and generates an ECG.
  • Delivers results in a file, which the master's Service Interface parses and notifies the operator.

5.7. Sequence of Communication

The communication sequence for FogBus-enabled Sleep Apnea analysis within the same WLAN is presented in Figure 8 from the original paper:

该图像是示意图，展示了在睡眠呼吸暂停分析过程中，脉氧仪、智能手机、主控设备、计算工作者和仓库工作者之间的通信序列。流程包括设备开机、发送数据、请求分析和返回结果等步骤。

Fig. 8. Sequence of communication during Sleep Apnea analysis.

1. Oximeter Configuration: The Pulse Oximeter is configured with the Smartphone using operator credentials.
2. Data Sensing & Forwarding: The Oximeter senses SpO2 and heart beat rate and forwards this data to the Smartphone via Bluetooth.
3. Data to Master: The Smartphone sends the data to the master node.
4. Data Storage: The master stores the data on a repository worker.
5. Acknowledgement: The repository worker sends an acknowledgement to the master, which is then visible to the operator via the Smartphone (extending the master's Service Interface).
6. Analysis Request: After recording data for a period, the operator prompts the master via Smartphone to analyze the stored data.
7. Resource Allocation: The master communicates with a suitable computing worker and issues privileges for data analysis.
8. Data & Analytic Retrieval: The computing worker requests the stored data and the analytic executable from the repository worker.
9. Data Analysis: Upon reception, the computing worker starts the analysis.
10. Result to Master: Once analysis is finished, the result is sent back to the master.
11. Result Display: The Smartphone pulls the result from the master and displays it to the operator.

5.8. Experiment Settings

The application deployment evaluation considers the impact of Blockchain and management overhead under various FogBus settings:

1. With / Without Interval:

   • With Interval: The master waits for a certain interval after receiving the outcome of a previous request before sending the next data processing request. This reduces overhead on both the master and computing worker.
   • Without Interval: The master sends the next request immediately after the outcome of the previous request is available. This ensures continuous activity and no idle time.

2. With / Without Blockchain:

   • With Blockchain: The Blockchain security feature of FogBus is enabled, ensuring data integrity and authentication.
   • Without Blockchain: The Blockchain feature is disabled, reducing security overhead but potentially compromising integrity.

3. Fog / Cloud Only / Integrated:

   • Fog Only: Application execution is solely conducted on the Fog infrastructure.
   • Cloud Only: Application execution is solely conducted on the Cloud infrastructure.
   • Integrated: Computational tasks are distributed to both Fog and Cloud infrastructures according to the system's context (e.g., Fog first, then Cloud if Fog is overloaded).

5.9. Monitoring Tools

• Master and Azure VM: Microsoft Performance Monitor.
• Raspberry Pi circuits: NMoN Performance Monitor.

6. Results & Analysis

6.1. Core Results Analysis

The evaluation of FogBus's framework characteristics focuses on its lightness, resource management performance, and responsiveness compared to existing frameworks.

6.1.1. Lightness of Frameworks

Figure 9 from the original paper illustrates the CPU and RAM usage of the core/master/broker Fog node across different frameworks.

该图像是图表，展示了不同框架下核心节点的 (a) CPU 和 (b) RAM 使用情况。数据表明，FogBus 在 CPU 和 RAM 使用上相较于其他框架表现出更轻量级的特征。

Fig. 9. (a) CPU and (b) RAM usage of core node in different frameworks.

• Analysis: FogBus consistently shows lower CPU and RAM usage compared to Stack4Things, Cloudlet-based PaaS, and Indie Fog.
  • CPU Usage (Figure 9a): FogBus exhibits the lowest CPU utilization, indicating it imposes less computational burden on the core Fog node.
  • RAM Usage (Figure 9b): Similarly, FogBus consumes the least RAM, demonstrating its lightweight nature in terms of memory footprint.
• Reasons for FogBus's Advantage:
  • The other frameworks (Stack4Things, Cloudlet-based PaaS, Indie Fog) extensively incorporate third-party software systems, leading to higher resource consumption.
  • Cloudlet-based PaaS operates system functionalities without distributing operations across multiple Fog nodes, thus centralizing the load on the core Fog node.
  • Indie Fog and Stack4Things, despite operating collaboratively, assign additional responsibilities (e.g., federated resource, repository and security management, service discovery, application service management) to the core Fog node's registry and virtual board, increasing CPU and RAM load.
• Validation: These results validate FogBus's claim of being a lightweight framework, making it suitable for deployment on resource-constrained Fog nodes.

6.1.2. Performance in Managing Resources

Figure 10 from the original paper compares the QoS expectation miss rate of different frameworks.

该图像是一个柱状图，展示了不同框架中 QoS 期望未达到的比例（%）。各个框架包括 Stack4Things、Cloudlet-based PaaS、InDie Fog 和 FogBus，显示 FogBus 的未达到比例最低，说明其在性能上的优势。

Fig. 10. QoS expectaion miss rate in different frameworks.

• Analysis: FogBus achieves the lowest QoS expectation miss rate among the compared frameworks.
• Reasons for FogBus's Advantage:
  • FogBus is designed to harness both edge (Fog nodes) and remote resources (Cloud datacenters) simultaneously. When Fog nodes become overloaded, FogBus seamlessly offloads processing to Cloud resources. This provides a broader scope to meet QoS expectations.
  • In contrast, Indie Fog's application execution is primarily managed by Fog node-based resources, limiting its ability to handle sudden surges in load.
  • Cloudlet-based PaaS suffers from high CPU and RAM load on its core Fog node, which slows down management operations and increases its QoS expectation miss rate.
  • Stack4Things also shows a slightly higher QoS expectation miss rate due to managing application services through a core Fog node deployed in a Cloudlet, which can become a bottleneck.
• Validation: These findings demonstrate FogBus's efficiency in resource management and its ability to maintain high QoS even under varying workloads by intelligently utilizing both Fog and Cloud resources.

6.1.3. Responsiveness of Frameworks

Figure 11 from the original paper presents the system initiation time and data retrieval time for the frameworks.

该图像是图表，展示了不同框架的系统启动时间和数据检索时间。图(a)中，FogBus的系统启动时间显著低于其他框架；图(b)中，FogBus的数据检索时间也是最低的，显示出其高效性。

Fig. 11. (a) Initiation and (b) Data retrieval time of different frameworks.

• Analysis: FogBus exhibits superior responsiveness in both system initiation and data retrieval.
  • System Initiation Time (Figure 11a): FogBus has a significantly shorter system initiation time compared to others.
  • Data Retrieval Time (Figure 11b): FogBus also demonstrates the fastest data retrieval time.
• Reasons for FogBus's Advantage:
  • System Initiation: FogBus benefits from fewer third-party software dependencies, on-demand Cloud interaction, and a design that favors horizontal-level connections over vertical-level ones, simplifying the startup process.
  • Data Retrieval: FogBus stores data in local repository nodes in a distributed manner, rather than relying on Cloud or centralized data storages. This proximity and distributed storage enable faster access during application execution.
• Validation: These results confirm FogBus's responsiveness, which is critical for latency-sensitive IoT applications where quick deployment and data access are paramount.

6.2. Application Deployment Evaluation

This section evaluates the impact of FogBus settings (Interval, Blockchain, Computing Infrastructure) on service latency, energy consumption, and network usage when deploying the Sleep Apnea analysis application. The master sequentially generates processing requests for recorded data chunks.

6.2.1. Number of Requests

Figure 12 from the original paper depicts the number of requests generated in FogBus under different experiment settings.

该图像是图表，展示了在有区块链 (a) 和无区块链 (b) 情况下的请求数量。图中以不同颜色代表三种计算模式：仅雾计算、仅云计算和集成的雾云模式。

Fig. 12. Number of requests (a) With and (b) Without Blockchain.

• Analysis:
  • Impact of Infrastructure: The Fog Only setting consistently handles a higher number of requests compared to Cloud Only and Integrated Fog-Cloud. This is because Fog infrastructure delivers outcomes faster due to proximity.
  • Impact of Interval: In the Without Interval setting, the number of requests generated is significantly higher than With Interval, as the master continuously generates requests without idle time.
  • Impact of Blockchain: Disabling the Blockchain feature leads to a comparatively higher number of requests being generated. This is attributed to a lower amount of additional security data being shared and processed, which improves the speed of receiving outcomes for previous requests.
• Insight: For scenarios requiring a high volume of requests with less stringent security, FogBus can be configured for Fog Only operation with Blockchain disabled. However, this increases management and processing overhead. Tuning the interval between request creation can help manage this overhead.

6.2.2. Service Latency

Figure 13 from the original paper presents the impact of different FogBus settings on service latency.

该图像是图表，展示了在有无区间情况下，基于区块链的服务延迟比较。图表中红色、蓝色和橙色条形分别代表仅使用雾计算、仅使用云计算和集成雾云计算的延迟。结果表明集成框架在不同情况下的延迟变化。

Fig. 13. Service latency (a) With and (b) Without Blockchain.

• Analysis: Service latency is defined as the sum of network propagation delay and task completion/application execution time.
  • Impact of Infrastructure: Fog Only setting shows minimal service delivery latency compared to Cloud Only and Integrated Fog-Cloud. This is primarily due to the significantly lower network propagation delay of the Fog infrastructure (closer to data source), as the data chunk size for this experiment is not huge, making task completion time similar across Fog and Cloud.
  • Impact of Blockchain: Disabling Blockchain further reduces latency, as its management overhead (hashing, proof-of-work, distribution) adds processing time.
  • Impact of Interval: The With Interval setting also contributes to improved service delivery latency by reducing overhead on the infrastructure and network.
• Insight: For latency-sensitive applications with strict deadlines, FogBus can be configured for Fog Only operation, potentially without Blockchain (if security requirements allow), and with a suitable interval to optimize responsiveness.

6.2.3. Network Usage

Figure 14 from the original paper illustrates network usage in different FogBus settings.

该图像是图表，展示了在使用区块链情况下，(a) 有间隔和 (b) 无间隔的网络使用情况。颜色分别代表 Fog、Cloud 和集成的 Fog-Cloud，显示了在不同配置下的网络带宽使用差异。

Fig. 14. Network usage (a) With and (b) Without Blockchain.

• Analysis:
  • Impact of Infrastructure: Fog Only setting results in improved network usage compared to Cloud Only and Integrated Fog-Cloud. This is because it primarily utilizes local networking resources (WLAN) rather than relying on internet-based Cloud connections.
  • Impact of Blockchain: Disabling Blockchain reduces network usage as less security-related data (hashes, signatures, block details) needs to be transferred across infrastructures.
  • Impact of Interval: Continuous request generation (Without Interval) leads to elevated network usage due to constant data and information exchange. Tuning the interval can mitigate this.
• Insight: When network resources are constrained, FogBus can be configured for Fog Only operation, with Blockchain disabled, and with a tuned request generation interval to minimize network usage.

6.2.4. Energy Consumption

Figure 15 from the original paper presents how different FogBus settings influence energy consumption of the infrastructure.

该图像是图表，显示了在区块链影响下的能量消耗，分为 (a) 含区块链条件和 (b) 不含区块链条件的比较。图中展示了不同计算模式（仅Fog、仅Cloud、集成Fog-Cloud）在有无时间间隔情况下的能耗，以焦耳为单位。

Fig. 15. Energy consumption (a) With and (b) Without Blockchain,

• Analysis:
  • Impact of Infrastructure: Fog Only setting requires less energy to conduct operations compared to Cloud Only and Integrated Fog-Cloud. This is because Cloud VMs consume significantly more energy than Fog nodes (e.g., Raspberry Pi). In Fog Only, Fog nodes handle both networking and computation.
  • Impact of Blockchain: Managing the Blockchain feature devours additional energy due to the computational overhead of hashing, mining, and verification. Therefore, disabling Blockchain saves energy.
  • Impact of Interval: An interval between subsequent request creation helps to improve energy usage by allowing idle time for the infrastructure, where energy consumption is lower than during busy periods.
• Insight: For energy-constrained scenarios, FogBus can be configured for Fog Only operation, without Blockchain, and with an optimized interval time to process requests efficiently while minimizing energy consumption. This comes with a trade-off in the number of requests processed, which needs careful tuning.

6.3. Ablation Studies / Parameter Analysis

The "Application deployment evaluation" section acts as a parameter analysis, demonstrating the impact of different FogBus settings on system performance. While not a classic ablation study (removing components to test their contribution), it systematically varies key operational parameters:

• With / Without Interval: Shows how batching or continuous processing affects number of requests, latency, network usage, and energy. The results indicate that With Interval generally reduces overhead and improves latency and energy efficiency, while Without Interval maximizes request throughput at the cost of higher resource consumption.

• With / Without Blockchain: Highlights the trade-off between security/integrity and performance. Enabling Blockchain adds to latency, network usage, and energy consumption, but ensures data integrity. Disabling it improves these performance metrics.

• Fog / Cloud Only / Integrated: Demonstrates the flexibility and performance advantages of selecting the appropriate computing infrastructure. Fog Only excels in latency, network usage, and energy consumption for the given data chunk size, while Cloud Only offers higher computational power for larger tasks (though not explicitly shown as advantageous for this specific small task size) and Integrated provides resilience and scalability.

  These analyses effectively illustrate that FogBus is not a one-size-fits-all solution but a customizable framework whose settings can be tuned to meet diverse application requirements and resource constraints.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper introduces FogBus, a novel, lightweight, and Blockchain-based framework designed for the seamless integration of IoT-enabled systems with Fog and Cloud infrastructures. The framework addresses critical limitations of existing solutions, particularly in platform independence, security, resource management, and multi-application execution. FogBus achieves platform independence by implementing its System Services using cross-platform programming languages (PHP for Service Interface and Java for Management Activity) and extensible application layer protocols (HTTP), thereby overcoming OS and P2P communication-level heterogeneity. Functioning as a Platform-as-a-Service (PaaS) model, FogBus empowers developers to build diverse IoT applications, allows users to customize services, and enables service providers to efficiently manage resources.

A key innovation is the integration of Blockchain for data integrity and authentication for data privacy, complemented by encryption techniques for secure data transfer over less secure networks. The effectiveness of FogBus is validated through a cost-efficient prototype for Sleep Apnea analysis, demonstrating its practical applicability in a latency-sensitive healthcare domain. Experimental evaluations reveal that FogBus is notably lightweight in terms of CPU and RAM usage, and highly responsive with faster system initiation and data retrieval times compared to other existing frameworks. Furthermore, the paper demonstrates how different FogBus settings can be tuned to optimize service latency, network usage, and energy consumption according to specific situational demands, highlighting its adaptability and efficiency in managing both edge and remote resources.

7.2. Limitations & Future Work

The authors acknowledge several limitations of the current FogBus framework and propose future research directions:

• Resource Management Policies: While FogBus offers flexibility for customized provisioning policies, there's a need to develop more sophisticated dynamic resource management policies. These policies should specifically target load balancing across computing infrastructures and QoS enhancement.
• Fog Infrastructure Virtualization: Although Cloud computing is virtualized, in-depth exploration is required to effectively virtualize the Fog infrastructure within FogBus.
• Artificial Intelligence (AI) Integration: The current version of FogBus lacks AI techniques for controlling operations in different infrastructures and improving system resilience. Integrating AI could be a significant contribution.
• Application Placement Techniques: For distributed application execution, efficient application placement techniques are crucial to optimize service latency, meet user expectations, and minimize deployment cost. Such techniques can be added to the FogBus software stack. Future work could also include comparative performance studies of FogBus with various compute-intensive, network-intensive, and storage-intensive applications.
• Runtime Application Migration: Developing different runtime application migration strategies for FogBus is essential to handle anomalies or predicted failures, ensuring continuous service.
• Lightweight Security Features: The existing Blockchain and other security features in FogBus can be computationally intensive, potentially affecting service delivery latency, energy, and network usage. Research into lightweight but effective security features is needed.
• Improvement of Embedded Blockchain Feature: The current Blockchain feature is generic. Future work can focus on enhancing it to reduce data retrieval time, manage smart contracts, and implement chain of chains over integrated Fog-Cloud environments.

7.3. Personal Insights & Critique

FogBus presents a well-thought-out and comprehensive framework that addresses several critical challenges in the evolving IoT-Fog-Cloud landscape. Its explicit focus on platform independence, lightweight design, and decentralized management is highly relevant given the inherent heterogeneity and resource constraints of Fog environments. The integration of Blockchain for data integrity is a significant strength, positioning FogBus as a secure solution for handling sensitive IoT data, particularly in domains like healthcare. The detailed breakdown of hardware instruments, software components, and network structure provides a clear roadmap for implementation and understanding. The real-world case study of Sleep Apnea analysis effectively demonstrates its practical utility and performance advantages.

However, some aspects could be further elaborated or improved:

• Blockchain Specifics: While the paper highlights Blockchain integration, specific details on the consensus mechanism used (e.g., Proof of Authority, Proof of Stake, or a simpler custom mechanism for Fog environments) are not provided. The computational overhead of Proof-of-Work for nonce mining, even if simplified, can still be substantial for resource-constrained Fog nodes. More insight into how FogBus minimizes this overhead or selects an appropriate consensus model for different Fog node capabilities would be valuable. The "chain of chains" concept mentioned in future work is intriguing and could mitigate some scalability issues.

• Dynamic Resource Management Depth: The paper mentions dynamic resource provisioning, application execution migration, and intermediate data storage as actions that Resource Manager can initiate but are currently kept in "abstract form" for user customization. While flexibility is good, concrete examples or a reference implementation of even a basic dynamic policy would significantly strengthen the framework's completeness and provide a clearer benchmark for its potential. This is a critical area for QoS in dynamic environments.

• Generalizability of Performance Benchmarks: The performance evaluation used a specific Sleep Apnea application with small data chunks. While this is valid for latency-sensitive scenarios, future studies could broaden the scope to include compute-intensive (e.g., video analytics), network-intensive (e.g., large-scale data ingestion), and storage-intensive (e.g., long-term archival) applications. This would provide a more holistic view of FogBus's performance across diverse IoT workloads.

• Scalability for Very Large-Scale Deployments: While FogBus addresses scalability through multiple masters and shared workers, the paper doesn't deeply explore the implications for extremely large-scale deployments (e.g., hundreds or thousands of Fog nodes across vast geographical areas). Issues like WAN communication between geographically dispersed WLANs, inter-master communication overhead, and robust global synchronization mechanisms would become more prominent.

• AI Integration for Optimization: The suggestion to integrate AI is promising. AI/ML could be used for predictive resource allocation, anomaly detection, intelligent load balancing, and even optimizing security protocols based on real-time threat landscapes, making FogBus more autonomous and resilient.

  Overall, FogBus offers a significant step forward in building secure and efficient IoT-Fog-Cloud integrated systems. Its emphasis on platform independence and the novel inclusion of Blockchain are particularly noteworthy. The paper provides a solid foundation for future research to refine its dynamic capabilities and explore its performance across a wider spectrum of IoT applications and deployment scales.