1. Bibliographic Information

• Title: Quantifying the Carbon Emissions of Machine Learning
• Authors:
  • Alexandre Lacoste (Element AI)
  • Alexandra Luccioni (Mila, Université de Montréal)
  • Victor Schmidt (Mila, Université de Montréal)
  • Thomas Dandres (Polytechnique Montréal, CIRAIG) The authors are affiliated with prominent AI research institutions like Mila and Element AI, as well as CIRAIG, an expert center in life cycle assessment, indicating a blend of expertise in both machine learning and environmental science.
• Journal/Conference: This paper is a preprint available on arXiv. An arXiv preprint is a version of a scholarly article that precedes formal peer review and publication in a journal or conference. It allows for rapid dissemination of research ideas.
• Publication Year: 2019
• Abstract: The paper identifies key factors that determine the carbon emissions from training a neural network: the server's geographical location (and its local energy grid), the duration of training, and the specific hardware used. To help researchers estimate these emissions, the authors introduce the Machine Learning Emissions Calculator. The paper complements this tool by explaining the influencing factors and providing concrete actions that individuals and organizations can take to reduce their environmental impact.
• Original Source Link:
  • arXiv Link: https://arxiv.org/abs/1910.09700
  • PDF Link: http://arxiv.org/pdf/1910.09700v2

2. Executive Summary

• Background & Motivation (Why):

  • Core Problem: The training of modern machine learning models, especially large neural networks, is becoming increasingly computationally intensive. This requires vast amounts of energy, which in turn generates significant carbon emissions. However, most ML practitioners are unaware of the environmental footprint of their work.
  • Importance & Gaps: Prior studies had begun to highlight the substantial environmental cost of training massive, state-of-the-art models, but there was a lack of accessible tools for the average researcher to quantify the emissions of their own, more typical projects. This paper aims to fill that gap by moving the conversation from merely identifying the problem to providing a practical solution for estimation and mitigation.
  • Fresh Angle: Instead of focusing only on the worst-case scenarios of training giant models, this work provides a tool and a set of actionable recommendations for the entire ML community. It empowers individual practitioners to make more environmentally conscious decisions in their daily work.

• Main Contributions / Findings (What):

  • ML Emissions Calculator: The primary contribution is a publicly available online tool (https://mlco2.github.io/impact/) that allows users to approximate the carbon dioxide equivalent emissions produced by training an ML model. The calculation is based on the hardware used, training duration, and the carbon intensity of the energy grid at the server's location.
  • Actionable Recommendations: The paper outlines several concrete best practices for reducing carbon emissions, categorized into:
    1. Choosing Greener Infrastructure: Selecting cloud providers with strong sustainability commitments and data centers in regions powered by renewable energy.
    2. Optimizing Model Development: Using more efficient hyperparameter search methods (e.g., random search over grid search), leveraging pre-trained models (fine-tuning), and reducing redundant or uninformative experiments.
    3. Selecting Efficient Hardware: Using hardware with better performance per watt, such as newer GPUs or specialized accelerators like TPUs.
  • Raising Awareness: The paper serves an educational purpose by clearly explaining the key factors that contribute to the carbon footprint of ML and compiling the necessary data to make these factors transparent.

3. Prerequisite Knowledge & Related Work

• Foundational Concepts:

  • Neural Network: A type of machine learning model inspired by the human brain, composed of interconnected nodes or "neurons." Training these models, especially deep ones with many layers, requires immense computational power.
  • GPU (Graphical Processing Unit): Specialized electronic circuits designed for parallel processing, making them highly effective for the matrix and vector operations that are fundamental to training neural networks. Their high performance comes at the cost of significant energy consumption.
  • CO₂eq (Carbon Dioxide Equivalent): A standard unit for measuring carbon footprints. It converts the impact of different greenhouse gases into the equivalent amount of carbon dioxide (CO₂) that would have the same global warming potential. This allows for a single, comparable metric for emissions.
  • Carbon Intensity: A measure of how much CO₂eq is emitted per unit of energy consumed, typically expressed in grams of CO₂eq per kilowatt-hour (gCO₂eq/kWh). This value varies dramatically depending on the energy source (e.g., coal is high intensity, hydro is low intensity).
  • PUE (Power Usage Effectiveness): A ratio that measures the energy efficiency of a data center. It is calculated by dividing the total facility energy by the IT equipment energy. A PUE of 1.0 is the ideal, meaning 100% of the energy is used by the computing hardware itself. Modern data centers, like Google's, have PUEs around 1.1, meaning only 10% of energy is used for overhead like cooling.
  • REC (Renewable Energy Certificate): A tradable, non-tangible energy commodity. One REC represents proof that 1 megawatt-hour (MWh) of electricity was generated from a renewable energy resource. Companies purchase RECs to offset their emissions and claim carbon neutrality.
  • TDP (Thermal Design Power): The maximum amount of heat generated by a computer chip or component (like a GPU or CPU) that the cooling system is designed to dissipate. It is often used as a proxy for the component's maximum power consumption, measured in watts.

• Previous Works:

  • The paper builds upon recent work that first sounded the alarm on the environmental impact of AI. Specifically, it cites:
    • Strubell et al. (2019), "Energy and policy considerations for deep learning in NLP": This influential paper quantified the massive carbon footprint of training large-scale Natural Language Processing (NLP) models, showing that a single model training run could emit as much carbon as several transatlantic flights. This work highlighted the severity of the problem.
    • Schwartz et al. (2019), "Green AI": This paper argued for a shift in the field towards "Green AI," where efficiency is considered a primary evaluation metric alongside accuracy. It distinguished between "Red AI" (which achieves state-of-the-art results by throwing more compute at a problem) and "Green AI" (which focuses on computational efficiency).
  • This paper differentiates itself by moving from quantifying the extreme cases (Strubell et al.) and proposing a conceptual shift (Schwartz et al.) to providing a practical tool and a guide for all ML practitioners.

• Technological Evolution: The paper is situated in a context where ML models are rapidly growing in size and complexity (BERT, VGG, etc.). This "arms race" for state-of-the-art performance has led to training procedures that run for weeks or months on multiple high-power GPUs, exponentially increasing energy consumption and, consequently, carbon emissions.

• Differentiation: While previous works focused on diagnosing the problem, this paper focuses on empowering the solution. Its main innovation is not a new algorithm but a synthesis of public data into an accessible calculator, coupled with a clear, practical guide for mitigation. It shifts the responsibility from a few researchers at top labs to the entire community.

4. Methodology (Core Technology & Implementation)

The core methodology of the paper is the creation and justification of the ML Emissions Calculator. The process can be broken down into data collection and the calculation itself.

• Principles: The fundamental principle is that the carbon emissions of training an ML model can be approximated by multiplying the total energy consumed by the carbon intensity of the electricity source. $$\text{Total Emissions} \approx \text{Total Energy Consumed} \times \text{Carbon Intensity}$$

• Steps & Procedures: The calculator takes user inputs and combines them with a pre-compiled database to estimate emissions.

  1. Input Collection: The user provides three key pieces of information:
     • Hardware: The type and number of GPUs (or other processors) used.
     • Training Time: The duration of the training process in hours or days.
     • Location/Provider: The cloud provider (e.g., AWS, GCP, Azure) and the specific server region (e.g., us-east-1, europe-west1).
  2. Data Lookup: The tool uses this input to look up values from its database:
     • Hardware Power: The power consumption of the selected hardware in kilowatts (kW), estimated from its TDP.
     • Carbon Intensity: The $gCO₂eq/kWh$ for the grid corresponding to the selected server location.
     • Provider Offsets: Information on whether the cloud provider is carbon neutral or purchases RECs, which can be factored in to adjust the final emissions number.
  3. Calculation: The approximate emissions are calculated as follows: $$\text{CO}_2\text{eq (kg)} = (\text{Power Consumption (kW)} \times \text{Training Time (hours)}) \times \text{Carbon Intensity (gCO}_2\text{eq/kWh)} \times \frac{1 \text{ kg}}{1000 \text{ g}}$$ The calculation also considers the Power Usage Effectiveness (PUE) of the data center, which accounts for energy used for cooling and other infrastructure. A more complete formula would be: $$\text{Energy Consumed (kWh)} = \text{Number of GPUs} \times \text{Power per GPU (kW)} \times \text{Training Time (h)} \times \text{PUE}$$ $$\text{Total Emissions (kg CO}_2\text{eq)} = \text{Energy Consumed (kWh)} \times \text{Carbon Intensity (kg CO}_2\text{eq/kWh)}$$

• Data Sources: The authors compiled their database from publicly available sources:

  • Carbon Intensity Data: From organizations like Ecometrica [5] and other academic studies [4], providing $gCO₂eq/kWh$ for various electrical grids worldwide.
  • Cloud Data Center Locations: Cross-referenced from the official documentation of Google Cloud, AWS, and Azure.
  • Hardware Power Consumption: Estimated from the TDP provided in manufacturers' specifications.
  • Provider Sustainability Claims: From the environmental reports of Google [13], Microsoft [14], and Amazon [15]. The authors emphasize transparency by making their data and sources available on GitHub, allowing for community contributions and updates.

5. Experimental Setup

This paper does not follow a traditional experimental structure. Instead, its "experiments" consist of data analysis and presenting illustrative comparisons to support its recommendations.

• Datasets: The data used is not for training models but for populating the calculator. This includes:

  • Grid Carbon Intensity Data: Emission factors for electricity grids across various countries and regions. The paper includes tables in Appendix A showing this data for different cloud provider regions.
  • Hardware Specifications Data: TDP and performance (FLOPS) for various CPUs, GPUs, and TPUs, as shown in Appendix B.

• Evaluation Metrics: The single, central metric is:

  • Conceptual Definition: CO₂eq (Carbon Dioxide Equivalent). It is the standardized measure used to express the global warming potential of all greenhouse gases combined, reported as the equivalent amount of carbon dioxide. It is the universal metric for quantifying a carbon footprint.
  • Mathematical Formula: This is not a performance metric with a formula in the typical sense but a physical quantity. The calculation method is detailed in the Methodology section above.
  • Symbol Explanation: The key components are Power (kW), Time (h), and Carbon Intensity (gCO₂eq/kWh).

• Baselines: The paper does not use "baselines" in the sense of comparing its method to others. Instead, it uses different scenarios as points of comparison to illustrate the impact of choices. For example:

  • Geographic Baseline: Training a model in a high-carbon region (e.g., Iowa, USA, at 736.6 $gCO₂eq/kWh$) versus a low-carbon region (e.g., Quebec, Canada, at 20 $gCO₂eq/kWh$).
  • Hardware Baseline: Comparing the efficiency (FLOPS/Watt) of a CPU versus a GPU versus a TPU.
  • Algorithmic Baseline: Using inefficient grid search for hyperparameter tuning versus more efficient random search.

6. Results & Analysis

The paper's results are presented as key findings derived from the data it collected, demonstrating the magnitude of impact that different choices can have.

• Core Results:

  • Impact of Location: Figure 1 starkly visualizes the massive variance in carbon intensity across the globe. Training in a region powered by fossil fuels can be over 40 times more carbon-intensive than training in a region powered by hydroelectricity. For example, the paper notes the range in North America from 20 $gCO₂eq/kWh$ (Quebec) to 736.6 $gCO₂eq/kWh$ (Iowa). This is the single most impactful choice a researcher can make if they have the flexibility.

    该图像是图表，展示了不同服务器区域的平均碳强度（单位：gCO2/kWh）的变化情况。图中以箱线图形式呈现各区域的碳强度分布，显示欧洲、亚洲和北美的碳排放范围较广，非洲和南美数据点较少且集中。

  • Impact of Hardware: The choice of hardware significantly affects energy efficiency. The paper suggests using FLOPS/Watt as a metric. Based on the data in Appendix B (transcribed below), the paper concludes:

    • CPUs are roughly 10 times less efficient than GPUs.
    • TPUs (v2/v3) can be 4 to 8 times more efficient than contemporary high-end GPUs (like the Tesla V100).
    • Specialized, low-power GPUs (like the Jetson AGX Xavier) can be exceptionally efficient for embedded applications.

  • Impact of Algorithmic Choices: The paper reiterates established findings that random search is more efficient than grid search for hyperparameter optimization [10]. Using more efficient search strategies directly translates to less computation time, thus saving energy and reducing emissions. Similarly, fine-tuning pre-trained models instead of training from scratch is highlighted as a major resource-saving practice.

• Data from Appendices (Manual Transcription):

  Appendix A: Energy Grid Data (gCO₂e/kWh)

  • Google Cloud Platform

    Region	Country	City	Estimated gCO2e/kWh)
    asia-east1	Taiwan	Changhua County	557
    asia-east2	China	Hong Kong	702
    asia-northeast1	Japan	Tokyo	516
    asia-northeast2	Japan	Osaka	516
    asia-south1	India	Mumbai	920
    asia-southeast1	Singapore	Jurong West	419
    australia-southeast1	Australia	Sydney	802
    europe-north1	Finland	Hamina	211
    europe-west1	Belgium	St. Ghislain	267
    europe-west2	United Kingdom	London	623
    europe-west3	Germany	Frankfurt	615
    europe-west4	Netherlands	Eemshaven	569
    europe-west6	Switzerland	Zürich	16
    northamerica-northeast1	Canada	Montréal	20
    southamerica-east1	Brazil	So Paulo	205
    us-central1	USA	Council Bluffs	566.3
    us-east1	USA	Moncks Corner	367.8
    us-east4	USA	Ashburn	367.8
    us-west1	USA	The Dalles	297.6
    us-west2	USA	Los Angeles	240.6

  • Amazon Web Services

    Region	Country	City	gCO2e/kWh
    us-east-2	USA	Columbus	568.2
    us-east-1	USA	Ashburn	367.8
    us-west-1	USA	San Francisco	240.6
    us-west-2	USA	Portland	297.6
    ap-east-1	China	Hong Kong	702
    ap-south-1	India	Mumbai	920
    ap-northeast-3	Japan	Osaka	516
    ap-northeast-2	South Korea	Seoul	517
    ap-southeast-1	Singapore	Singapore	419
    ap-southeast-2	Australia	Sydney	802
    ap-northeast-1	Japan	Tokyo	516
    ca-central-1	Canada	Montreal	20
    cn-north-1	China	Beijing	680
    cn-northwest-1	China	Zhongwei	680
    eu-central-1	Germany	Frankfurt am Main	615
    eu-west-1	Ireland	Dublin	617
    eu-west-2	United Kingdom	London	623
    eu-west-3	France	Paris	105
    eu-north-1	Sweden	Stockholm	47
    sa-east-1	Brazil	Sao Paulo	205
    us-gov-east-1	USA	Dublin	568.2
    us-gov-west-1	USA	Seattle	297.6

  • Microsoft Azure

    Region	Country	City	gCO2e/kWh
    eastasia	Hong Kong	Wan Chai	702
    southeastasia	Singapore	Singapore	419
    centralus	USA	Des Moines	736.6
    eastus	USA	Blue Ridge	367.8
    eastus2	USA	Boydton	367.8
    westus	USA	San Francisco	240.6
    northcentralus	USA	Chicago	568.2
    southcentralus	USA	San Antonio	460.4
    northeurope	Ireland	Dublin	617
    westeurope	Netherlands	Amsterdam	569
    japanwest	Japan	Osaka-shi	516
    japaneast brazilsouth	Japan	Tokyo	516
    australiaeast	Brazil	Sao Paulo	205
    australiasoutheast	Australia	Sydney	802
    southindia	Australia	Melbourne	805
    	India	Pallavaram	920
    centralindia	India	Lohogaon	920
    westindia	India	Mumbai	920
    canadacentral	Canada	Toronto	69.3
    canadaeast	Canada	Quebec	20
    uksouth	United Kingdom	Midhurst	623
    ukwest	United Kingdom	Wallasey	623
    westcentralus	USA	Mountain View	297.6
    westus2	USA	Quincy	297.6
    koreacentral	South Korea	Seoul	517
    koreasouth	South Korea	Busan	517
    francecentral	France	Huriel	105
    francesouth	France	Realmont	105
    australiacentral	Australia	Forrest	900
    australiacentral2	Australia	Forrest	900
    southafricanorth	South Africa	Pretoria	1009
    southafricawest	South Africa	Stellenbosch	1009

  Note: There appear to be minor data entry errors in the original PDF's tables, which have been transcribed as is (e.g., japaneast brazilsouth, and formatting inconsistencies in the southindia row).

  Appendix B: Hardware Efficiency (Manual Transcription of Table 4)

  Name	Watt (TDP)	TFLOPS32	TFLOPS16	GFLOPS32/W	GFLOPS16/W
  RTX 2080 Ti	250	13.45	26.90	53.80	107.60
  RTX 2080	215	10.00	20.00	46.51	93.02
  GTX 1080 Ti	250	11.34	0.17	45.36	0.68
  GTX 1080	180	8.00	0.13	44.44	0.72
  AMD RX480	150	5.80	5.80	38.67	38.67
  Titan V	250	15.00	29.80	59.60	119.20
  Tesla V100	300	15.00	30.00	50.00	100.00
  TPU2	250	22.00	45.00	88.00	180.00
  TPU3	200	45.00	90.00	225.00	450.00
  Intel Xeon E5-2699	145	0.70	0.70	4.83	4.83
  AGX Xavier	30	16.00	32.00	533.33	1066.67

7. Conclusion & Reflections

• Conclusion Summary: The paper successfully makes the case that the carbon footprint of ML is a significant problem that individual practitioners can help mitigate. By creating the ML Emissions Calculator and providing a clear set of actionable recommendations, the authors empower the community to become more aware and environmentally responsible. The key takeaway is that simple choices regarding infrastructure, hardware, and algorithmic efficiency can lead to substantial reductions in carbon emissions.

• Limitations & Future Work: The authors openly acknowledge several limitations of their work:

  • Approximation: The calculator provides an estimate, not a precise measurement. Factors like global load balancing (where cloud providers shift jobs between data centers) and the lack of complete transparency from providers introduce margins of error.
  • Scope: The current tool focuses only on the training phase. The inference phase (deploying a model for real-world use) can also be highly energy-intensive, especially for large-scale services, and is not covered.
  • Data Accuracy: The calculations depend on publicly available data, which may not be perfectly accurate or up-to-date. The authors invite community collaboration to improve the data over time.
  • Broader Context: The paper touches on, but does not solve, the larger philosophical debate about whether the scientific or social value of certain ML research justifies its environmental cost.

• Personal Insights & Critique:

  • Strengths: The paper's primary strength is its pragmatism and impact. It shifted the conversation from an abstract problem to a tangible, personal responsibility. The creation of a simple tool was a brilliant move to engage the broader community beyond academic discourse. It is a landmark paper in the "Green AI" movement for its practical contribution.
  • Critique: While the calculator is an excellent awareness tool, its reliance on TDP as a proxy for power consumption is a known simplification; actual power draw varies significantly depending on the workload. Furthermore, the true carbon intensity of a specific server is complex, as it may draw power from a mix of sources that RECs only account for on paper.
  • Future Impact: This paper has been highly influential in promoting a culture of carbon-aware computing in AI. It has spurred further research into more accurate carbon accounting, encouraged conferences to ask for statements on computational cost, and pushed cloud providers to be more transparent about the carbon footprint of their services. Its ultimate legacy is in making environmental efficiency a key pillar of responsible AI development.