AI Data Centers, Part 2: Energy
A Primer on Energy Markets, Technology, History, and Some Thoughts on the Path Forward
Among the many bottlenecks for AI data centers, energy might be the most important and the most difficult to address. IF estimates of data center energy consumption turn out to be true (or even in the vicinity of truth), our current energy infrastructure will not be able to support those demands.
Before the AI boom, data center power consumption was expected to grow consistently. Compute demands would continue to grow; data center centers would grow to meet that demand.
However, with the addition of AI and its power-hungry architectures, estimates are up and to the right!
Unfortunately, there’s no easy way to increase energy capacity quickly. Data centers have two options: on-grid energy and off-grid energy. On-grid energy passes through the electric grid and is doled out by utilities. Off-grid energy (or “behind the meter”) bypasses the electric grid, such as on-site solar, wind, and batteries. Or even better, building a GW data center next to a 2.5 GW nuclear power plant!
The problem with on-grid energy is the time it takes to expand grid capacity; we can see transmission wait times below from request to commercial operations. (This refers to the time from an energy source applying for transmission capacity to actually being in use).
So, inevitably, data centers must resort to off-grid energy. Off-grid power isn’t a great solution either; it’s less reliable than grid power and requires more work for the end user to manage.
Without one clear option to address the energy demand, I think it will take a combination of the following factors to meet the necessary energy demand:
Off-Grid Power Generation
Nuclear
Long-Duration Energy Storage (Batteries)
Building Where the Power’s At
Grid Optimization Tech
These are just a few solutions amongst others like policy reform. This will be a deep dive into energy markets, data center energy demands, and potential opportunities to address those demands.
1. The Electricity Value Chain - How We Get Power
Consider this section a primer on the energy industry; if you’re familiar with this process already, feel free to skip. [For those new to energy markets, bear with me. This market isn’t for the faint of heart.]
The energy supply chain can broadly be broken down into:
Sources - Fossil fuels, renewables, and nuclear provide raw materials that can turn into energy.
Generation - Power plants turn fossil fuels or nuclear material into electricity; for renewables, this happens much closer to the source.
Transmission - Electricity is then transmitted via high-voltage lines close to the destination.
Utilities/Distribution - Utilities manage the distribution of energy to end users.
“The Grid” refers to the infrastructure that transfers energy from power generation to the end user. This is made up of transmission lines, interconnects, substations, and power lines.
Historically, vertically integrated utility companies owned each step of this process. However, in recent decades, this has become a more fragmented process.
While utilities still manage power distribution to end users (i.e., consumers pay their power bill to utilities), they aren’t the only power generators. About 60% of energy is generated by utility companies; independent power producers generate the other 40%. This energy is then sold through “energy markets” based on supply and demand:
That energy is transmitted through high-voltage transmission lines managed by “Independent System Operators” or ISOs (also referred to as RTOs).
An example of this is PJM Interconnect, which manages the transmission of energy in the Northern Virginia region:
PJM's transmission system, or grid, connects generation sources to consumers across 13 states and the District of Columbia. PJM manages the grid's reliability, dispatches generation, and coordinates the movement of wholesale electricity. PJM's transmission system helps ensure consumers have access to reliable electricity at a low cost.
About ⅔ of the US electric grid is managed by ISOs, while ⅓ is managed by utilities. A map of transmission systems is shown here:
These ISOs manage transmission but do not own it; they are non-profit organizations. One of the challenges with expanding the grid is aligning incentives (and costs) for expanding the grid.
Summarizing the energy value chain: energy is generated from some natural resource and transmitted and distributed to end users. Simple enough, right?
2. How We Got Here - A History of the Electric Grid
[Much of the information from this section came from an exceptional series of articles on the electric grid from Construction Physics.]
To understand the complexity of energy markets and the problems we face today, we have to take a step back and look at how we got here.
The parallels between the early development of the electric grid 120 years ago and today’s AI data center buildout are uncanny. Over the initial buildout of the electric grid, we saw three trends very similar to what we’re seeing today:
The scaling of power plants: From 1890-1920, electric companies built the largest power plants they could. Relative performance increased as power plants grew, so electric companies supersized their power plants, much like the mega data centers of today.
“Astronomical” CapEx investments: In 1900, it was estimated that the electric industry would need $2B (or $62B in today’s dollars) over the coming five years to meet demand. As Construction Physics describes, “As one financier observed at the time, the amount of money required by the burgeoning US electrical system was “bewildering” and “sounded more like astronomical mathematics than totals of round, hard-earned dollars.”
The plummeting cost of electricity: Much like the cost of compute today, electricity prices dropped precipitously with scale. These price decreases led to the conclusion that electric utilities were “natural monopolies,” meaning that electricity was a race to the bottom and that the largest providers with scale would dominate the market. The US government decided to regulate these companies and grant them local monopoly statuses.
The (Abridged) Early History of the Electric Grid
The grid was to electricity what the data center buildout is to AI today. From 1890 to 1900, the number of power plants increased by a factor of 9. In 1882, Thomas Edison invented the first incandescent lightbulb and opened his first power plant in NYC. In 1892, he merged with one of his largest competitors to form General Electric (GE). In 1896, the world’s first large-scale power plant opened in Niagara Falls, with 11.1 MW of electricity. The basic elements of those electric systems still form the basis of how our electric grid works today.
Hydroelectric power was the most successful early form of power generation. By 1912, 225 hydroelectric plants had opened in the US, and most large-scale power plants were hydroelectric. This brought about two problems which we still deal with today:
Since electricity couldn’t be efficiently stored, it needed to be consumed immediately, which meant energy would be measured based on peak capacity.
The inconsistency of renewable power made planning the grid difficult. Hydroelectric plants could generate up to 300x more power when rivers flowed stronger than at their lows. The grid must be built for peak capacity, which can make it inefficient in low times.
With the rise of wind and solar power, we’ve seen these two problems rise again over the last decade. The solution in the 1910s was to scale out the electric grid as much as possible; the more area the power plants covered, the more fluctuations could be mitigated.
The network effects were in motion!
The Consolidation of the Grid
In the early days of the grid, many small, local utilities powered communities with their grid. As the benefits of scale became apparent, they began to connect to other utilities. Over time, these utilities continued to consolidate through mergers and acquisitions into massive companies. For example, PG&E in California consolidated more than twenty utilities to form the giant it is today.
This didn’t stop locally. It scaled nationally to further smooth out peaks and valleys of supply and demand, leading to the rise of high-voltage transmission lines.
This consolidation came with its own problems, but the grid is mostly in this structure today: the Eastern Interconnection, the Western Interconnection, and the Texas Interconnection, each of which can share power over long-distance transmission lines.
Industry Maturation
By the 1970s, most of our electric grid had been built, and the focus was building appliances. As US households got access to electricity, new markets opened up for applications of that electricity:
Into the 70s and 80s, several factors (environmental movements and slowing performance improvements) led to the slowing of electrical infrastructure buildout. Experts and analysts saw the value of increased competition in energy markets, which led to deregulation in the 1980s and 1990s.
The industry gets rather complicated at this point.
Before this, utilities were vertically integrated with power plants, transmission, and distribution.
Independent power producers (IPPs) could now build power plants and sell power to the utilities. Additionally, Regional Transmission Organizations (RTOs) were invented; these were middlemen for managing transmission lines. This was so utilities couldn’t prefer their power supply over the new IPPs.
Because of this, utilities are still primarily responsible for building out the electric grid, but the ISOs/RTOs will manage it.
This industry complexity, as well as the government’s involvement at varying degrees, is what makes new electrical infrastructure so hard to build.
Aligning incentives for everyone, gaining the necessary approvals, and then physically building the infrastructure takes years. Most industry participants are aligned on modernizing our infrastructure, but the hard part is executing on that future.
3. Overview of Energy Markets Today
In 2023, the US energy breakdown was 43% natural gas, 19% nuclear, 16% coal, 10% wind, 6% hydro, and 4% solar.
The problem with increasing the amount of renewable energy isn’t generation but transmission. The queue for new energy sources to connect to the grid passed 2600GW in 2023, with 95% of that being solar, battery storage, and wind! For reference, the total US capacity in 2023 was 1280 GW.
To reiterate: the queue for new energy sources is twice that of current generation capacity.
Now, inevitably, some of this is developers submitting requests hoping to be approved; regardless, the numbers are staggering.
So what are Data Centers using for Energy?
The short answer is anything they can get their hands on!
Northern Virginia, the largest data center market in the world, has only .2% energy availability. PJM, the interconnect operator in that region, saw its energy auction prices 10x year/year.
Data center operators' first option will be to purchase energy from a utility that generates and distributes energy. We can see some of the largest utility companies here (note: most but not all are utilities).
If utility power isn’t available, they have to resort to other options, typically off-grid energy. For example, AWS is building a $11B data center campus in Indiana, with four solar fields and a wind farm to power it.
4. Nuclear & Batteries: Two of Many Paths Forward
There are two areas I’m particularly excited about to help address our energy problems: nuclear & batteries.
We’ve recently seen a rise in interest in nuclear power. See just a few examples of this here:
Amazon invested in a $500M round into nuclear startup X-energy while supporting the construction of 320MW worth of nuclear generation.
Google entered an agreement for 500MW of nuclear generation from startup Kairos Power.
Amazon bought a nuclear-powered data center for $650M, intending to expand energy consumption from 300MW to 960MW over the coming years.
On its last earnings call, Oracle said they are building a data center powered by nuclear SMRs.
On top of this, Microsoft’s agreement to restart the reactor at Three Mile Island added real fuel to the fire. It is rumored that they have signed a power purchase agreement (PPA) for approximately $800M/year for 20 years.
Over the long term, nuclear is the area I’m most excited about in energy innovation.
The other is long-duration batteries. We’ve seen a huge investment in wind and solar over the last decade. The problem is that they’re inconsistent. Solar panels run at 20-25% efficiency, and wind turbines at 20-40% efficiency.
Since the grid has to be built for maximum capacity, we need to significantly overbuild the grid to account for the fluctuations in energy generation. Long-duration batteries provide a solution for flattening these fluctuations, storing energy at peak generation, and deploying it in valleys.
On the bright side, we continue to see the cost of batteries come down. Unfortunately, that cost curve has flattened, again opening the opportunity for innovation.
The more the costs of batteries come down, the more affordable and reliable renewable power becomes.
5. What Comes Next?
No one solution is going to solve the energy challenge we’re facing. For the data center-specific problem, consider me highly optimistic in our ability to solve this problem.
The capital from the hyperscalers and private markets and the urgency of solving this problem (i.e., missing out on a transformational shift) have created the right environment to move quickly.
In my opinion, that innovation looks like this:
Off-Grid Power Generation: Hyperscalers are willing to invest in solar, wind, and nuclear energy to power their data centers. They’ll get priority on this energy generation and discharge extra capacity to the grid.
Long-duration energy storage: Long-term battery storage provides the best opportunity to reduce the variability of renewable power.
Nuclear: Over a 15-year time horizon, large nuclear power plants could help address these issues. However, in the near term, this is unlikely to address the energy crunch (the most recent plant took 11 years and over $30B to build). Ergo, the rise of new, smaller reactors from companies like NuScale, Oklo, and Radiant.
Building where the Power’s at: Data center developers must build data centers based on power availability. Hyperscalers will then need to migrate non-latency-sensitive workloads to remote data centers. Crusoe’s approach of ‘Bringing Computing to Where the Energy Happens’ has positioned them as well as any computing startup in the world.
Grid Optimization: Finding ways to optimize our current grid will be critical to maximizing power efficiency; this includes advanced transmission technologies and power optimization software.
Outside of the data center, there’s a broader problem we’ll need to address.
Structurally, we’ll need to find faster and cheaper ways to build infrastructure. This will be through faster approval processes, earned learnings, and lots of investment.
Feel free to reach out if you’re working on a solution in any of these areas. I’d love to hear your thoughts.
As always, thanks for reading!
Disclaimer: The information contained in this article is not investment advice and should not be used as such. Investors should do their own due diligence before investing in any securities discussed in this article. While I strive for accuracy, I can’t guarantee the accuracy or reliability of this information. This article is based on my opinions and should be considered as such, not a point of fact. Views expressed in posts and other content linked on this website or posted to social media and other platforms are my own and are not the views of Felicis Ventures Management Company, LLC.
"Historically, vertically integrated utility companies owned each step of this process. However, in recent decades, this has become a more fragmented process. "- any thoughts on why this is the case. Would this be a trend across industries?
DC close to power source is great for training, but when inference is the game may start to see latency issues when you want to be closer to the end users.
Software optimizations to reduce reliance on grid
Great presentation here as well:
https://www.youtube.com/watch?v=b6uCUuQwEog