The Current State of AI Markets
The Current State of AI Markets
A Quantitative Look at where Value has accrued in the AI Value Chain
Given the recent increased skepticism on the ROI of AI and the fear of a popping of the “AI Bubble”, I wanted to quantitively break down where revenue has occurred thus far in the AI Value Chain. My goal is to provide a comprehensive picture of revenue and commentary from the primary markets exposed to AI. I’ll be structuring the article as follows:
The AI ROI Debate
A Summary of the Current State of AI Markets
Semiconductor Markets
Data Center Markets
Cloud Markets
AI Applications
This will also act as a primer of sorts on the companies exposed to AI and where value may accrue moving forward.
1. The AI ROI Debate
For the first time in a year and a half, common opinion is now shifting to the narrative “Hyperscaler spending is crazy. AI is a bubble.” Now, I’m not one for following public opinion, but investing is about finding undervalued opportunities; the more pessimism, the better.
We shouldn’t be surprised at the increasing pessimism around AI. The gap between infrastructure buildout and application value was expected. For every large infrastructure buildout, capacity never perfectly meets demand. And that’s okay. There will not be a perfect transition from infrastructure build-out to value creation.
In part, the recent rise in AI pessimism came from the latest hyperscaler capital expenditure numbers.
Amazon, Google, Microsoft, and Meta have spent a combined $177B on capital expenditures over the last four quarters. On average, that grew 59% from Q2 of 2023 to Q2 of 2024. Those numbers are eye-watering, but two things can be true:
There’s not a clear ROI on AI investments right now.
Hyperscalers are making the right CapEx business decisions.
We need to separate the conversation between ROI on AI investments and ROI on hyperscaler AI spending. ROI on AI will ultimately be driven by application value to end users. When a company decides to build out an AI app, it’s based on the potential for revenue or cost replacement.
For the large cloud service providers, there’s a different equation.
~50% of the spend is on the data center “kit” like GPUs, storage, and networking. Thus far, this has been demand-driven spending! All three hyperscalers noted they’re capacity-constrained on AI compute power.
Much of the other 50% is spent on securing natural resources that will continue to be scarce over the next 15 years: real estate and power.
They’ll do this themselves or through a developer like QTS, Vantage, or CyrusOne. They buy real estate for data centers with proximity to compute end usage, power availability, and cheap real estate. They’ve acquired most of the prime locations for data centers already; now, they’re buying locations primarily based on power availability. Power will continue to be a bottleneck for compute demand, and the hyperscalers know it.
The equation for hyperscalers is relatively simple: data centers are at least 15-year investments. They are betting compute demand will be higher in 15 years (not a crazy assumption).
If they do not secure these natural resources, three things could happen:
They will lose business to competitors who have capacity.
They’ll have to secure sub-optimal land with worse price/performance profiles.
Challengers will buy this land and power capacity and attempt to encroach on the hyperscalers.
If they secure this land and don’t need the computing power right away, then they’ll wait to build out the “kit” inside the data center until that demand is ready. They might’ve spent tens of billions of dollars a few years too early, and that’s not ideal. However, the three big cloud providers have a combined revenue run rate of $225B.
As Google’s CEO Sundar said, “The risk of under-investing is higher than the risk of over-investing.”
Thus far, we’ve started to see the buildout of AI infrastructure in the form of GPU purchases and data center construction costs, and we’re at the point where we ask ourselves, “Where has value been created thus far, and where will it be created moving forward?”
2. A Summary of the Current State of AI Markets
Currently, most AI value has accrued to the infrastructure level, specifically Nvidia and the companies exposed to it. We’re starting to see the buildout of “AI Data Centers”, but we’re early on in this trend, and energy is a real bottleneck to this buildout.
The cloud providers are seeing some revenue from model APIs or “GPUs as a Service”, where they buy GPUs and rent them out via the cloud.
We haven’t seen wide-scale application revenue yet. AI applications have generated a very rough estimate of $20B in revenue with multiples higher than that in value creation so far (cost reduction), but this pales in comparison to infrastructure costs.
The industry will ultimately be driven by AI application value (either revenue or cost replacement). That value will be driven by the end value to the consumer or enterprise.
So, we come to this conclusion: the bigger problems that AI applications can solve, the more value will accrue across the value chain.
At this point in time, we’ve seen a massive amount of infrastructure investment with the expectation of application revenue. The question is this: where is the meaningful value creation from AI?
The answer is that it’s unclear. It’s early. And that’s okay.
Before we dive in, I want to map out a theoretical framework for how I think about AI investing (of course, this is simplified).
Broadly, we have semiconductors (and their value chain), data centers (and energy), cloud platforms, models, data, and the application layer.
3. AI & Semiconductor Markets
Summary: Nvidia is taking the lion’s share of AI semiconductor revenue, Nvidia’s supply chain has benefitted to varying extents, and HBM memory & exposure to custom ASICs follow those two categories.
Nvidia, Nvidia’s Supply Chain, and AMD
Last quarter, Nvidia did $26.3B in data center revenue, with $3.7B of that coming from networking. Tying back to the Capex numbers from earlier, 45% of revenue or $11.8B came from the cloud providers.
TSMC, the semiconductor foundry manufacturing Nvidia’s leading-edge chips, is the most direct beneficiary of Nvidia’s AI revenue, commenting:
“We forecast the revenue contribution from several AI processors to more than double this year and account for low-teens percent of our total revenue in 2024. For the next five years, we forecast it to grow at 50% CAGR and increase to higher than 20% of our revenue by 2028. Several AI processors are narrowly defined as GPUs, AI accelerators and CPU's performing, training, and inference functions and do not include the networking edge or on-device AI.”
TSMC’s average 2024 revenue estimate is $86.3B; estimating 12% of that from AI gets us to a projected AI revenue of ~$10.4B.
As TSMC’s revenue grows, eventually, so does their CapEx, generating revenue for the semiconductor capital equipment providers as well.
And what about AMD?
In last quarter’s earnings, Lisa Su said she expected MI300X revenues to surpass $4.5B, up from the previously expected $4B. But, the most important note came in the earnings call prior where she commented:
“From a full year standpoint, our $4 billion number is not supply capped.”
This is not an ideal position to be in when your biggest competitor is selling ~$20B of GPUs and can’t sell enough. With that being said, it takes time to develop chip lines, and going from chip release to $4B+ in revenue in a year is an impressive achievement.
HBM, Broadcom, and Marvell
High-bandwidth memory chips are designed to reduce the memory bottleneck for AI workloads; SK Hynix, Samsung, and Micron are the three manufacturers of these chips.
According to Trendforce, “Overall, SK Hynix currently commands about 50% of the HBM market, having largely split the market with Samsung over the last couple of years. Given that share, and DRAM industry revenue is expected to increase to $84.150 billion in 2024, SK Hynix could earn as much as $8.45 billion on HBM in 2024 if TrendForce's estimates prove accurate.”
Additionally, Marvell and Broadcom provide two primary AI products:
The semiconductors for most networking equipment in data centers (supplying customers like Arista and Cisco).
Codesign partnerships helping companies build custom AI chips (the best example is Broadcom and Google’s partnership for developing TPUs).
Broadcom: “Revenue from our AI products was a record $3.1 billion during the quarter” driven by the TPU partnership with Google and networking semiconductors. Last quarter, they disclosed about ⅔ of that comes from TPUs and ⅓ from networking.
Marvell: “We had set targets of $1.5 billion for this year and $2.5 billion for next year…all I can tell you is both of them have upsized.”
Summarizing the semiconductor section, we see Nvidia at an $105B AI run rate, Broadcom at a $12.4B run rate, AMD with an expected $4.5B in 2024 AI revenue, TSMC with an expected $10.4B in AI revenue, the memory providers earning ~$16B in HBM revenue (SK Hynix with ~46-52% market share, Samsung with ~40-45% market share, Micron with ~4-6% market share), and Marvell likely around ~$2B in AI revenue.
4. AI Data Center Markets
The AI Data Center narrative is interesting. We’re seeing revenue flow to two locations: GPUs/AI Accelerators and data center construction. We haven’t seen revenue flow to traditional data center equipment providers like Arista, Pure Storage, or NetApp.
Data Center Expenditures
The most direct driver of data center construction is hyperscaler capital expenditures. Amazon, Google, Microsoft, and Meta have spent a combined $177B on CapEx over the last four quarters. On average, that grew 59% from Q2 of 2023 to Q2 of 2024.
Approximately 50% of that goes directly to real estate, leasing, and construction costs.
Synergy Research notes that it has taken just four years for the total capacity of all hyperscaler data centers to double, and that capacity will double again in the next four years.
"We're also seeing something of a bifurcation in data center scale. While the core data centers are getting ever bigger, there is also an increasing number or relatively smaller data centers being deployed in order to push infrastructure nearer to customers. Putting it all together though, all major growth trend lines are heading sharply up and to the right."
It’s clear data centers are being built, but who benefits from this revenue?
Theoretically, value should flow through the traditional data center value chain:
The interesting divergence in this data is that we haven’t seen revenue flow to traditional data center networking and storage providers…yet. That means revenue thus far is mostly going to rack-scale implementations and data center construction.
Data center developers like Vantage, QTS, and CyrusOne buy land and power, who then lease that out to big tech companies. The developers will then go through the construction process to build the data center before it’s filled with equipment.
The other primary beneficiaries thus far have been server manufacturers like Dell and SMCI, who deliver racks with Nvidia GPUs. Dell sold $3.2B worth of AI servers last quarter, up 23% Q/Q. SMCI’s revenue last quarter was $5.3B, up 143% Y/Y.
What About Energy?
It’s clear there will be value created for the energy value chain; I’ll be publishing more on this in future articles. Demand is increasing, and the question is what bottlenecks will be alleviated to fulfill that demand (I don’t have a good answer to that question right now).
One anecdote to highlight the ensuing energy demand:
PJM Interconnection, the largest electrical grid operator in the US, had its annual energy auction the week of July 22nd, 2024. PJM services the northern Virginia region, which is the largest data center hub in the world. Their auction ultimately sold $269.93/MW-day for most of their region, up from $28.92/MW-day for their last auction.
Prices approximately 10xed from this year to next. This highlights the energy concern for the hyperscalers, and how creative they’re willing to get to secure capacity.
AI Cloud Markets
I covered this more fully in my quarterly cloud update here. All three hyperscalers want to be vertically integrated AI providers, allowing customers to tap into whichever service level they prefer (infrastructure, platforms, applications). This happens to be the exact playbook they’ve been successfully deploying with traditional cloud software.
Microsoft has undoubtedly executed the best thus far and gives us a good idea of cloud AI revenue. We can do some back-of-the-napkin math to estimate their Azure AI revenue. For each of the last 5 quarters, they’ve disclosed Azure AI revenue growth contribution. Combining that with net new Azure revenue gives us an idea of how much AI revenue they added each quarter:
They hit approximately a $5B Azure AI run rate last quarter.
Google and Amazon both noted they have “billions” in AI revenue run rate but didn’t disclose details past that.
Oracle has signed $17B worth of AI compute contracts (this is RPO, so it will be recognized over a 3-5 year period, depending on the specific contract. If we average that out over 12 quarters, Oracle will be doing around $1.3B in AI revenue a quarter, which puts it in the same vicinity as the other hyperscalers.
Let’s throw in the GPU Cloud data to round out the picture for AI revenue:
Coreweave did an estimated $500M in revenue in 2023, and is contracted for $2B in revenue in 2024. Lambda Labs did an estimated $250M in 2023 revenue. Crusoe is estimated to have done $300M+ in 2023 revenue. Combined these companies will add a few billion more to the AI Cloud pie.
Summarizing this section: we have annual run rates of a very rough estimate of ~$20B thus far for AI cloud services.
Everything discussed above is what I consider “AI infrastructure”, and has been built out to support the eventual coming of AI application value. If I can point out the STEEP drop off in revenue from semiconductors, to data centers, to the cloud. This tells the story of AI revenue so far.
Semiconductors will earn $100-$200B in AI revenue this year, hyperscalers have spent $175B+ in capex over the last four quarters, the cloud providers will do ~$20-25B in AI revenue this year.
The cloud revenue gives us the real indication of how much value is being invested into AI applications; needless to say it’s much less than the infrastructure investments thus far.
AI Models & Application Revenue
AI application value is the most important question in AI markets right now. AI application value will ultimately drive investments across the value chain. All of this infrastructure buildout becomes a bubble without value creation on the backend.
The scale of AI application revenue will be determined by the value it creates for customers. What are the problems it solves? What’s the scale of those problems?
Quite frankly, we don’t have good answers to that right now. Sapphire estimated some of these figures below:
https://view.sapphireventures.com/viewer/b9ae0b14dc721ac3f7a7c82830624354
Data companies like Scale AI (reportedly at $1B in revenue this year) and VAST Data ($200M in ARR last year, much higher this year) show the clear demand for AI apps. However, they still fall into an infrastructure-like category, not directly touching the end user.
Our best indication of AI app revenue comes from model revenue (OpenAI at an estimated $1.5B in API revenue, Anthropic at an estimated 2024 $600M revenue). Freda Duan from Altimeter broke down the projected financials for OpenAI and Anthropic.
https://twitter.com/FredaDuan/status/1816258556552962525
The problem comes with analyzing ROI on this revenue. For LLM-based applications, we could assume an 80% gross margin and say between $5B-$10B in application revenue has been sold. However, what about the value created from labor replacement costs like Klarna saving $40M on customer service costs?
I can confidently say two things:
AI application will not generate a net positive ROI on infrastructure buildout for some time.
AI applications must continue to improve and solve new problems to justify the infrastructure buildout.
On the latter point, it does appear that the future of AI is agentic (LLMs with memory, planning, and tool integrations to execute tasks). This seems to represent the next potential step function of AI application value. In reality, AI value may manifest itself in ways we can’t predict. (How could you predict in 2000 the internet would give rise to staying in other people’s houses, or getting rides in other people’s cars?)
As Doug O’Laughlin pointed out, we’re in the very early stages of AI:
New technologies are hard to figure out, especially in the short term. And that’s okay. Value will be created in unforeseen ways. Over the long run, betting against technology is a bad bet to make; and certainly won’t be one I’m making.
As always, thanks for reading!
Disclaimer: The information contained in this article is not investment advice and should not be used as such. Investors should do their own due diligence before investing in any securities discussed in this article. While I strive for accuracy, I can’t guarantee the accuracy or reliability of this information. This article is based on my opinions and should be considered as such, not a point of fact. Views expressed in posts and other content linked on this website or posted to social media and other platforms are my own and are not the views of Felicis Ventures Management Company, LLC.
By the way, I love your graphics. One of mentors used to do this for various industries in almost an identical fashion, and I ended up copying him. It is always great to start off a pitch and gets everybody engaged.
My only thought is as you are thinking about value-chains, just because something is in a value-chain, doesn't mean that it brings substantial value. We are so overwhelmed by data and companies, that is become very difficult to digest where we should focus. An example that I've used in the past is to say, "we could claim that ballpoint pens are critical because you need them to sign the contract." However, we all recognize that this is not where the value, thus the investment choice should be.
Something I have done, which I would love to see in yours, is to scale any level by some factor of revenue and gross margin. Depending on the situation, look at both or either is help to think what is the true critical path through the value chain.
Finally, a good old Porter 5 force analysis is often insightful through thinking about where to draw attention in your chain, although I like Grove's version better. Buffet just calls this a moat, but you get it through a force analysis.
The ROI of AI is really hard to understand. From what I can see, the people that immediately say "oh, they'll never hit it" are the ones that simply are not using the tech. If you are a software programmer, and you've been using Co-Pilot or Claude, it will strike you that "my job is going to go away if they keep scaling." The problem with all the revolutions before this is that they didn't replace people to lower OpEx. With the virtualization and work from home, it may be possible for widespread rifs and cost savings.
Mind you, I am NOT saying that their is a clear path make up the $1T capex figure (which I agree with), but if you said, there's 2M truck drivers at an average cost of $100,000 full employment cost, I can find you $200B if the government allow me to implement AI today. It is already generally accepted in the Valley that the biggest problem with rolling AI into driving is the government, not the tech. (The tech isn't perfect, but it would have a lower death rate than humans today for freight.)
I'm not saying that this will happen as it completely depends on the double cycle for LLM or other tensor based capabilities to happen. This is equivalent to the now dead Moore's law. If we continue on the current doubling cycle, it is going to happen really quick. Other than the CPU, AI is software. AI in terms of ramp must be thought of as a software play (once through training) and not a hardware play.
Regardless, you have a great substack.
Very solid, very comprehensive, very interesting. Thanks a lot!