Thanks for the write-up. I'm keenly interested in watching this market develop over the next few years. If we really do get AGI, or something very close to it, I can imagine that the compute associated with inference will be so unbelievably large that this industry has a decade of incredible growth ahead of it.
What are your thoughts on AMD dominating the Inference market? With the Xilinx acquisition, the FPGA technology and some of the best CPU architecture in the business. Lisa Su has claimed their chips are the best in the market when it comes to inference and Meta has already made official that they are using exclusively MI300s for their Llama inference requirements.
Possibly a $130B+ opportunity for AMD in the coming years
You mention a caveat: "If small, customized models become the default for AI applications, this is a market unlocker for these companies, and the value prop becomes much more attractive."
I strongly believe this is the case. We already know that inference-as-a-service (IaaS) does scale the same as SaaS, economically there might not be a good reason to build big datacenters just for inference. Customers have rightful security concerns with handing over their precious data into a cloud; they will rather prefer to have inference on-site, AND customized & integrated into their software. Model distillation will help here.
Thanks for the write-up. I'm keenly interested in watching this market develop over the next few years. If we really do get AGI, or something very close to it, I can imagine that the compute associated with inference will be so unbelievably large that this industry has a decade of incredible growth ahead of it.
Agreed. Certainly a world where the market becomes so large that it creates several very large companies.
What are your thoughts on AMD dominating the Inference market? With the Xilinx acquisition, the FPGA technology and some of the best CPU architecture in the business. Lisa Su has claimed their chips are the best in the market when it comes to inference and Meta has already made official that they are using exclusively MI300s for their Llama inference requirements.
Possibly a $130B+ opportunity for AMD in the coming years
Very valuable. Thanks for sharing
This is a good read and thanks for the analysis. Having part of this ecosystem, I do understand the pain points.
You mention a caveat: "If small, customized models become the default for AI applications, this is a market unlocker for these companies, and the value prop becomes much more attractive."
I strongly believe this is the case. We already know that inference-as-a-service (IaaS) does scale the same as SaaS, economically there might not be a good reason to build big datacenters just for inference. Customers have rightful security concerns with handing over their precious data into a cloud; they will rather prefer to have inference on-site, AND customized & integrated into their software. Model distillation will help here.
Very interesting read, thank you.
What is your thought on Positron, Etched ?