Before You Buy AI Compute, Ask These Five Questions

At AWS, the biggest blocks of committed cloud capacity are now booked years ahead. Twenty years of GPU history show where else extra capacity can come from.

IndicStack ·

Ink drawing: a traveller with a suitcase stands in the aisle of a grand booking hall, where almost every bench has a folded orange cloth on it to mark it as reserved.

Key takeaways

  • When the biggest blocks are booked years ahead, look first at capacity you can recover.
  • At scale, failure is normal. Keeping machines healthy protects the capacity you already have.
  • A model that runs is not yet a model that runs fast, so test speed separately.
  • If you pay for an AI model through an API, ask your provider which of that work it covers.

Contents

Booked years ahead

"2027 is largely sold out." That was Amazon's CEO, Andy Jassy, on AWS's large blocks of committed cloud capacity. A good part of 2028 is already allocated, though AWS keeps headroom for on-demand use [1].

An analyst argued that the reporting overstated the position, since the pre-reserved blocks Jassy described differ from what a team can allocate day to day. A commenter warned that teams then reserve resources they fear they might need, "an anti-cloud pattern". AWS also raised its Capacity Blocks for ML rates by about 15% in January. These are reserved blocks of GPU and Trainium capacity [1].

A five-question capacity check

#Question
1Which of our compute is reserved ahead (committed), and which do we rent as needed (on-demand)?
2Do we have a second region or provider, tested and not only planned?
3Do our GPU machines have health checks that remove bad ones automatically?
4Do we measure what each machine type delivers, and place work by measured speed?
5Can our model run on a second kind of chip, and have we measured its speed there?

The past: a capability that waited for its moment

Ink drawing: a seedling with orange leaves grows under a glass bell jar on a workbench beside a circuit board; through the window, a large tree with orange leaves.

GPUs do many simple calculations at the same time. The book Chip War (2022) records that in 2006 Nvidia released CUDA, letting GPUs be programmed for work beyond graphics [2].

Meanwhile, ordinary processors met a limit. Dennard scaling was the projection that power per transistor would drop as chips packed in more transistors. It slowed from 2007 and had faded almost entirely by 2012. Hennessy and Patterson's 2019 lecture lists GPUs and neural network processors among the specialised chips of this era [3].

The breakthrough came in 2012, when a picture-recognition network trained on two NVIDIA GTX 580 GPUs won the ImageNet competition. Training took five to six days. Its top-5 error rate was 15.3%, against 26.2% for the next-best entry. Each GPU had only 3GB of memory, which its 2012 authors said limited the network's size [4].

The lesson: GPUs could be programmed for work beyond graphics from 2006, six years before the 2012 breakthrough. A capability can be ready long before the moment that makes it famous.

Google then built its own AI chip. By 2017, Google reported that its TPU had been deployed in its datacenters since 2015. Its 2017 measurements put it about 15 to 30 times faster at running trained models than the K80 GPU, on Google's own workloads [5].

Computing speed per dollar also rose. Epoch AI's 2022 study of 470 GPUs found that floating-point operations per second per dollar doubled about every 2.5 years, 2006 to 2021 [6].

The present: at scale, failure is normal

Ink drawing: a worker with tongs replaces one cracked link in a long iron chain that pulls a loaded cart; the new link glows orange.

In 2024, Meta reported 466 job interruptions in a 54-day snapshot of pre-training its Llama 3 405B model, run on up to 16K H100 GPUs. Of these, 419 were unexpected, and about 78% of those were attributed to confirmed or suspected hardware issues. Across those 2024 interruptions, significant manual intervention was needed only three times; automation handled the rest [7].

One failed GPU, degraded network adapter or slow machine can stop or slow a whole large training job. Netflix built its GPU platform to detect such failures, remove unhealthy capacity automatically and let long jobs recover. Netflix reports more than 99.95% GPU-node availability, and job success up from approximately 98% to more than 99.5% [8].

Machines also differ in speed. Across Uber's fleet, single-core performance may vary by up to a factor of almost 2. Uber also had to adjust its speed factors to match the actual performance seen in production [9]. Placing work by real performance cut the total CPU allocation of Uber's Kafka service by approximately 8% [10]. These are CPUs, not GPUs. Placing work by measured speed is worth testing on GPU work too.

Software is widening the choice of chip. Hugging Face reports that 96.5% of its tests that run on Google's TPUs pass [11]. Its separate lookup for AWS Trainium chips tests only models that fit on 4 Trainium2 devices. It warns that a pass "says nothing about throughput, or about longer generations" [12]. In other words, a model that runs is not yet a model that runs fast, so test speed separately.

India is building a shared public pool. The government approved the IndiaAI Mission in March 2024 [13]. The existing 38,000 GPUs were offered at ₹65 per hour, to lower compute barriers for startups, researchers, students and public institutions. On 17 February 2026, Minister Ashwini Vaishnaw announced that 20,000 more GPUs would join them in the coming weeks [14].

The future: more chips, from more makers

Ink drawing: a wall of many different keys; a hand takes the one orange key, and a padlock waits on the counter below.

NVIDIA announced on 27 October 2026 that its Vera Rubin NVL72 systems are generally available on Oracle Cloud Infrastructure. It notes that slow CPU work in agentic AI can leave GPUs waiting between model calls [15]. NVIDIA also says factory-built DSX POD units cut deployment time at a prepared site from six months to as little as 14 days [16]. Partners plan to deliver them worldwide in early 2027.

Oracle plans to offer Google's TPUs on its cloud in calendar year 2027 [17]. Meta will deploy a custom AMD Instinct GPU based on the MI450 architecture, and that work is informing AMD's future MI500 series [18]. AMD's CEO, Lisa Su, put it plainly: "There are going to be a lot of accelerators out there. It's not one-size-fits-all." [19]

Our reading: more kinds of chips make one question more important: does my model run well on this one?

Three lessons from twenty years

  1. A capability can be ready long before its moment. GPUs could be programmed beyond graphics from 2006, six years before the 2012 breakthrough.
  2. At scale, failure is normal. Keeping machines healthy protects the capacity you already have.
  3. When the biggest blocks are booked years ahead, look first at capacity you can recover. Keep machines healthy, place work by measured speed, and keep code portable.

What this means for infrastructure and inference

If you run GPUs yourself, that is your work. It means health checks, automatic removal of bad machines, saved progress (checkpoints), and measuring each machine type.

If you pay for an AI model through an API, ask your provider which of that work it covers. Ask how it detects failures, where your requests run, and what happens when capacity is short.

For builders and leaders

Builders: design long jobs to survive the loss of one machine, with checkpoints and restarts.

Leaders: treat compute as a supply contract with a tested fallback. Ask vendors for their reliability method, not only their price.

Where we fit

IndicStack is one OpenAI-compatible API for open-weight models such as DeepSeek, GLM, Qwen and Gemma, billed in rupees. Earlier, we suggested asking your API provider where your requests run. On IndicStack, every model is labelled India-hosted or Global, and one switch keeps your requests on India-hosted models only. Teams that need capacity of their own can ask us for a dedicated GPU deployment in India.

Disclaimer

This article is for information only. It is not investment, procurement, legal or technical advice, and it does not replace your own testing of any hardware, cloud service or model. It was drafted with the help of AI. IndicStack chose the sources, every figure and reference was checked against its source, and the article was reviewed before publication. Product plans, prices and capacity are as at the date of publication and may change; roadmap dates are the makers' own statements. Naming an organisation, study or product does not imply endorsement by, or affiliation with, IndicStack.

Copyright © IndicStack Consultancy Services LLP 2026

References

  1. AWS Capacity Largely Sold Out Through 2027, With On Demand Headroom Reserved. InfoQ, Nov 01, 2026. https://www.infoq.com/news/2026/11/aws-capacity-sold-out/ (accessed 2 November 2026).
  2. Christopher Miller. Chip War. Scribner (Simon & Schuster). Scribner, 2022; ISBN 978-1-9821-7200-8 (as printed in the book) (accessed 2 November 2026).
  3. A New Golden Age for Computer Architecture. Communications of the ACM. https://doi.org/10.1145/3282307 (accessed 2 November 2026).
  4. ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS Proceedings (NIPS 2012). https://papers.nips.cc/paper_files/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html (accessed 2 November 2026).
  5. In-Datacenter Performance Analysis of a Tensor Processing Unit. Proceedings of ISCA 2017. https://doi.org/10.1145/3079856.3080246 (accessed 2 November 2026).
  6. Trends in GPU price-performance. Epoch AI, Jun. 27, 2022. https://epoch.ai/publications/trends-in-gpu-price-performance (accessed 2 November 2026).
  7. The Llama 3 Herd of Models. Meta (arXiv), July 23, 2024. https://arxiv.org/html/2407.21783 (accessed 2 November 2026).
  8. Netflix Treats GPU Reliability as a Distributed Systems Problem. InfoQ, Oct 30, 2026. https://www.infoq.com/news/2026/10/netflix-gpu-distributed-systems/ (accessed 2 November 2026).
  9. Ensuring Similar Computational Resources to Stateful Workloads Running on a Diverse Server Fleet. Uber Engineering Blog, October 13, 2026. https://www.uber.com/us/en/blog/diverse-server-fleet/ (accessed 2 November 2026).
  10. Uber Moves beyond CPU Cores with Performance-Aware Scheduling for Stateful Workloads. InfoQ, Oct 27, 2026. https://www.infoq.com/news/2026/10/uber-cpu-stateful-workloads/ (accessed 2 November 2026).
  11. Hugging Face ❤️ TorchTPU: TPUs, Natively. Hugging Face, October 20, 2026. https://huggingface.co/blog/hf-torchtpu (accessed 2 November 2026).
  12. Does Your Model Run on AWS Trainium? Look It Up. Hugging Face, October 23, 2026. https://huggingface.co/blog/torch-neuronx-model-support (accessed 2 November 2026).
  13. The Present and The Future of India’s AI Infrastructure. IndiaAI, Jul 04, 2024. https://indiaai.gov.in/article/the-present-and-the-future-of-india-s-ai-infrastructure (accessed 2 November 2026).
  14. 20,000 additional GPUs power India’s next phase of AI leadership. Press Information Bureau, Government of India, 17 February 2026. https://www.pib.gov.in/PressReleasePage.aspx?PRID=2229397&reg=48&lang=2 (accessed 2 November 2026).
  15. NVIDIA and Oracle Expand NVIDIA Vera Rubin to Multi-Gigawatt Scale and Bring NVIDIA Vera CPU to Oracle Cloud Infrastructure. NVIDIA Newsroom, October 27, 2026. https://nvidianews.nvidia.com/news/nvidia-and-oracle-expand-nvidia-vera-rubin-to-multi-gigawatt-scale-and-bring-nvidia-vera-cpu-to-oracle-cloud-infrastructure (accessed 2 November 2026).
  16. NVIDIA Introduces DSX POD AI Factories to Accelerate Europe’s Infrastructure Deployment. NVIDIA Newsroom, October 21, 2026. https://nvidianews.nvidia.com/news/nvidia-introduces-dsx-pod-ai-factories-to-accelerate-europes-infrastructure-deployment (accessed 2 November 2026).
  17. Oracle Brings Google TPUs to Oracle Cloud Infrastructure. Oracle, Oct 27, 2026. https://www.oracle.com/news/announcement/ai-world-oracle-brings-google-tpus-to-oracle-cloud-infrastructure-2026-10-27/ (accessed 2 November 2026).
  18. AMD, Meta Advance Partnership to power Meta's Infrastructure. AMD Newsroom, Oct. 27, 2026. https://newsroom.amd.com/news/amd-meta-advance-strategic-partnership-power-metas-ai-infrastructure/ (accessed 2 November 2026).
  19. OCP 2026: AMD CEO Highlights Open Infrastructure for AI. AMD Newsroom, October 12, 2026. https://newsroom.amd.com/news/ocp-2026-amd-ceo-lisa-su-keynote-takeaways/ (accessed 2 November 2026).

We use cookies

We use essential cookies to run this site, analytics to understand how it is used, and advertising technologies to measure campaigns. You can turn off analytics and advertising with Reject all or Settings. Cookie Policy

Cookie settings

Essential cookies are always on. Analytics and advertising are on unless you turn them off below. You can change your choice at any time.

Changing an existing tracking choice may refresh this page. Read the Cookie Policy.