The AI race is on, and the gap between the tech haves and have-nots is widening. But what if countries like Brazil and Mexico could leapfrog ahead? A groundbreaking study by Sandra Malagon, Monica A. Ulloa Ruiz, Tatiana Elizabeth Sandoval Plaza, and their team explores the feasibility of training massive language models in these nations, even with limited resources. Their research delves into the practicalities and costs of training a 10-trillion-token model, considering hardware, energy, and funding constraints.
This study shines a light on the potential for 'sovereign' AI models, addressing the computational inequalities that exist between wealthy and developing nations. The researchers used the DeepSeek-V3 model, a model with 671 billion parameters trained on 14.8 trillion tokens, as a benchmark. This scale was chosen to be ambitious yet achievable, supporting applications in science, government, and education without requiring cutting-edge technology.
The team evaluated four different infrastructure setups, combining new and old accelerator technologies with training periods of 90 and 150 days. This allowed them to analyze the trade-offs between hardware efficiency, energy use, and deployment feasibility. Newer accelerators were assigned a peak throughput of 2,000 TFLOPs, while older ones managed 312 TFLOPs. The study also factored in real-world training inefficiencies. The number of accelerators needed varied greatly, from around 350 of the newer generation (in the 150-day scenario) to over 2,200 of the older generation (in the 90-day scenario), highlighting the impact of hardware efficiency and training duration on infrastructure needs.
The researchers meticulously calculated resource requirements for each scenario. They estimated energy consumption based on training time, accelerator power draw, and the total number of accelerators, adjusting for data center overhead. They estimated power draw of 700W per newer generation accelerator and 400W per older generation accelerator, resulting in energy needs ranging from 0.3 to 3.3 GWh. Peak electrical load estimates, assuming full simultaneous accelerator usage, ranged from 0.41MW (newer generation, 150 days) to 1.49MW (older generation, 90 days), staying within the capacity of standard medium-voltage distribution infrastructure. Capital expenditures were calculated using hardware prices and integration costs, with import duties applied to Brazil. This detailed methodology offers a nuanced view of the financial and logistical challenges of building sovereign AI capabilities in middle-income countries.
The big takeaway? Sovereign-scale language model training is indeed technically and financially possible for countries like Brazil and Mexico, even with limited resources. The study found that all configurations stay within export control limits and don't overload typical electrical infrastructure. But here's where it gets controversial: The key to fiscal success lies in hardware efficiency. Newer generation accelerators significantly reduce overall costs. Training a substantial model using newer hardware could cost between 8 to 14 million USD, while older processors could range from 19 to 32 million USD due to higher energy consumption and hardware demands.
Extending training timelines is a viable strategy to manage hardware limitations. This approach enables the creation of locally relevant language models without necessarily competing at the global AI frontier. The authors acknowledge that, while all scenarios are technically feasible, practical deployment in urban areas might require additional permits and infrastructure upgrades, especially for configurations with higher power demands.
And this is the part most people miss: This research opens the door for further exploration. Future studies could investigate the impact of different model sizes and data requirements, as well as the potential of distributed training methods to reduce costs and improve accessibility.
What do you think? Do you believe that developing nations can successfully compete in the AI race? Share your thoughts in the comments below!
