Training Large Language Models: From TRPO to GRPO

Stay Ahead, Stay ONMINE

Training Large Language Models: From TRPO to GRPO

Deepseek has recently made quite a buzz in the AI community, thanks to its impressive performance at relatively low costs. I think this is a perfect opportunity to dive deeper into how Large Language Models (LLMs) are trained. In this article, we will focus on the Reinforcement Learning (RL) side of things: we will cover […]

Deepseek has recently made quite a buzz in the AI community, thanks to its impressive performance at relatively low costs. I think this is a perfect opportunity to dive deeper into how Large Language Models (LLMs) are trained. In this article, we will focus on the Reinforcement Learning (RL) side of things: we will cover TRPO, PPO, and, more recently, GRPO (don’t worry, I will explain all these terms soon!)

I have aimed to keep this article relatively easy to read and accessible, by minimizing the math, so you won’t need a deep Reinforcement Learning background to follow along. However, I will assume that you have some familiarity with Machine Learning, Deep Learning, and a basic understanding of how LLMs work.

I hope you enjoy the article!

The 3 steps of LLM training

Before diving into RL specifics, let’s briefly recap the three main stages of training a Large Language Model:

Pre-training: the model is trained on a massive dataset to predict the next token in a sequence based on preceding tokens.
Supervised Fine-Tuning (SFT): the model is then fine-tuned on more targeted data and aligned with specific instructions.
Reinforcement Learning (often called RLHF for Reinforcement Learning with Human Feedback): this is the focus of this article. The main goal is to further refine responses’ alignments with human preferences, by allowing the model to learn directly from feedback.

Reinforcement Learning Basics

Before diving deeper, let’s briefly revisit the core ideas behind Reinforcement Learning.

RL is quite straightforward to understand at a high level: an agent interacts with an environment. The agent resides in a specific state within the environment and can take actions to transition to other states. Each action yields a reward from the environment: this is how the environment provides feedback that guides the agent’s future actions.

Consider the following example: a robot (the agent) navigates (and tries to exit) a maze (the environment).

The state is the current situation of the environment (the robot’s position in the maze).
The robot can take different actions: for example, it can move forward, turn left, or turn right.
Successfully navigating towards the exit yields a positive reward, while hitting a wall or getting stuck in the maze results in negative rewards.

Easy! Now, let’s now make an analogy to how RL is used in the context of LLMs.

RL in the context of LLMs

When used during LLM training, RL is defined by the following components:

The LLM itself is the agent
Environment: everything external to the LLM, including user prompts, feedback systems, and other contextual information. This is basically the framework the LLM is interacting with during training.
Actions: these are responses to a query from the model. More specifically: these are the tokens that the LLM decides to generate in response to a query.
State: the current query being answered along with tokens the LLM has generated so far (i.e., the partial responses).
Rewards: this is a bit more tricky here: unlike the maze example above, there is usually no binary reward. In the context of LLMs, rewards usually come from a separate reward model, which outputs a score for each (query, response) pair. This model is trained from human-annotated data (hence “RLHF”) where annotators rank different responses. The goal is for higher-quality responses to receive higher rewards.

Note: in some cases, rewards can actually get simpler. For example, in DeepSeekMath, rule-based approaches can be used because math responses tend to be more deterministic (correct or wrong answer)

Policy is the final concept we need for now. In RL terms, a policy is simply the strategy for deciding which action to take. In the case of an LLM, the policy outputs a probability distribution over possible tokens at each step: in short, this is what the model uses to sample the next token to generate. Concretely, the policy is determined by the model’s parameters (weights). During RL training, we adjust these parameters so the LLM becomes more likely to produce “better” tokens— that is, tokens that produce higher reward scores.

We often write the policy as:

where a is the action (a token to generate), s the state (the query and tokens generated so far), and θ (model’s parameters).

This idea of finding the best policy is the whole point of RL! Since we don’t have labeled data (like we do in supervised learning) we use rewards to adjust our policy to take better actions. (In LLM terms: we adjust the parameters of our LLM to generate better tokens.)

TRPO (Trust Region Policy Optimization)

An analogy with supervised learning

Let’s take a quick step back to how supervised learning typically works. you have labeled data and use a loss function (like cross-entropy) to measure how close your model’s predictions are to the true labels.

We can then use algorithms like backpropagation and gradient descent to minimize our loss function and update the weights θ of our model.

Recall that our policy also outputs probabilities! In that sense, it is analogous to the model’s predictions in supervised learning… We are tempted to write something like:

where s is the current state and a is a possible action.

A(s, a) is called the advantage function and measures how good is the chosen action in the current state, compared to a baseline. This is very much like the notion of labels in supervised learning but derived from rewards instead of explicit labeling. To simplify, we can write the advantage as:

In practice, the baseline is calculated using a value function. This is a common term in RL that I will explain later. What you need to know for now is that it measures the expected reward we would receive if we continue following the current policy from the state s.

What is TRPO?

TRPO (Trust Region Policy Optimization) builds on this idea of using the advantage function but adds a critical ingredient for stability: it constrains how far the new policy can deviate from the old policy at each update step (similar to what we do with batch gradient descent for example).

It introduces a KL divergence term (see it as a measure of similarity) between the current and the old policy:

It also divides the policy by the old policy. This ratio, multiplied by the advantage function, gives us a sense of how beneficial each update is relative to the old policy.

Putting it all together, TRPO tries to maximize a surrogate objective (which involves the advantage and the policy ratio) subject to a KL divergence constraint.

PPO (Proximal Policy Optimization)

While TRPO was a significant advancement, it’s no longer used widely in practice, especially for training LLMs, due to its computationally intensive gradient calculations.

Instead, PPO is now the preferred approach in most LLMs architecture, including ChatGPT, Gemini, and more.

It is actually quite similar to TRPO, but instead of enforcing a hard constraint on the KL divergence, PPO introduces a “clipped surrogate objective” that implicitly restricts policy updates, and greatly simplifies the optimization process.

Here is a breakdown of the PPO objective function we maximize to tweak our model’s parameters.

GRPO (Group Relative Policy Optimization)

How is the value function usually obtained?

Let’s first talk more about the advantage and the value functions I introduced earlier.

In typical setups (like PPO), a value model is trained alongside the policy. Its goal is to predict the value of each action we take (each token generated by the model), using the rewards we obtain (remember that the value should represent the expected cumulative reward).

Here is how it works in practice. Take the query “What is 2+2?” as an example. Our model outputs “2+2 is 4” and receives a reward of 0.8 for that response. We then go backward and attribute discounted rewards to each prefix:

“2+2 is 4” gets a value of 0.8
“2+2 is” (1 token backward) gets a value of 0.8γ
“2+2” (2 tokens backward) gets a value of 0.8γ²
etc.

where γ is the discount factor (0.9 for example). We then use these prefixes and associated values to train the value model.

Important note: the value model and the reward model are two different things. The reward model is trained before the RL process and uses pairs of (query, response) and human ranking. The value model is trained concurrently to the policy, and aims at predicting the future expected reward at each step of the generation process.

What’s new in GRPO

Even if in practice, the reward model is often derived from the policy (training only the “head”), we still end up maintaining many models and handling multiple training procedures (policy, reward, value model). GRPO streamlines this by introducing a more efficient method.

Remember what I said earlier?

In PPO, we decided to use our value function as the baseline. GRPO chooses something else: Here is what GRPO does: concretely, for each query, GRPO generates a group of responses (group of size G) and uses their rewards to calculate each response’s advantage as a z-score:

where rᵢ is the reward of the i-th response and μ and σ are the mean and standard deviation of rewards in that group.

This naturally eliminates the need for a separate value model. This idea makes a lot of sense when you think about it! It aligns with the value function we introduced before and also measures, in a sense, an “expected” reward we can obtain. Also, this new method is well adapted to our problem because LLMs can easily generate multiple non-deterministic outputs by using a low temperature (controls the randomness of tokens generation).

This is the main idea behind GRPO: getting rid of the value model.

Finally, GRPO adds a KL divergence term (to be exact, GRPO uses a simple approximation of the KL divergence to improve the algorithm further) directly into its objective, comparing the current policy to a reference policy (often the post-SFT model).

See the final formulation below:

And… that’s mostly it for GRPO! I hope this gives you a clear overview of the process: it still relies on the same foundational ideas as TRPO and PPO but introduces additional improvements to make training more efficient, faster, and cheaper — key factors behind DeepSeek’s success.

Conclusion

Reinforcement Learning has become a cornerstone for training today’s Large Language Models, particularly through PPO, and more recently GRPO. Each method rests on the same RL fundamentals — states, actions, rewards, and policies — but adds its own twist to balance stability, efficiency, and human alignment:

• TRPO introduced strict policy constraints via KL divergence

• PPO eased those constraints with a clipped objective

• GRPO took an extra step by removing the value model requirement and using group-based reward normalization. Of course, DeepSeek also benefits from other innovations, like high-quality data and other training strategies, but that is for another time!

I hope this article gave you a clearer picture of how these methods connect and evolve. I believe that Reinforcement Learning will become the main focus in training LLMs to improve their performance, surpassing pre-training and SFT in driving future innovations.

If you’re interested in diving deeper, feel free to check out the references below or explore my previous posts.

Thanks for reading, and feel free to leave a clap and a comment!

Want to learn more about Transformers or dive into the math behind the Curse of Dimensionality? Check out my previous articles:

Transformers: How Do They Transform Your Data?
Diving into the Transformers architecture and what makes them unbeatable at language taskstowardsdatascience.com

The Math Behind “The Curse of Dimensionality”
Dive into the “Curse of Dimensionality” concept and understand the math behind all the surprising phenomena that arise…towardsdatascience.com

References:

Stay Ahead

Explore More Insights

Stay ahead with more perspectives on cutting-edge power, infrastructure, energy, bitcoin and AI solutions. Explore these articles to uncover strategies and insights shaping the future of industries.

LogicMonitor closes Catchpoint buy, targets AI observability

The acquisition combines LogicMonitor’s observability platform with Catchpoint’s internet-level intelligence, which monitors performance from thousands of global vantage points. Once integrated, Catchpoint’s synthetic monitoring, network data, and real-user monitoring will feed directly into Edwin AI, LogicMonitor’s intelligence engine. The goal is to let enterprise customers shift from reactive alerting to

Akamai acquires Fermyon for edge computing as WebAssembly comes of age

Spin handles compilation from source to WebAssembly bytecode and manages execution on target platforms. The runtime abstracts the underlying technology while preserving WebAssembly’s performance and security characteristics. This bet on WebAssembly standards has paid off as the technology matured. WebAssembly has evolved significantly beyond its initial browser-focused design to support

Winners and losers in the latest Top500 supercomputer list

Winner: Slingshot-11 Slingshot-11 is a 200G proprietary interconnect developed by HPE and its Cray supercomputer subsidiary. As the number of Cray systems increases on the list, so goes the number of Slingshot-11 based systems. The total number of Slingshot-11 systems jumped from 37 and 2024 to 52 this year. Loser:

Chevron Joins TotalEnergies in New Nigerian Exploration Blocks

Chevron Corp has signed a deal to acquire 40 percent in Petroleum Prospecting License (PPL) 2000 and PPL 2001 offshore Nigeria from TotalEnergies SE. TotalEnergies will retain operatorship with a 40 percent interest. Local player South Atlantic Petroleum Ltd owns 20 percent. “This new joint venture aims at derisking and

Perenco Raises Oil Production Capacity in Chad

Perenco said it has increased its oil production capacity in Chad to over 18,000 barrels per day (bpd), coming from the Badila and Mangara fields. The completion of a 12-well drilling campaign in Badila added a peak production of 7,000 bpd, the Perrodo family-owned company said in an online statement, noting it has exceeded its goal of 16,000 bpd when the fields restarted flows three years ago. “Eight horizontal wells targeting the Upper Cretaceous reservoir were drilled during the campaign, alongside three water disposal wells and one appraisal well”, Perenco said. “The campaign also consisted of the installation of four gas turbines, providing extra power generation from the field, as well as an uplift in processing capabilities, in order to handle increased production from Badila”. “The GWDC rig has now moved to PCM’s Krim development in the Doba Basin in southern Chad where it will conduct an additional eight-well drilling campaign”, Perenco said, referring to its subsidiary PetroChad Mangara (PCM). “Using the associated gas from its production, PetroChad now provides a sustainable energy solution to the residents of Moundou, the country’s second-largest economic city with a population of around 100,000”, Perenco said. Elsewhere in Central Africa, Perenco earlier this year announced the construction of a new platform to serve Republic of the Congo’s Kombi-Likalala-Libondo 2 permit. Expected to start operations “early 2026”, Kombi 2 will recover about seven million cubic feet of gas per day, Perenco said in a press release June 12. The platform will develop an additional 10 million barrels of reserves by optimizing existing wells. Kombi 2 will have two gas turbines connected to a 33-kilovolt electrical hub. “The Kombi 2 construction project, including the upcoming drilling phases, represents an investment of over $200 million”, Perenco said. It added, “The recent renewal of the Ikalou II and

OEUK Raising Awareness of New Worker Weight Limit Policy

Industry body Offshore Energies UK (OEUK) announced, in a statement sent to Rigzone recently, that “practical solutions for healthier lifestyles and support for managing weight loss” will be shared by the group as part of its campaign to raise awareness of a new safety policy for people working offshore. That new safety initiative is called the Safe Weight Limit Policy and comes into effect in November 2026, the statement highlighted. In the statement, OEUK described the policy as “an industry-wide initiative that has been developed following extensive engagement between OEUK, HM Coastguard, helicopter operators, and member companies”. “It addresses the increase in the weight of offshore workers – a trend also seen across the UK population – which has associated challenges for evacuation, rescue, and medical response offshore,” OEUK said in the statement. According to an explainer page on the Safe Weight Limit Policy on OEUK’s website, the clothed weight limit for offshore workers under the policy is 124kg, including a 0.7kg margin. Clothed weight refers to a person’s weight while dressed in accordance with the industry travel clothing policy for the relevant season, the page notes. The page outlines that the limit “ensures every person can be safely evacuated or rescued, particularly by search and rescue (SAR) helicopter winch”. “The combined load of the winchman, stretcher, and equipment leaves a maximum capacity of 124.7kg for a patient. Anyone above this weight cannot be guaranteed rescue by SAR helicopter,” the page highlights. The policy applies to all offshore installations operating under accepted Safety Cases as defined in the Offshore Installations (Offshore Safety Directive) (Safety Case etc.) Regulations 2015 and does not apply to marine vessels, which are governed by different regulations, the page shows. It also applies only to outbound (offshore) flights, according to the page, which points out that

EU Seals Deal to Phase Out Russian Gas by 2027

The European Union has reached a deal to phase out Russian gas faster than originally planned, a move that aims to finally sever ties between the bloc and its once-primary energy supplier. In the aftermath of the invasion of Ukraine, traders and energy companies have closely monitored the EU’s shift away from Russia toward alternative suppliers such as the US and the Middle East. But while Europe halved purchases after the war began in 2022, Russian gas has continued to account for roughly a fifth of imports. Negotiators representing member states, the European Parliament and the European Commission cut that remaining link in the early hours of Wednesday, agreeing to gradually prohibit liquefied natural gas imports from Moscow by the end of 2026. That’s a year earlier than originally proposed by the Commission and in line with a ban on seaborne deliveries already approved by the EU under its latest sanctions package on Russia. Pipeline gas imports under long-term deals will have to halt by Sept. 30, 2027, with a possibility of an extension to Nov. 1, 2027, depending on fulfillment of gas storage targets set by the EU. That compares with an end-2027 ban on those contracts originally put forward by the commission. “Finally, and for good, we are turning off the tap on Russian gas,” EU Energy Commissioner Dan Jorgensen said on X. “Europe has chosen energy security and independence. We will never go back to volatile supplies and market manipulation.” The EU had proposed the measure in June to address risks to its energy security after the crisis triggered by Russia’s invasion of Ukraine and Moscow’s subsequent curbs on gas flows to the bloc. The US has sought to broker a peace deal between Russia and Ukraine, and speculation that a potential agreement could eventually ease sanctions on

Eni Starts Up Congo LNG Phase 2

Eni SpA said Tuesday the second phase of Congo LNG in the Republic of the Congo has started operations, bringing the project’s capacity to three million metric tons per annum (MMtpa) of liquefied natural gas (LNG) or 4.5 billion cubic meters (158.92 billion cubic feet) a year of natural gas equivalent. Feed gas has been introduced to the new Nguya floating liquefaction unit. Eni expects to dispatch phase 2’s first LNG cargo “early 2026”, it said in an online statement. “Congo LNG Phase 2 features three production platforms as well as the Scarabeo 5 unit dedicated to gas treatment and compression and the Nguya FLNG for liquefaction and export… This integrated configuration enables the full development of gas resources from the offshore Nene and Litchendjili fields, in the Marine 12 license, and ensures flexible, phased management of volumes, guaranteeing a steady flow to both the Tango FLNG unit, operational since late 2023, and the Nguya FLNG”, Eni said. “Phase 2 has come on stream ahead of the project schedule, just 35 months after construction of the Nguya FLNG began, setting a new benchmark within the industry for execution speed and efficiency. “This milestone was achieved thanks to a combination of technological innovation, rigorous industrial planning and strong engagement with local stakeholders. “A significant part of the project was carried out entirely in Congo, enhancing the skills of the local workforce and further strengthening the national industrial sector. “The Nguya FLNG, 376 meters long and 60 meters wide, employs advanced technologies to reduce its carbon footprint and is designed to process gas with different compositions, supporting the potential development of additional fields in the area. “The Scarabeo 5, converted from a drilling rig into a gas treatment, separation and compression unit, also incorporates decarbonization-oriented solutions, serving as a concrete example of

Nearly 250 Energy Projects Gain EU PCI, PMI Status

The European Commission has listed 235 cross-border energy projects as Projects of Common Interest (PCIs) and Projects of Mutual Interest (PMIs), only the second such list since its launch in 2023. “The selected projects will be eligible to apply for EU financing from the Connecting Europe Facility and will benefit from expedited permitting and regulatory processes for swift execution and delivery”, the Commission said in an online statement. “These cross-projects will strengthen energy connectivity across the continent, bringing nearer the completion of the Energy Union. By allowing vital interconnections across the EU and with neighboring countries, these projects can play a strategic role in increasing EU’s competitiveness, decarbonization and enhancing Europe’s energy security and independence. “According to a recent Commission study, investment needs in European energy infrastructure – electricity, hydrogen and CO2 networks – will near EUR 1.5 trillion [$1.75 trillion] from 2024 to 2040. This project lineup and the related expected investments volumes will contribute to reaching the needs identified for 2040”. The new list includes 113 electricity, smart electricity and offshore grid projects to accommodate the growing share of renewables; 100 hydrogen and electrolyzer projects; 17 carbon transport infrastructure projects; three smart gas grid projects to digitalize the natural gas network. The Commission has also retained two projects to link Malta and Cyprus to the gas network of mainland Europe. The list will be submitted to the European Parliament and Council in the form of a Delegated Act for scrutiny, as mandated by the Trans-European Networks for Energy Regulation. “Both co-legislators have two months to either accept or reject the list in full but may not amend it”, the Commission said. “This process can be extended by two months, if requested by the co-legislators. “Once the list is adopted, the Commission will further reinforce its work with project promoters and member states to help ensure that the selected

EU Nearing Deal to End Russian Fossil Fuel Imports

The European Union is closing in on a deal to phase out Russian fossil fuels, a move that will embed into law the end of the bloc’s reliance on its former top energy supplier. Negotiators representing member states, the European Parliament and the European Commission are scheduled to meet on Tuesday evening in Brussels to iron out the final shape of a regulation that will set a date for banning Russian gas imports. The measure was proposed by the commission in June to address risks to EU energy security after the crisis triggered by Russia’s invasion of Ukraine and Moscow’s subsequent curbs on gas flows to the bloc. Despite recent attempts by the US to broker a peace deal in Ukraine, the EU has no plans to give up on the shift away from Russian gas. Speculation that a potential agreement could eventually lead to an easing of sanctions on Moscow’s energy exports, allowing other regions to buy fuel, has contributed to benchmark European gas futures recording their longest downward streak in almost four years. The EU talks will need to resolve the exact timeline for the phaseout. While member states in the EU Council endorsed the commission’s plan to ban all Russian gas supplies by the end of 2027, the Parliament is pushing to accelerate it by one year. That would align the end of piped-gas imports with the halt to seaborne deliveries already approved by the EU under its latest sanctions package on Russia. But whereas sanctions are temporary by design, the regulation known as RePowerEU is a separate, long-term plan to cut reliance on Moscow for good. The commission has made it clear that the measure will remain, regardless of any peace deal. “The European Union can make history tonight and change the course of our energy

What is co-packaged optics? A solution for surging capacity in AI data center networks

When it announced its CPO-capable switches, Nvidia said they would improve resiliency by 10 times at scale compared to previous switch generations. Several factors contribute to this claim, including the fact that the optical switches require four times fewer lasers, Shainer says. Whereas the laser source was previously part of the transceiver, the optical engine is now incorporated onto the ASIC, allowing multiple optical channels to share a single laser. Additionally, in Nvidia’s implementation, the laser source is located outside of the switch. “We want to keep the ability to replace a laser source in case it has failed and needs to be replaced,” he says. “They are completely hot-swappable, so you don’t need to shut down the switch.” Nonetheless, you may often hear that when something fails in a CPO box, you need to replace the entire box. That may be true if it’s the photonics engine embedded in silicon inside the box. “But they shouldn’t fail that often. There are not a lot of moving parts in there,” Wilkinson says. While he understands the argument around failures, he doesn’t expect it to pan out as CPO gets deployed. “It’s a fallacy,” he says. There’s also a simple workaround to the resiliency issue, which hyperscalers are already talking about, Karavalas says: overbuild. “Have 10% more ports than you need or 5%,” he says. “If you lose a port because the optic goes bad, you just move it and plug it in somewhere else.” Which vendors are backing co-packaged optics? In terms of vendors that have or plan to have CPO offerings, the list is not long, unless you include various component players like TSMC. But in terms of major switch vendors, here’s a rundown: Broadcom has been making steady progress on CPO since 2021. It is now shipping “to

Nvidia’s $2B Synopsys stake tests independence of open AI interconnect standard

But the concern for enterprise IT leaders is whether Nvidia’s financial stakes in UALink consortium members could influence the development of an open standard specifically designed to compete with Nvidia’s proprietary technology and to give enterprises more choices in the datacenter. Organizations planning major AI infrastructure investments view such open standards as critical to avoiding vendor lock-in and maintaining competitive pricing. “This does put more pressure on UALink since Intel is also a member and also took investment from Nvidia,” Sag said. UALink and Synopsys’s critical role UALink represents the industry’s most significant effort to prevent vendor lock-in for AI infrastructure. The consortium ratified its UALink 200G 1.0 Specification in April, defining an open standard for connecting up to 1,024 AI accelerators within computing pods at 200 Gbps per lane — directly competing with Nvidia’s NVLink for scale-up applications. Synopsys plays a critical role. The company joined UALink’s board in January and in December announced the industry’s first UALink design components, enabling chip designers to build UALink-compatible accelerators. Analysts flag governance concerns Gaurav Gupta, VP analyst at Gartner, acknowledged the tension. “The Nvidia-Synopsys deal does raise questions around the future of UALink as Synopsys is a key partner of the consortium and holds critical IP for UALink, which competes with Nvidia’s proprietary NVLink,” he said. Sanchit Vir Gogia, chief analyst at Greyhound Research, sees deeper structural concerns. “Synopsys is not a peripheral player in this standard; it is the primary supplier of UALink IP and a board member within the UALink Consortium,” he said. “Nvidia’s entry into Synopsys’ shareholder structure risks contaminating that neutrality.”

Cooling crisis at CME: A wakeup call for modern infrastructure governance

Organizations should reassess redundancy However, he pointed out, “the deeper concern is that CME had a secondary data center ready to take the load, yet the failover threshold was set too high, and the activation sequence remained manually gated. The decision to wait for the cooling issue to self-correct rather than trigger the backup site immediately revealed a governance model that had not evolved to keep pace with the operational tempo of modern markets.” Thermal failures, he said, “do not unfold on the timelines assumed in traditional disaster recovery playbooks. They escalate within minutes and demand automated responses that do not depend on human certainty about whether a facility will recover in time.” Matt Kimball, VP and principal analyst at Moor Insights & Strategy, said that to some degree what happened in Aurora highlights an issue that may arise on occasion: “the communications gap that can exist between IT executives and data center operators. Think of ‘rack in versus rack out’ mindsets.” Often, he said, the operational elements of that data center environment, such as cooling, power, fire hazards, physical security, and so forth, fall outside the realm of an IT executive focused on delivering IT services to the business. “And even if they don’t fall outside the realm, these elements are certainly not a primary focus,” he noted. “This was certainly true when I was living in the IT world.” Additionally, said Kimball, “this highlights the need for organizations to reassess redundancy and resilience in a new light. Again, in IT, we tend to focus on resilience and redundancy at the app, server, and workload layers. Maybe even cluster level. But as we continue to place more and more of a premium on data, and the terms ‘business critical’ or ‘mission critical’ have real relevance, we have to zoom out

Microsoft loses two senior AI infrastructure leaders as data center pressures mount

Microsoft did not immediately respond to a request for comment. Microsoft’s constraints Analysts say the twin departures mark a significant setback for Microsoft at a critical moment in the AI data center race, with pressure mounting from both OpenAI’s model demands and Google’s infrastructure scale. “Losing some of the best professionals working on this challenge could set Microsoft back,” said Neil Shah, partner and co-founder at Counterpoint Research. “Solving the energy wall is not trivial, and there may have been friction or strategic differences that contributed to their decision to move on, especially if they saw an opportunity to make a broader impact and do so more lucratively at a company like Nvidia.” Even so, Microsoft has the depth and ecosystem strength to continue doubling down on AI data centers, said Prabhu Ram, VP for industry research at Cybermedia Research. According to Sanchit Gogia, chief analyst at Greyhound Research, the departures come at a sensitive moment because Microsoft is trying to expand its AI infrastructure faster than physical constraints allow. “The executives who have left were central to GPU cluster design, data center engineering, energy procurement, and the experimental power and cooling approaches Microsoft has been pursuing to support dense AI workloads,” Gogia said. “Their exit coincides with pressures the company has already acknowledged publicly. GPUs are arriving faster than the company can energize the facilities that will house them, and power availability has overtaken chip availability as the real bottleneck.”

What is Edge AI? When the cloud isn’t close enough

Many edge devices can periodically send summarized or selected inference output data back to a central system for model retraining or refinement. That feedback loop helps the model improve over time while still keeping most decisions local. And to run efficiently on constrained edge hardware, the AI model is often pre-processed by techniques such as quantization (which reduces precision), pruning (which removes redundant parameters), or knowledge distillation (which trains a smaller model to mimic a larger one). These optimizations reduce the model’s memory, compute, and power demands so it can run more easily on an edge device. What technologies make edge AI possible? The concept of the “edge” always assumes that edge devices are less computationally powerful than data centers and cloud platforms. While that remains true, overall improvements in computational hardware have made today’s edge devices much more capable than those designed just a few years ago. In fact, a whole host of technological developments have come together to make edge AI a reality. Specialized hardware acceleration. Edge devices now ship with dedicated AI-accelerators (NPUs, TPUs, GPU cores) and system-on-chip units tailored for on-device inference. For example, companies like Arm have integrated AI-acceleration libraries into standard frameworks so models can run efficiently on Arm-based CPUs. Connectivity and data architecture. Edge AI often depends on durable, low-latency links (e.g., 5G, WiFi 6, LPWAN) and architectures that move compute closer to data. Merging edge nodes, gateways, and local servers means less reliance on distant clouds. And technologies like Kubernetes can provide a consistent management plane from the data center to remote locations. Deployment, orchestration, and model lifecycle tooling. Edge AI deployments must support model-update delivery, device and fleet monitoring, versioning, rollback and secure inference — especially when orchestrated across hundreds or thousands of locations. VMware, for instance, is offering traffic management

Networks, AI, and metaversing

Our first, conservative, view says that AI’s network impact is largely confined to the data center, to connect clusters of GPU servers and the data they use as they crunch large language models. It’s all “horizontal” traffic; one TikTok challenge would generate way more traffic in the wide area. WAN costs won’t rise for you as an enterprise, and if you’re a carrier you won’t be carrying much new, so you don’t have much service revenue upside. If you don’t host AI on premises, you can pretty much dismiss its impact on your network. Contrast that with the radical metaverse view, our third view. Metaverses and AR/VR transform AI missions, and network services, from transaction processing to event processing, because the real world is a bunch of events pushing on you. They also let you visualize the way that process control models (digital twins) relate to the real world, which is critical if the processes you’re modeling involve human workers who rely on their visual sense. Could it be that the reason Meta is willing to spend on AI, is that the most credible application of AI, and the most impactful for networks, is the metaverse concept? In any event, this model of AI, by driving the users’ experiences and activities directly, demands significant edge connectivity, so you could expect it to have a major impact on network requirements. In fact, just dipping your toes into a metaverse could require a major up-front network upgrade. Networks carry traffic. Traffic is messages. More messages, more traffic, more infrastructure, more service revenue…you get the picture. Door number one, to the AI giant future, leads to nothing much in terms of messages. Door number three, metaverses and AR/VR, leads to a message, traffic, and network revolution. I’ll bet that most enterprises would doubt

Microsoft will invest $80B in AI data centers in fiscal 2025

And Microsoft isn’t the only one that is ramping up its investments into AI-enabled data centers. Rival cloud service providers are all investing in either upgrading or opening new data centers to capture a larger chunk of business from developers and users of large language models (LLMs). In a report published in October 2024, Bloomberg Intelligence estimated that demand for generative AI would push Microsoft, AWS, Google, Oracle, Meta, and Apple would between them devote $200 billion to capex in 2025, up from $110 billion in 2023. Microsoft is one of the biggest spenders, followed closely by Google and AWS, Bloomberg Intelligence said. Its estimate of Microsoft’s capital spending on AI, at $62.4 billion for calendar 2025, is lower than Smith’s claim that the company will invest $80 billion in the fiscal year to June 30, 2025. Both figures, though, are way higher than Microsoft’s 2020 capital expenditure of “just” $17.6 billion. The majority of the increased spending is tied to cloud services and the expansion of AI infrastructure needed to provide compute capacity for OpenAI workloads. Separately, last October Amazon CEO Andy Jassy said his company planned total capex spend of $75 billion in 2024 and even more in 2025, with much of it going to AWS, its cloud computing division.

John Deere unveils more autonomous farm machines to address skill labor shortage

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Self-driving tractors might be the path to self-driving cars. John Deere has revealed a new line of autonomous machines and tech across agriculture, construction and commercial landscaping. The Moline, Illinois-based John Deere has been in business for 187 years, yet it’s been a regular as a non-tech company showing off technology at the big tech trade show in Las Vegas and is back at CES 2025 with more autonomous tractors and other vehicles. This is not something we usually cover, but John Deere has a lot of data that is interesting in the big picture of tech. The message from the company is that there aren’t enough skilled farm laborers to do the work that its customers need. It’s been a challenge for most of the last two decades, said Jahmy Hindman, CTO at John Deere, in a briefing. Much of the tech will come this fall and after that. He noted that the average farmer in the U.S. is over 58 and works 12 to 18 hours a day to grow food for us. And he said the American Farm Bureau Federation estimates there are roughly 2.4 million farm jobs that need to be filled annually; and the agricultural work force continues to shrink. (This is my hint to the anti-immigration crowd). John Deere’s autonomous 9RX Tractor. Farmers can oversee it using an app. While each of these industries experiences their own set of challenges, a commonality across all is skilled labor availability. In construction, about 80% percent of contractors struggle to find skilled labor. And in commercial landscaping, 86% of landscaping business owners can’t find labor to fill open positions, he said. “They have to figure out how to do

2025 playbook for enterprise AI success, from agents to evals

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More 2025 is poised to be a pivotal year for enterprise AI. The past year has seen rapid innovation, and this year will see the same. This has made it more critical than ever to revisit your AI strategy to stay competitive and create value for your customers. From scaling AI agents to optimizing costs, here are the five critical areas enterprises should prioritize for their AI strategy this year. 1. Agents: the next generation of automation AI agents are no longer theoretical. In 2025, they’re indispensable tools for enterprises looking to streamline operations and enhance customer interactions. Unlike traditional software, agents powered by large language models (LLMs) can make nuanced decisions, navigate complex multi-step tasks, and integrate seamlessly with tools and APIs. At the start of 2024, agents were not ready for prime time, making frustrating mistakes like hallucinating URLs. They started getting better as frontier large language models themselves improved. “Let me put it this way,” said Sam Witteveen, cofounder of Red Dragon, a company that develops agents for companies, and that recently reviewed the 48 agents it built last year. “Interestingly, the ones that we built at the start of the year, a lot of those worked way better at the end of the year just because the models got better.” Witteveen shared this in the video podcast we filmed to discuss these five big trends in detail. Models are getting better and hallucinating less, and they’re also being trained to do agentic tasks. Another feature that the model providers are researching is a way to use the LLM as a judge, and as models get cheaper (something we’ll cover below), companies can use three or more models to

OpenAI’s red teaming innovations define new essentials for security leaders in the AI era

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More OpenAI has taken a more aggressive approach to red teaming than its AI competitors, demonstrating its security teams’ advanced capabilities in two areas: multi-step reinforcement and external red teaming. OpenAI recently released two papers that set a new competitive standard for improving the quality, reliability and safety of AI models in these two techniques and more. The first paper, “OpenAI’s Approach to External Red Teaming for AI Models and Systems,” reports that specialized teams outside the company have proven effective in uncovering vulnerabilities that might otherwise have made it into a released model because in-house testing techniques may have missed them. In the second paper, “Diverse and Effective Red Teaming with Auto-Generated Rewards and Multi-Step Reinforcement Learning,” OpenAI introduces an automated framework that relies on iterative reinforcement learning to generate a broad spectrum of novel, wide-ranging attacks. Going all-in on red teaming pays practical, competitive dividends It’s encouraging to see competitive intensity in red teaming growing among AI companies. When Anthropic released its AI red team guidelines in June of last year, it joined AI providers including Google, Microsoft, Nvidia, OpenAI, and even the U.S.’s National Institute of Standards and Technology (NIST), which all had released red teaming frameworks. Investing heavily in red teaming yields tangible benefits for security leaders in any organization. OpenAI’s paper on external red teaming provides a detailed analysis of how the company strives to create specialized external teams that include cybersecurity and subject matter experts. The goal is to see if knowledgeable external teams can defeat models’ security perimeters and find gaps in their security, biases and controls that prompt-based testing couldn’t find. What makes OpenAI’s recent papers noteworthy is how well they define using human-in-the-middle