Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash

Stay Ahead, Stay ONMINE

Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash

Speculative decoding addresses the fundamental challenge of slow, sequential token generation in language model inference. DFlash speculative decoding support for CPUs was recently enabled in vLLM v0.25.0. In our testing with Qwen3.5-9B on an r8i AWS instance, powered by Intel® Xeon® 6 processors with Performance-cores, DFlash increased average token generation throughput to 3.92x that of the autoregressive baseline at concurrency 1—a 74% cost reduction per generated token.This post explains what speculative decoding is, and how DFlash in particular allows you to speed up your AI workloads on CPU. Our configurations are detailed at the end of this post if you’d like to follow along and reproduce the results. (Full disclosure: the author is affiliated with Intel.)What is Speculative Decoding?A typical autoregressive decoder produces one token, appends it to the context, and runs again to predict the next token. No kernel optimization can parallelize across a dependency that has not been resolved yet, so a 500-token response requires about 500 dependent cycles. Each token generation step requires the transfer of the entire model’s parameters from memory to the processor’s compute units. This makes the operation memory bound and leaves those compute units mostly idle at low concurrency. Speculative decoding works around that serial dependency without changing the model’s output distribution. A lightweight draft model proposes several future tokens. The larger target model checks all of them in a single pass, accepts the longest valid prefix, and corrects the first miss, and—if every token is accepted—generates one bonus token. When the draft proposals are high-quality and fast, each expensive target pass commits several tokens instead of one. Unlike lossy optimization techniques such as quantization, speculative decoding is a lossless acceleration since rejection sampling recovers the target distribution.Figure 1 Ordinary decoding pays for one target pass per token. Speculative decoding spends a cheap draft pass, verifies candidates together, and commits only the target-approved prefix plus a correction or bonus token. Image by author.This technique is especially relevant to inference on Xeon. At small batch sizes, decode often spends much of its time moving model weights from memory for very little work per weight. Verification turns per-token matrix-vector operations that lean on Intel® Advanced Vector Extensions 512 (Intel® AVX-512) into matrix-matrix operations that Intel® Advanced Matrix Extensions (Intel® AMX) accelerates: the target weights, once loaded, are reused across several candidate positions. The catch is that speculation adds a drafter and widens verification. It only pays off when the accepted work exceeds that overhead: we’re trading spare compute for saved memory bandwidth; when there is no spare compute, the trade becomes a loss.DFlash: Block Diffusion for Drafting Plus Target KV InjectionDFlash is a novel speculative decoding method developed by Z Lab. Instead of generating draft tokens serially as in some older algorithms, the DFlash speculator predicts a block in one pass with a small block-diffusion drafter. It also injects hidden features from the target model into the draft model’s KV cache, which improves draft quality without requiring the drafter to reconstruct the full context by itself.Figure 2 DFlash inference design borrowed from [1]. Hidden context features from the target model are fused and injected into each draft layer’s KV cache.The acceptance rate depends on your model and is typically higher for more structured prompts such as code or math compared to conversation. If the target model is domain-specific, the acceptance rate drops on unrelated prompts. You should not expect to see much performance improvement, if any, when benchmarking a coder model on a conversational dataset.The drafter predicts all masked positions in one forward pass, with bidirectional attention inside the block. The number of masked positions—num_speculative_tokens in the command below—is a server parameter, and its optimal value depends on several factors, including the target and draft models, the benchmark dataset, and maximum concurrency as well as your hardware.Let’s see how to boost a model’s performance with DFlash using vLLM with an additional configuration flag:docker run –rm –name vllm-cpu-server –network host –ipc host –security-opt seccomp=unconfined –cap-add SYS_NICE -e VLLM_TARGET_DEVICE=cpu -e VLLM_CPU_KVCACHE_SPACE=40 -v ~/.cache/huggingface:/root/.cache/huggingface vllm/vllm-openai-cpu:latest Qwen/Qwen3.5-9B –dtype bfloat16 –trust-remote-code –speculative-config ‘{“method”: “dflash”, “model”: “z-lab/Qwen3.5-9B-DFlash”, “num_speculative_tokens”: 15}’The effect is easy to see in a streamed response. With the same prompt and deterministic decoding settings, DFlash completes the response substantially faster while producing the same output.Figure 3 Qwen3.5-9B generating the same response using autoregressive decoding (left) and DFlash with 15 speculative tokens (right) on an r8i.16xlarge instance powered by Intel® Xeon® 6 processors. This single-prompt comparison is illustrative; Table 1 reports results across broader benchmark datasets. Image by author.Some recent models such as Muse Glimmer ship with a drafter model. If that’s not the case for your use case, start with Z Lab’s DFlash collection to explore drafters for popular models. For additional options, see Red Hat AI’s model collection, which includes models trained with the Speculator library. Below are the speedup results for Qwen3.5-9B at concurrency 1 and output length 128 compared to the autoregressive baseline.[i] Token generation cost reduction is measured at fixed concurrency on the same instance, so it does not depend on the hourly rate. The average is taken across three datasets for programming (HumanEval), mathematics (GSM8K), and multi-turn conversational questions (MT-Bench).DatasetBaseline tok/sDFlash tok/sDFlash SpeedupAcceptance LengthCost ReductionGSM8K9.9541.424.16×7.0875.98%HumanEval9.8839.413.99×6.9374.93%MT-Bench9.9535.893.61×6.0172.28%Average9.9338.913.92×6.6774.40%Understanding Speculative Decoding MetricsvLLM reports several statistics that describe how effectively the target model accepts DFlash’s proposals. Consider the HumanEval run from Table 1, with num_speculative_tokens set to 15:Acceptance rate (%): 39.56Acceptance length: 6.93Drafts: 381Draft tokens: 5715Accepted tokens: 2261Drafts is the number of speculative blocks verified by the target model: 381 verification steps carried this run’s 2560 output tokens, which the autoregressive baseline would have needed about 2560 sequential passes to generate. Because each of the 381 blocks contained 15 proposals, there are 15 x 381 = 5715 draft tokens. Of those proposals, the target accepted 2261, producing the reported acceptance rate 2261/5715 = 39.56%. Finally, each round (or block) produced an average of 2261/381 = 5.93 accepted draft tokens. Verification also emits one target token, either as a correction, or as a bonus when the block is fully accepted. The resulting acceptance length is therefore 5.93 + 1 = 6.93.A word of caution is in order. For fixed-length drafting, as used by DFlash, the overall acceptance rate is the average of the per-position values below, so it typically falls as the block grows longer, even when throughput improves. Use acceptance length rather than acceptance rate to compare progress per verification round, but choose the block size from measured throughput or latency. Also, speculative decoding counters can include tokens accepted in a final block but discarded when a request reaches its output length limit, so the token count implied by these counters may exceed the number returned to clients.The per-position statistics show how far proposals typically survive into the block:Position 0: 85.83%…Position 4: 51.44%…Position 14: 11.81%These are survival (or acceptance) probabilities. Position n being accepted means the target accepted the complete speculative prefix through position n, so the column is decreasing by construction and sums to the accepted draft tokens per round (593.44% = 5.93). Dividing consecutive values gives the conditional acceptance rate at each position. A collapsing conditional rate indicates that later proposals are increasingly unlikely to survive.An average acceptance length of 6.93 = 5.93 + 1 does not imply that the block should be shortened to six tokens as the distribution is skewed: 14% of blocks have every proposal rejected and 12% are accepted in full. Doing so would eliminate all the rounds in which the target could accept longer prefixes. Also, we cannot increase the block size indefinitely because the rejected tail positions still consume drafting and verification resources. The optimal block size balances useful progress against total round cost:Both terms in the numerator generally grow with the block size while the denominator saturates as per-position survival probabilities typically approach zero, so the achievable gain is bounded.Acceptance statistics help explain performance, but they do not determine the optimum by themselves. To tune num_speculative_tokens, benchmark several block sizes on the intended model, dataset, hardware, and concurrency, then select the value producing the best throughput or latency.Key TakeawaysSpeculative decoding does not make an autoregressive dependency disappear; it moves uncertain future work into a cheaper parallel proposal path and lets the target validate several positions at once, with rejection sampling ensuring a bad proposal costs time, not output quality.DFlash improves the proposal path in two complementary ways: Block diffusion replaces several serial draft invocations with one parallel block pass, and KV injection gives every draft layer direct access to the target’s contextual representation, raising acceptance without turning the drafter into another large language model. That combination is a natural fit for low-batch CPU inference, where target weight movement dominates and wider verification improves weight reuse.The payoff compounds in agentic workloads, where a model is called repeatedly across multi-step loops and every decode step counts which is why speculative decoding underpins Intel’s open source agentic stacks like the Intel® AI for Enterprise Agent Toolkit and Intel® AI SuperClaw.Work on parallel drafting is advancing rapidly beyond DFlash. DSpark adds a semi-autoregressive correction stage and variable-length verification, while DFlash 2 uses a lightweight path selector to trace a coherent path through each position’s top candidates and local convolutions to reduce draft-accuracy decay near the end of each block. Keep an eye on newer speculative decoding methods as support reaches CPU inference frameworks.AcknowledgmentsThe author would like to thank Alex Sin, Eric Petit, Eze Lanza, and Pradeep Surabhi for their review and feedback on this post.ReferencesNotices and DisclaimersPerformance varies by use, configuration, and other factors. Learn more at www.Intel.com/PerformanceIndex.Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure.Your costs and results may vary.Intel technologies may require enabled hardware, software, or service activation.© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.[i] Configurations: 1-node, Amazon EC2 r8i.16xlarge, 1x Intel(R) Xeon(R) 6975P-C, 32 cores, Unknown TDP, HT On, Turbo On, Total Memory 512GB (1x512GB DDR5 7200MT/s [Unknown]), BIOS 1.0, microcode 0x1000434, 1x Elastic Network Adapter, 1x 400G Amazon Elastic Block Store, Ubuntu 24.04.4 LTS, 7.0.0-1009-aws, vLLM 0.26.1rc1.dev124+gb88916617. Test by Intel as of July 2026.

This post explains what speculative decoding is, and how DFlash in particular allows you to speed up your AI workloads on CPU. Our configurations are detailed at the end of this post if you’d like to follow along and reproduce the results. (Full disclosure: the author is affiliated with Intel.)

What is Speculative Decoding?

A typical autoregressive decoder produces one token, appends it to the context, and runs again to predict the next token. No kernel optimization can parallelize across a dependency that has not been resolved yet, so a 500-token response requires about 500 dependent cycles. Each token generation step requires the transfer of the entire model’s parameters from memory to the processor’s compute units. This makes the operation memory bound and leaves those compute units mostly idle at low concurrency. Speculative decoding works around that serial dependency without changing the model’s output distribution. A lightweight draft model proposes several future tokens. The larger target model checks all of them in a single pass, accepts the longest valid prefix, and corrects the first miss, and—if every token is accepted—generates one bonus token. When the draft proposals are high-quality and fast, each expensive target pass commits several tokens instead of one. Unlike lossy optimization techniques such as quantization, speculative decoding is a lossless acceleration since rejection sampling recovers the target distribution.

Ordinary decoding uses one serial target pass per token. Speculative decoding drafts several candidates, verifies them in one target pass, accepts a prefix, and corrects the first rejected token. — Figure 1 Ordinary decoding pays for one target pass per token. Speculative decoding spends a cheap draft pass, verifies candidates together, and commits only the target-approved prefix plus a correction or bonus token. Image by author.

This technique is especially relevant to inference on Xeon. At small batch sizes, decode often spends much of its time moving model weights from memory for very little work per weight. Verification turns per-token matrix-vector operations that lean on Intel® Advanced Vector Extensions 512 (Intel® AVX-512) into matrix-matrix operations that Intel® Advanced Matrix Extensions (Intel® AMX) accelerates: the target weights, once loaded, are reused across several candidate positions. The catch is that speculation adds a drafter and widens verification. It only pays off when the accepted work exceeds that overhead: we’re trading spare compute for saved memory bandwidth; when there is no spare compute, the trade becomes a loss.

DFlash: Block Diffusion for Drafting Plus Target KV Injection

DFlash is a novel speculative decoding method developed by Z Lab. Instead of generating draft tokens serially as in some older algorithms, the DFlash speculator predicts a block in one pass with a small block-diffusion drafter. It also injects hidden features from the target model into the draft model’s KV cache, which improves draft quality without requiring the drafter to reconstruct the full context by itself.

DFlash system — *Figure 2 DFlash inference design borrowed from [1]. Hidden context features from the target model are fused and injected into each draft layer’s KV cache.*

The acceptance rate depends on your model and is typically higher for more structured prompts such as code or math compared to conversation. If the target model is domain-specific, the acceptance rate drops on unrelated prompts. You should not expect to see much performance improvement, if any, when benchmarking a coder model on a conversational dataset.

The drafter predicts all masked positions in one forward pass, with bidirectional attention inside the block. The number of masked positions—num_speculative_tokens in the command below—is a server parameter, and its optimal value depends on several factors, including the target and draft models, the benchmark dataset, and maximum concurrency as well as your hardware.

Let’s see how to boost a model’s performance with DFlash using vLLM with an additional configuration flag:

docker run --rm   --name vllm-cpu-server   --network host --ipc host --security-opt seccomp=unconfined --cap-add SYS_NICE   -e VLLM_TARGET_DEVICE=cpu   -e VLLM_CPU_KVCACHE_SPACE=40   -v ~/.cache/huggingface:/root/.cache/huggingface   vllm/vllm-openai-cpu:latest   Qwen/Qwen3.5-9B   --dtype bfloat16   --trust-remote-code   --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.5-9B-DFlash", "num_speculative_tokens": 15}'

The effect is easy to see in a streamed response. With the same prompt and deterministic decoding settings, DFlash completes the response substantially faster while producing the same output.

DFlash compared with autoregressive baseline on the same machine. — Figure 3 Qwen3.5-9B generating the same response using autoregressive decoding (left) and DFlash with 15 speculative tokens (right) on an r8i.16xlarge instance powered by Intel® Xeon® 6 processors. This single-prompt comparison is illustrative; Table 1 reports results across broader benchmark datasets. Image by author.

Some recent models such as Muse Glimmer ship with a drafter model. If that’s not the case for your use case, start with Z Lab’s DFlash collection to explore drafters for popular models. For additional options, see Red Hat AI’s model collection, which includes models trained with the Speculator library. Below are the speedup results for Qwen3.5-9B at concurrency 1 and output length 128 compared to the autoregressive baseline.[i] Token generation cost reduction is measured at fixed concurrency on the same instance, so it does not depend on the hourly rate. The average is taken across three datasets for programming (HumanEval), mathematics (GSM8K), and multi-turn conversational questions (MT-Bench).

Dataset	Baseline tok/s	DFlash tok/s	DFlash Speedup	Acceptance Length	Cost Reduction
GSM8K	9.95	41.42	4.16x	7.08	75.98%
HumanEval	9.88	39.41	3.99x	6.93	74.93%
MT-Bench	9.95	35.89	3.61x	6.01	72.28%
Average	9.93	38.91	3.92x	6.67	74.40%

Understanding Speculative Decoding Metrics

vLLM reports several statistics that describe how effectively the target model accepts DFlash’s proposals. Consider the HumanEval run from Table 1, with num_speculative_tokens set to 15:

Acceptance rate (%): 39.56Acceptance length:   6.93Drafts:              381Draft tokens:        5715Accepted tokens:     2261

Drafts is the number of speculative blocks verified by the target model: 381 verification steps carried this run’s 2560 output tokens, which the autoregressive baseline would have needed about 2560 sequential passes to generate. Because each of the 381 blocks contained 15 proposals, there are 15 x 381 = 5715 draft tokens. Of those proposals, the target accepted 2261, producing the reported acceptance rate 2261/5715 = 39.56%. Finally, each round (or block) produced an average of 2261/381 = 5.93 accepted draft tokens. Verification also emits one target token, either as a correction, or as a bonus when the block is fully accepted. The resulting acceptance length is therefore 5.93 + 1 = 6.93.

A word of caution is in order. For fixed-length drafting, as used by DFlash, the overall acceptance rate is the average of the per-position values below, so it typically falls as the block grows longer, even when throughput improves. Use acceptance length rather than acceptance rate to compare progress per verification round, but choose the block size from measured throughput or latency. Also, speculative decoding counters can include tokens accepted in a final block but discarded when a request reaches its output length limit, so the token count implied by these counters may exceed the number returned to clients.

The per-position statistics show how far proposals typically survive into the block:

Position 0:  85.83%…Position 4:  51.44%…Position 14: 11.81%

These are survival (or acceptance) probabilities. Position n being accepted means the target accepted the complete speculative prefix through position n, so the column is decreasing by construction and sums to the accepted draft tokens per round (593.44% = 5.93). Dividing consecutive values gives the conditional acceptance rate at each position. A collapsing conditional rate indicates that later proposals are increasingly unlikely to survive.

An average acceptance length of 6.93 = 5.93 + 1 does not imply that the block should be shortened to six tokens as the distribution is skewed: 14% of blocks have every proposal rejected and 12% are accepted in full. Doing so would eliminate all the rounds in which the target could accept longer prefixes. Also, we cannot increase the block size indefinitely because the rejected tail positions still consume drafting and verification resources. The optimal block size balances useful progress against total round cost:

Both terms in the numerator generally grow with the block size while the denominator saturates as per-position survival probabilities typically approach zero, so the achievable gain is bounded.

Acceptance statistics help explain performance, but they do not determine the optimum by themselves. To tune num_speculative_tokens, benchmark several block sizes on the intended model, dataset, hardware, and concurrency, then select the value producing the best throughput or latency.

Key Takeaways

Speculative decoding does not make an autoregressive dependency disappear; it moves uncertain future work into a cheaper parallel proposal path and lets the target validate several positions at once, with rejection sampling ensuring a bad proposal costs time, not output quality.
DFlash improves the proposal path in two complementary ways: Block diffusion replaces several serial draft invocations with one parallel block pass, and KV injection gives every draft layer direct access to the target’s contextual representation, raising acceptance without turning the drafter into another large language model. That combination is a natural fit for low-batch CPU inference, where target weight movement dominates and wider verification improves weight reuse.

The payoff compounds in agentic workloads, where a model is called repeatedly across multi-step loops and every decode step counts which is why speculative decoding underpins Intel’s open source agentic stacks like the Intel® AI for Enterprise Agent Toolkit and Intel® AI SuperClaw.

Work on parallel drafting is advancing rapidly beyond DFlash. DSpark adds a semi-autoregressive correction stage and variable-length verification, while DFlash 2 uses a lightweight path selector to trace a coherent path through each position’s top candidates and local convolutions to reduce draft-accuracy decay near the end of each block. Keep an eye on newer speculative decoding methods as support reaches CPU inference frameworks.

Acknowledgments

The author would like to thank Alex Sin, Eric Petit, Eze Lanza, and Pradeep Surabhi for their review and feedback on this post.

References

Notices and Disclaimers

Performance varies by use, configuration, and other factors. Learn more at www.Intel.com/PerformanceIndex.
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure.
Your costs and results may vary.
Intel technologies may require enabled hardware, software, or service activation.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

[i] Configurations: 1-node, Amazon EC2 r8i.16xlarge, 1x Intel(R) Xeon(R) 6975P-C, 32 cores, Unknown TDP, HT On, Turbo On, Total Memory 512GB (1x512GB DDR5 7200MT/s [Unknown]), BIOS 1.0, microcode 0x1000434, 1x Elastic Network Adapter, 1x 400G Amazon Elastic Block Store, Ubuntu 24.04.4 LTS, 7.0.0-1009-aws, vLLM 0.26.1rc1.dev124+gb88916617. Test by Intel as of July 2026.

Stay Ahead

Explore More Insights

Stay ahead with more perspectives on cutting-edge power, infrastructure, energy, bitcoin and AI solutions. Explore these articles to uncover strategies and insights shaping the future of industries.

AI agents are reshaping the threat landscape, shifting IT skills requirements

The AI Workforce Consortium is focused on studying the impact of AI on the workforce. Members include Accenture, Cisco, Cornerstone, Eightfold AI, Google, IBM, Indeed, Intel, Microsoft, Pearson, and SAP, in partnership with global advisors such as Digitaleurope and the European Vocational Training Association. The consortium’s report found that between

Palo Alto Networks and NTT Data target $1B in joint AI security sales with new managed offerings

“Backed by joint investments, more than 2,000 certified experts, as well as dedicated forward deployed engineers, the alliance will deliver a seamless approach to streamline deployments and speed client outcomes. Through direct engineering collaboration, NTT Data will gain early access to new platform features — further accelerating the delivery of

Network architecture pay climbs amid AI shift

Network architects today must make decisions that involve multiple technologies. From topology to WAN and SD-WAN to segmentation and security, the scope of networking skills continues to evolve. Those networking decisions become more complicated as enterprises incorporate AI workloads, edge computing, Wi-Fi 7, private 5G, and more. AI and automation

Golar contracts CIMC Raffles for fourth FLNG

Golar LNG Ltd. last week executed an engineering, procurement, and construction (EPC) contract with Yantai CIMC Raffles Offshore Ltd. (CIMC Raffles) for its fourth floating LNG (FLNG) and second MKII design FLNG vessel with an annual liquefaction capacity of 3.5 million tonnes/year (tpy). Golar’s fourth FLNG is expected to be

Energy Department Announces $500 Million Award to Revitalize American Steelmaking

WASHINGTON—The U.S. Department of Energy (DOE) today announced a $500 million award to support a $1 billion investment at Cleveland-Cliffs’ Middletown Works facility in Middletown, Ohio. Vice President JD Vance and U.S. Energy Secretary Chris Wright visited Middletown Works today to highlight the Trump Administration’s commitment to American steelworkers and the resurgence of American manufacturing. The investment will modernize American steelmaking, protect 2,300 American jobs, and strengthen the domestic steel supply chain. The project advances President Trump’s commitment to put American workers first, bring investment back to American communities, and strengthen the industries critical to America’s economic and national security. Cleveland-Cliffs determined that the business case for the original project scope no longer made sense given customers’ unwillingness to pay a “green premium” for steel. Working with DOE, Cleveland-Cliffs identified a viable alternative that will upgrade and improve the efficiency of the existing coal-fired blast furnace while also capturing and commercializing co-product blast furnace gas (BFG). “President Trump is rebuilding America’s industrial base,” said Secretary Wright. “This investment puts American workers and American manufacturing first. It will modernize one of our nation’s critical steelmaking facilities, protect thousands of jobs, and strengthen our domestic steel production—keeping Ohio at the heart of American manufacturing and strengthening our national security.” The investment will modernize critical steelmaking operations at Middletown Works by rebuilding and upgrading the plant’s main coal-fired ironmaking furnace, deploying AI to optimize furnace operations and improve energy efficiency, and building an on-site facility to convert steel mill process gases into electricity. Follow-on investments will turn industrial byproducts into materials for concrete used in regional infrastructure. “This landmark investment at Middletown Works will secure a reliable domestic supply of high-purity steel while protecting thousands of quality jobs in Ohio,” said Assistant Secretary of Energy Audrey Robertson. “DOE is proud to partner with Cleveland-Cliffs to reduce America’s dependence on foreign products

Energy Secretary Keeps Critical Generation Available in Mid-Atlantic

WASHINGTON—U.S. Secretary of Energy Chris Wright today issued an emergency order to address critical grid reliability issues facing the Mid-Atlantic region of the United States. The emergency order directs PJM Interconnection L.L.C. (PJM), in coordination with Constellation Energy Corporation, to ensure Units 3 and 4 of the Eddystone Generating Station in Pennsylvania remain available to operate and to employ economic dispatch to minimize costs for the American people. The units were originally slated to shut down on May 31, 2025. “The energy sources that perform when you need them most are the most valuable,” Secretary Wright said. “During recent Mid-Atlantic heat waves, coal, natural gas, and nuclear kept the lights and air conditioners on. President Trump and the Energy Department are committed to keeping critical generation available when demand is highest, reducing the risk of blackouts and ensuring Americans have affordable, reliable, and secure power—regardless of whether the wind is blowing or the sun is shining.” As outlined in DOE’s Resource Adequacy Report, power outages could increase by 100 times in 2030 if the U.S. continues to take reliable power offline. This order is in effect beginning on August 23, 2026, through November 20, 2026. ###

Energy Department Announces $500 Million to Secure America’s Critical Mineral and Battery Supply Chains

WASHINGTON—The U.S. Department of Energy’s (DOE) Office of Critical Minerals and Energy Innovation (CMEI) today announced $500 million for seven selected projects to expand critical mineral and material processing, battery manufacturing, and recycling capacity in the United States. In accordance with President Trump’s Executive Order, Unleashing American Energy, the selected projects advance the President’s agenda to strengthen America’s domestic critical minerals and materials supply chains, reduce reliance on foreign sources, bolster national security, and advance American energy dominance. “For too long, America has depended on foreign actors for critical materials essential to modern life that underpin our economy, energy security, and national security,” said U.S. Secretary of Energy Chris Wright. “President Trump is reversing that dependence by securing our critical supply chains, unleashing American industry, and bringing critical materials production and processing back to the United States.” “DOE is taking decisive action to secure the critical supply chains necessary to power our nation,” said Assistant Secretary of Energy Audrey Robertson. “These projects underscore DOE’s commitment to driving innovation, reducing reliance on foreign sources, and promoting American energy dominance.” This is the third round of funding from DOE’s Battery Materials Processing and Battery Manufacturing and Recycling programs, which support battery materials processing, recycling, and manufacturing projects. These include demonstration projects, construction of commercial-scale facilities, and retrofitting or retooling existing facilities. Critical minerals and materials are essential to American industry, energy production, and national security. Expanding domestic capacity will help ensure the resources America needs are processed, manufactured, and recycled in the United States. Information on the selected projects is available here and here.

bp lets Shah Deniz compression automation contract

bp has let a contract to Emerson to deliver automation technologies for the Shah Deniz Compression project offshore Azerbaijan. Emerson will provide integrated control and safety systems aimed at enhancing production, safety, and reliability on the new offshore compression platform. The contract includes systems to provide process control, safety shutdown, fire and gas detection, and power management. Together, these systems deliver real-time visibility and remote control of critical operations, Emerson said. The $2.9 billion Shah Deniz Compression project, which includes an electrically powered, normally unattended offshore production platform, is a next stage development of the Caspian Sea Shah Deniz field. Designed to access low-pressure gas reserves and maximize overall recovery, the platform will be equipped with four 11 Mw compressors and serve as the central compression hub for gas from the Shah Deniz Alpha and Bravo platforms. The platform will operate remotely from bp’s onshore Sangachal terminal 55 km south of Baku. The project is expected to enable about 50 billion cu m of additional gas and about 25 million bbl of condensate production and export. Construction is scheduled to be completed in 2029, with first gas compression expected from the Shah Deniz Alpha platform in 2029 and from the Shah Deniz Bravo platform in 2030. The agreement follows a previous automation contract bp signed with Emerson for the Azeri Central East and Shah Deniz Stage 2 developments. bp is operator at Shah Deniz (29.99%) with partners Lukoil (19.99%), TPAO (19%), Cenub Qaz Dehlizi (16.02%), NICO (10%), and MVM (5%).

Federal court voids Texas GulfLink license over agency’s ‘serious procedural errors’

The ruling voids the license, halting all construction or progress. Sentinel Midstream declined comment on the ruling and would not answer questions about the status of construction. GulfLink, sited about 30 miles offshore Freeport, Tex., is designed to export up to 1 million b/d via Very Large Crude Carriers (VLCCs) to the government of Japan and Freeport Commodities. The project involves a 44-mile, 42-in. OD pipeline and was scheduled to begin operations around 2028. The estimated $2.1 billion investment was funded as part of a broader trade agreement between the US and Japan. The legal battle stems from a specific rule in the Deepwater Port Act of 1974 that dictates that the federal government can only permit one crude oil deepwater port, including any supporting infrastructure, within a single designated “application area.” Because the competing SPOT project’s pipeline route physically overlaps and intersects GulfLink’s lines, the plaintiff—Citizens for Clean Air & Clean Water in Brazoria County (Better Brazoria), represented by Earthjustice—successfully argued that MARAD violated the “one port” rule when issuing GulfLink’s license in February. The three-judge panel found that MARAD “improperly drew” the map designing the project’s official boundaries to exclude the pipelines and approved two overlapping projects in the same zone instead of only licensing one. The court wrote that the scope of the error made vacatur, not the less serious remand without vacatur, the appropriate remedy. Vacatur deems the license invalid and is used when the court finds “serious procedural errors” that cannot be easily explained or fixed with minor changes. Remand without vacatur sends the decision back to the agency for corrections but leaves the current license in place in the meantime. SPOT project status The $2.5-3-billion SPOT project, developed by Enterprise Products Partners in partnership with Enbridge Inc., also lies about 30 miles from Freeport. Designed to handle VLCCs,

IEA: Emergency reserve withdrawals slow

The International Energy Agency (IEA) member countries continued to release emergency oil stocks in July, but the pace of withdrawals slowed sharply as crude supply availability improved in parts of the Asia Pacific and the market faced increasing product tightness. IEA countries released 26 million bbl of emergency stocks in July, bringing cumulative releases to 300 million bbl since the agency announced a coordinated 400-million bbl action on Mar. 11. Government stock draws averaged 750,000 b/d in July, down from 1.5 million b/d in June and 2.5 million b/d in May. The slowdown was particularly pronounced among IEA members in Asia Oceania, which released 4 million bbl from emergency stocks in July, compared with 8 million bbl in June and 44 million bbl in May. The decline reflected improved crude oil supply availability in Japan and Korea. The US also reduced the pace of emergency stock releases. Withdrawals from the Strategic Petroleum Reserve totaled 17 million bbl in July, roughly half the volume released in June. More than 100 million bbl of the emergency stocks committed under the IEA’s 400-million bbl coordinated action has yet to reach the market. The timing of the remaining releases will depend on market developments and broader oil supply security considerations in coming months, according to the agency. Most of the remaining emergency stocks consist of crude oil, however, limiting their ability to ease increasingly tight oil product markets, IEA said. At the same time, global observed oil inventories fell sharply in July amid severely constrained shipping through the Strait of Hormuz. Stocks declined by 69 million bbl, equivalent to 2.2 million b/d, with oil on water accounting for more than 90% of the decline. Oil on water fell by 63 million bbl, or about 2 million b/d, reflecting higher arrivals and lower exports amid

PJM’s New Data Center Power Equation

PJM Interconnection has now filed one of the most consequential proposed changes yet in the relationship between data centers and the electric grid. Rather than simply treating a new hyperscale or AI facility like any other customer whose demand will be backed through regional capacity procurement, PJM is proposing a framework under which the largest new loads would need to be supported by new capacity, have their needs covered through the Reliability Backstop Procurement, or face potential curtailment when the regional power system is short of supply. The approach has been developing since PJM launched its Critical Issue Fast Path process for large loads in 2025, but it became substantially more concrete in late July and August 2026. PJM filed its proposed Reliability Backstop Procurement with FERC on July 31 and began accepting applications that day for its FERC-approved Expedited Interconnection Track. On Aug. 13, PJM filed its proposed Interim Resource Adequacy Service, or IRAS, along with the Large Load Registry that would support it. The immediate numbers explain the urgency. PJM’s July 2026 capacity auction for the 2028/2029 delivery year procured 138,318 MW of unforced capacity through the centralized auction. Even after including Fixed Resource Requirement resources, however, PJM came up 6,831 MW short of its reliability requirement. The auction cleared at the FERC-approved $325/MW-day price cap. It was the second consecutive auction in which the PJM region failed to procure its full reliability requirement, something that had not happened before these two auctions. That gap is occurring while demand continues to accelerate. PJM’s 2026 long-term forecast projects summer peak demand growing at an average 3.6% annually over the next decade, compared with just 0.3% in the comparable forecast issued in 2021. Summer peak demand is projected to rise by nearly 66 GW over 10 years. Data centers are

Zayo, NVIDIA Build the Long-Haul Backbone for Distributed AI

The data center industry’s increasingly power-first approach to site selection has created a follow-on question: Once the megawatts are found, is there enough network infrastructure to make the site useful at AI scale? Zayo and NVIDIA are putting real infrastructure behind that question. Zayo said it is working with NVIDIA to expand network capacity supporting AI factories across North America, including an 8,000-route-mile program targeting some of the fastest-growing AI corridors in the United States. The project encompasses six new long-haul routes along with overbuilds of existing network across 10 high-demand corridors. The announcement arrives as AI data center development moves beyond the largest established hubs toward markets where power and land may be more readily available, but fiber capacity cannot necessarily be taken for granted. That geography is increasingly important. NVIDIA has separately developed “scale-across” networking technology designed to allow AI infrastructure distributed among different buildings — or even data centers separated by hundreds of kilometers — to operate as a more unified computing environment. Put together, the developments suggest that networking is becoming inseparable from the AI factory buildout itself. Power may determine where the next generation of AI infrastructure can be built. Fiber will increasingly determine how effectively those sites can participate in the larger AI ecosystem. Fiber Follows the Power Zayo CEO Steve Smith said AI demand is changing both where network infrastructure is needed and how aggressively capacity must be deployed ahead of development. “AI is fundamentally reshaping where and how network infrastructure needs to be built across the U.S.,” Smith said. The company’s 8,000-mile program is more nuanced than that top-line number might suggest. Zayo disclosed in April that the expansion includes approximately 3,000 route miles across six new long-haul routes, plus more than 5,000 route miles of overbuilds across 10 existing corridors. Zayo

Southern’s 17 GW Pipeline Puts AI Power Demand Into Utility Math

The headline number from Southern Company’s latest earnings report is hard to miss: electricity use by data centers across the utility’s system increased 55% in the second quarter compared with a year earlier. But the more consequential numbers may be the ones sitting behind it. Southern now has more than 1.2 GW of operating data center load, up by more than 500 MW from a year ago. At the same time, its electric utilities have signed contracts and large-load agreements totaling more than 17 GW by the mid-2030s, with another 8 GW in late-stage development and a prospective pipeline of large industrial and data center projects exceeding 75 GW. That leaves an enormous gap between the data center megawatts consuming electricity today and the load Southern has contractually positioned itself to serve during the next decade. For the data center industry, that gap may be the most important part of Southern’s second-quarter story. It offers a look at how utilities are beginning to convert the AI infrastructure boom from forecasts and campus announcements into contracts, generation procurement, transmission investment and eventually energized capacity. From Contracts to Megawatts Southern added roughly 6 GW of contracted large load during the quarter alone. Alabama Power signed three projects representing about 3 GW, while Georgia Power reached a 25-year agreement to serve OpenAI’s planned project in Effingham County near Savannah. That facility is expected to require approximately 3.2 GW and begin taking electric service in phases in 2028. The numbers nevertheless require an important distinction. Seventeen gigawatts contracted does not mean 17 GW will suddenly appear on Southern’s grid. Large data center campuses ramp gradually, often over several years, and Southern executives acknowledged that actual customer ramp schedules do not always match the assumptions made when projects are first approved. CEO Chris Womack said

PORTS-Pike Takes Shape as an 8-GW AI Infrastructure Model

Back on March 31, 2026, we discussed we discussed SoftBank and SB Energy’s plans to redevelop the former Portsmouth Gaseous Diffusion Plant site near Piketon as a 10-GW artificial intelligence data center campus supported by almost an equal amount of new power generation. At the time, the plan called for as much as 10 GW of new generation, including 9.2 GW of natural gas capacity, along with approximately $4.2 billion of high-voltage transmission infrastructure developed with AEP Ohio. An initial 800-MW data center phase was targeted for service in 2028. The March story was notable because Pike County appeared to offer a preview of a new model for building hyperscale infrastructure: develop the generation, transmission and data center simultaneously rather than wait for an increasingly congested regional grid to deliver multiple gigawatts of capacity. Not to mention the reuse of a brownfield site with the encouragement of the federal government. Since then, almost every important part of the project has moved forward, and on August 17, the most consequential missing pieces fell into place. NVIDIA announced that it will become the exclusive AI compute infrastructure provider for the PORTS-Pike Technology Campus. OpenAI will be the data center customer, signing a 20-year lease with SB Energy for approximately 8 GW of IT capacity. NVIDIA will invest another $1.5 billion in SB Energy and provide credit support for the land, power and shell infrastructure behind an initial 4.25 GW of IT load, with an option covering approximately another 3.75 GW. The Securities and Exchange Commission filing accompanying the announcement makes the financial commitment even more significant. NVIDIA disclosed that its aggregate payment obligation associated with its initial commitment is capped at $105 billion. That is not a conventional capital commitment to spend $105 billion building the campus, nor is it simply a

Nvidia scales back financing guarantee for OpenAI data center

Nvidia is scaling back a proposed financial guarantee tied to a massive OpenAI data center project in Ohio, reducing its initial commitment from as much as $250 billion to less than $120 billion, according to report in the Wall Street Journal. Earlier this month, Nvidia announced partnerships with major financial firms including Apollo Global Management, BlackRock, Blackstone, Brookfield Asset Management, Goldman Sachs and KKR, aimed at mobilizing more than $500 billion in capital for AI computing infrastructure. The change represents a significant restructuring of Nvidia’s role in financing the planned facility, which is being developed by SB Energy, a subsidiary of SoftBank. Under the revised arrangement, Nvidia would guarantee financing for the project’s first phase, representing roughly 5 gigawatts of capacity, or half of the total proposed capacity. Financing for the remaining capacity would be considered separately at a later stage.

Texas Tightens Oversight of Data Center Development

Texas has spent the past decade building one of the most data center-friendly policy environments in the United States. But the state’s political posture is tightening. The emerging message from Austin is that continued data center growth will face greater scrutiny over grid costs, water use, tax incentives and community impacts. What is interesting about this policy conversation is that the Texas Legislature is not in regular session. The 89th regular session ended June 2, 2025, and the 90th Legislature does not convene until January 12, 2027. What has occurred instead is a concentrated period of interim committee work, gubernatorial recommendations, implementation of Senate Bill 6, calls for a special session, and regulatory action by the Public Utility Commission of Texas and the Electric Reliability Council of Texas. Together, those efforts are creating the framework for a broader legislative debate in 2027 while already affecting projects seeking ERCOT interconnection, infrastructure costs and site-selection decisions. Abbott Sets Out a New Policy Framework The policy shift accelerated June 10, when Gov. Greg Abbott directed the PUCT to require data centers to fully fund the electric infrastructure needed to serve their operations and directed PUCT and ERCOT to identify additional actions available under existing authority. Separately, Abbott pledged to work with lawmakers in 2027 on legislation requiring data centers to add electric capacity, use water-efficient cooling systems, report electricity and water use, phase out outdated tax incentives and adopt additional protections for neighboring communities. The most consequential shift began June 10, when Gov. Greg Abbott sent state electricity regulators a sweeping list of data center policy priorities. Abbott called for future legislation requiring new facilities to add generation to the Texas grid, pay the full cost of their interconnection and related infrastructure, use closed-loop or similarly water-efficient cooling systems, and file annual reports

Microsoft will invest $80B in AI data centers in fiscal 2025

And Microsoft isn’t the only one that is ramping up its investments into AI-enabled data centers. Rival cloud service providers are all investing in either upgrading or opening new data centers to capture a larger chunk of business from developers and users of large language models (LLMs). In a report published in October 2024, Bloomberg Intelligence estimated that demand for generative AI would push Microsoft, AWS, Google, Oracle, Meta, and Apple would between them devote $200 billion to capex in 2025, up from $110 billion in 2023. Microsoft is one of the biggest spenders, followed closely by Google and AWS, Bloomberg Intelligence said. Its estimate of Microsoft’s capital spending on AI, at $62.4 billion for calendar 2025, is lower than Smith’s claim that the company will invest $80 billion in the fiscal year to June 30, 2025. Both figures, though, are way higher than Microsoft’s 2020 capital expenditure of “just” $17.6 billion. The majority of the increased spending is tied to cloud services and the expansion of AI infrastructure needed to provide compute capacity for OpenAI workloads. Separately, last October Amazon CEO Andy Jassy said his company planned total capex spend of $75 billion in 2024 and even more in 2025, with much of it going to AWS, its cloud computing division.

John Deere unveils more autonomous farm machines to address skill labor shortage

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Self-driving tractors might be the path to self-driving cars. John Deere has revealed a new line of autonomous machines and tech across agriculture, construction and commercial landscaping. The Moline, Illinois-based John Deere has been in business for 187 years, yet it’s been a regular as a non-tech company showing off technology at the big tech trade show in Las Vegas and is back at CES 2025 with more autonomous tractors and other vehicles. This is not something we usually cover, but John Deere has a lot of data that is interesting in the big picture of tech. The message from the company is that there aren’t enough skilled farm laborers to do the work that its customers need. It’s been a challenge for most of the last two decades, said Jahmy Hindman, CTO at John Deere, in a briefing. Much of the tech will come this fall and after that. He noted that the average farmer in the U.S. is over 58 and works 12 to 18 hours a day to grow food for us. And he said the American Farm Bureau Federation estimates there are roughly 2.4 million farm jobs that need to be filled annually; and the agricultural work force continues to shrink. (This is my hint to the anti-immigration crowd). John Deere’s autonomous 9RX Tractor. Farmers can oversee it using an app. While each of these industries experiences their own set of challenges, a commonality across all is skilled labor availability. In construction, about 80% percent of contractors struggle to find skilled labor. And in commercial landscaping, 86% of landscaping business owners can’t find labor to fill open positions, he said. “They have to figure out how to do

2025 playbook for enterprise AI success, from agents to evals

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More 2025 is poised to be a pivotal year for enterprise AI. The past year has seen rapid innovation, and this year will see the same. This has made it more critical than ever to revisit your AI strategy to stay competitive and create value for your customers. From scaling AI agents to optimizing costs, here are the five critical areas enterprises should prioritize for their AI strategy this year. 1. Agents: the next generation of automation AI agents are no longer theoretical. In 2025, they’re indispensable tools for enterprises looking to streamline operations and enhance customer interactions. Unlike traditional software, agents powered by large language models (LLMs) can make nuanced decisions, navigate complex multi-step tasks, and integrate seamlessly with tools and APIs. At the start of 2024, agents were not ready for prime time, making frustrating mistakes like hallucinating URLs. They started getting better as frontier large language models themselves improved. “Let me put it this way,” said Sam Witteveen, cofounder of Red Dragon, a company that develops agents for companies, and that recently reviewed the 48 agents it built last year. “Interestingly, the ones that we built at the start of the year, a lot of those worked way better at the end of the year just because the models got better.” Witteveen shared this in the video podcast we filmed to discuss these five big trends in detail. Models are getting better and hallucinating less, and they’re also being trained to do agentic tasks. Another feature that the model providers are researching is a way to use the LLM as a judge, and as models get cheaper (something we’ll cover below), companies can use three or more models to

OpenAI’s red teaming innovations define new essentials for security leaders in the AI era

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More OpenAI has taken a more aggressive approach to red teaming than its AI competitors, demonstrating its security teams’ advanced capabilities in two areas: multi-step reinforcement and external red teaming. OpenAI recently released two papers that set a new competitive standard for improving the quality, reliability and safety of AI models in these two techniques and more. The first paper, “OpenAI’s Approach to External Red Teaming for AI Models and Systems,” reports that specialized teams outside the company have proven effective in uncovering vulnerabilities that might otherwise have made it into a released model because in-house testing techniques may have missed them. In the second paper, “Diverse and Effective Red Teaming with Auto-Generated Rewards and Multi-Step Reinforcement Learning,” OpenAI introduces an automated framework that relies on iterative reinforcement learning to generate a broad spectrum of novel, wide-ranging attacks. Going all-in on red teaming pays practical, competitive dividends It’s encouraging to see competitive intensity in red teaming growing among AI companies. When Anthropic released its AI red team guidelines in June of last year, it joined AI providers including Google, Microsoft, Nvidia, OpenAI, and even the U.S.’s National Institute of Standards and Technology (NIST), which all had released red teaming frameworks. Investing heavily in red teaming yields tangible benefits for security leaders in any organization. OpenAI’s paper on external red teaming provides a detailed analysis of how the company strives to create specialized external teams that include cybersecurity and subject matter experts. The goal is to see if knowledgeable external teams can defeat models’ security perimeters and find gaps in their security, biases and controls that prompt-based testing couldn’t find. What makes OpenAI’s recent papers noteworthy is how well they define using human-in-the-middle

Stay Ahead, Stay ONMINE