Stay Ahead, Stay ONMINE

Linear Regression in Time Series: Sources of Spurious Regression

1. Introduction It’s pretty clear that most of our work will be automated by AI in the future. This will be possible because many researchers and professionals are working hard to make their work available online. These contributions not only help us understand fundamental concepts but also refine AI models, ultimately freeing up time to focus on other activities. However, there is one concept that remains misunderstood, even among experts. It is spurious regression in time series analysis. This issue arises when regression models suggest strong relationships between variables, even when none exist. It is typically observed in time series regression equations that seem to have a high degree of fit — as indicated by a high R² (coefficient of multiple correlation) — but with an extremely low Durbin-Watson statistic (d), signaling strong autocorrelation in the error terms. What is particularly surprising is that almost all econometric textbooks warn about the danger of autocorrelated errors, yet this issue persists in many published papers. Granger and Newbold (1974) identified several examples. For instance, they found published equations with R² = 0.997 and the Durbin-Watson statistic (d) equal to 0.53. The most extreme found is an equation with R² = 0.999 and d = 0.093. It is especially problematic in economics and finance, where many key variables exhibit autocorrelation or serial correlation between adjacent values, particularly if the sampling interval is small, such as a week or a month, leading to misleading conclusions if not handled correctly. For example, today’s GDP is strongly correlated with the GDP of the previous quarter. Our post provides a detailed explanation of the results from Granger and Newbold (1974) and Python simulation (see section 7) replicating the key results presented in their article. Whether you’re an economist, data scientist, or analyst working with time series data, understanding this issue is crucial to ensuring your models produce meaningful results. To walk you through this paper, the next section will introduce the random walk and the ARIMA(0,1,1) process. In section 3, we will explain how Granger and Newbold (1974) describe the emergence of nonsense regressions, with examples illustrated in section 4. Finally, we’ll show how to avoid spurious regressions when working with time series data. 2. Simple presentation of a Random Walk and ARIMA(0,1,1) Process 2.1 Random Walk Let 𝐗ₜ be a time series. We say that 𝐗ₜ follows a random walk if its representation is given by: 𝐗ₜ = 𝐗ₜ₋₁ + 𝜖ₜ. (1) Where 𝜖ₜ is a white noise. It can be written as a sum of white noise, a useful form for simulation. It is a non-stationary time series because its variance depends on the time t. 2.2 ARIMA(0,1,1) Process The ARIMA(0,1,1) process is given by: 𝐗ₜ = 𝐗ₜ₋₁ + 𝜖ₜ − 𝜃 𝜖ₜ₋₁. (2) where 𝜖ₜ is a white noise. The ARIMA(0,1,1) process is non-stationary. It can be written as a sum of an independent random walk and white noise: 𝐗ₜ = 𝐗₀ + random walk + white noise. (3) This form is useful for simulation. Those non-stationary series are often employed as benchmarks against which the forecasting performance of other models is judged. 3. Random walk can lead to Nonsense Regression First, let’s recall the Linear Regression model. The linear regression model is given by: 𝐘 = 𝐗𝛽 + 𝜖. (4) Where 𝐘 is a T × 1 vector of the dependent variable, 𝛽 is a K × 1 vector of the coefficients, 𝐗 is a T × K matrix of the independent variables containing a column of ones and (K−1) columns with T observations on each of the (K−1) independent variables, which are stochastic but distributed independently of the T × 1 vector of the errors 𝜖. It is generally assumed that: 𝐄(𝜖) = 0, (5) and 𝐄(𝜖𝜖′) = 𝜎²𝐈. (6) where 𝐈 is the identity matrix. A test of the contribution of independent variables to the explanation of the dependent variable is the F-test. The null hypothesis of the test is given by: 𝐇₀: 𝛽₁ = 𝛽₂ = ⋯ = 𝛽ₖ₋₁ = 0, (7) And the statistic of the test is given by: 𝐅 = (𝐑² / (𝐊−1)) / ((1−𝐑²) / (𝐓−𝐊)). (8) where 𝐑² is the coefficient of determination. If we want to construct the statistic of the test, let’s assume that the null hypothesis is true, and one tries to fit a regression of the form (Equation 4) to the levels of an economic time series. Suppose next that these series are not stationary or are highly autocorrelated. In such a situation, the test procedure is invalid since 𝐅 in (Equation 8) is not distributed as an F-distribution under the null hypothesis (Equation 7). In fact, under the null hypothesis, the errors or residuals from (Equation 4) are given by: 𝜖ₜ = 𝐘ₜ − 𝐗𝛽₀ ; t = 1, 2, …, T. (9) And will have the same autocorrelation structure as the original series 𝐘. Some idea of the distribution problem can arise in the situation when: 𝐘ₜ = 𝛽₀ + 𝐗ₜ𝛽₁ + 𝜖ₜ. (10) Where 𝐘ₜ and 𝐗ₜ follow independent first-order autoregressive processes: 𝐘ₜ = 𝜌 𝐘ₜ₋₁ + 𝜂ₜ, and 𝐗ₜ = 𝜌* 𝐗ₜ₋₁ + 𝜈ₜ. (11) Where 𝜂ₜ and 𝜈ₜ are white noise. We know that in this case, 𝐑² is the square of the correlation between 𝐘ₜ and 𝐗ₜ. They use Kendall’s result from the article Knowles (1954), which expresses the variance of 𝐑: 𝐕𝐚𝐫(𝐑) = (1/T)* (1 + 𝜌𝜌*) / (1 − 𝜌𝜌*). (12) Since 𝐑 is constrained to lie between -1 and 1, if its variance is greater than 1/3, the distribution of 𝐑 cannot have a mode at 0. This implies that 𝜌𝜌* > (T−1) / (T+1). Thus, for example, if T = 20 and 𝜌 = 𝜌*, a distribution that is not unimodal at 0 will be obtained if 𝜌 > 0.86, and if 𝜌 = 0.9, 𝐕𝐚𝐫(𝐑) = 0.47. So the 𝐄(𝐑²) will be close to 0.47. It has been shown that when 𝜌 is close to 1, 𝐑² can be very high, suggesting a strong relationship between 𝐘ₜ and 𝐗ₜ. However, in reality, the two series are completely independent. When 𝜌 is near 1, both series behave like random walks or near-random walks. On top of that, both series are highly autocorrelated, which causes the residuals from the regression to also be strongly autocorrelated. As a result, the Durbin-Watson statistic 𝐝 will be very low. This is why a high 𝐑² in this context should never be taken as evidence of a true relationship between the two series. To explore the possibility of obtaining a spurious regression when regressing two independent random walks, a series of simulations proposed by Granger and Newbold (1974) will be conducted in the next section. 4. Simulation results using Python. In this section, we will show using simulations that using the regression model with independent random walks bias the estimation of the coefficients and the hypothesis tests of the coefficients are invalid. The Python code that will produce the results of the simulation will be presented in section 6. A regression equation proposed by Granger and Newbold (1974) is given by: 𝐘ₜ = 𝛽₀ + 𝐗ₜ𝛽₁ + 𝜖ₜ Where 𝐘ₜ and 𝐗ₜ were generated as independent random walks, each of length 50. The values 𝐒 = |𝛽̂₁| / √(𝐒𝐄̂(𝛽̂₁)), representing the statistic for testing the significance of 𝛽₁, for 100 simulations will be reported in the table below. Table 1: Regressing two independent random walks The null hypothesis of no relationship between 𝐘ₜ and 𝐗ₜ is rejected at the 5% level if 𝐒 > 2. This table shows that the null hypothesis (𝛽 = 0) is wrongly rejected in about a quarter (71 times) of all cases. This is awkward because the two variables are independent random walks, meaning there’s no actual relationship. Let’s break down why this happens. If 𝛽̂₁ / 𝐒𝐄̂ follows a 𝐍(0,1), the expected value of 𝐒, its absolute value, should be √2 / π ≈ 0.8 (√2/π is the mean of the absolute value of a standard normal distribution). However, the simulation results show an average of 4.59, meaning the estimated 𝐒 is underestimated by a factor of: 4.59 / 0.8 = 5.7 In classical statistics, we usually use a t-test threshold of around 2 to check the significance of a coefficient. However, these results show that, in this case, you would need to use a threshold of 11.4 to properly test for significance: 2 × (4.59 / 0.8) = 11.4 Interpretation: We’ve just shown that including variables that don’t belong in the model — especially random walks — can lead to completely invalid significance tests for the coefficients. To make their simulations even clearer, Granger and Newbold (1974) ran a series of regressions using variables that follow either a random walk or an ARIMA(0,1,1) process. Here is how they set up their simulations: They regressed a dependent series 𝐘ₜ on m series 𝐗ⱼ,ₜ (with j = 1, 2, …, m), varying m from 1 to 5. The dependent series 𝐘ₜ and the independent series 𝐗ⱼ,ₜ follow the same types of processes, and they tested four cases: Case 1 (Levels): 𝐘ₜ and 𝐗ⱼ,ₜ follow random walks. Case 2 (Differences): They use the first differences of the random walks, which are stationary. Case 3 (Levels): 𝐘ₜ and 𝐗ⱼ,ₜ follow ARIMA(0,1,1). Case 4 (Differences): They use the first differences of the previous ARIMA(0,1,1) processes, which are stationary. Each series has a length of 50 observations, and they ran 100 simulations for each case. All error terms are distributed as 𝐍(0,1), and the ARIMA(0,1,1) series are derived as the sum of the random walk and independent white noise. The simulation results, based on 100 replications with series of length 50, are summarized in the next table. Table 2: Regressions of a series on m independent ‘explanatory’ series. Interpretation of the results : It is seen that the probability of not rejecting the null hypothesis of no relationship between 𝐘ₜ and 𝐗ⱼ,ₜ becomes very small when m ≥ 3 when regressions are made with random walk series (rw-levels). The 𝐑² and the mean Durbin-Watson increase. Similar results are obtained when the regressions are made with ARIMA(0,1,1) series (arima-levels). When white noise series (rw-diffs) are used, classical regression analysis is valid since the error series will be white noise and least squares will be efficient. However, when the regressions are made with the differences of ARIMA(0,1,1) series (arima-diffs) or first-order moving average series MA(1) process, the null hypothesis is rejected, on average: (10 + 16 + 5 + 6 + 6) / 5 = 8.6 which is greater than 5% of the time. If your variables are random walks or close to them, and you include unnecessary variables in your regression, you will often get fallacious results. High 𝐑² and low Durbin-Watson values do not confirm a true relationship but instead indicate a likely spurious one. 5. How to avoid spurious regression in time series It’s really hard to come up with a complete list of ways to avoid spurious regressions. However, there are a few good practices you can follow to minimize the risk as much as possible. If one performs a regression analysis with time series data and finds that the residuals are strongly autocorrelated, there is a serious problem when it comes to interpreting the coefficients of the equation. To check for autocorrelation in the residuals, one can use the Durbin-Watson test or the Portmanteau test. Based on the study above, we can conclude that if a regression analysis performed with economical variables produces strongly autocorrelated residuals, meaning a low Durbin-Watson statistic, then the results of the analysis are likely to be spurious, whatever the value of the coefficient of determination R² observed. In such cases, it is important to understand where the mis-specification comes from. According to the literature, misspecification usually falls into three categories : (i) the omission of a relevant variable, (ii) the inclusion of an irrelevant variable, or (iii) autocorrelation of the errors. Most of the time, mis-specification comes from a mix of these three sources. To avoid spurious regression in a time series, several recommendations can be made: The first recommendation is to select the right macroeconomic variables that are likely to explain the dependent variable. This can be done by reviewing the literature or consulting experts in the field. The second recommendation is to stationarize the series by taking first differences. In most cases, the first differences of macroeconomic variables are stationary and still easy to interpret. For macroeconomic data, it’s strongly recommended to differentiate the series once to reduce the autocorrelation of the residuals, especially when the sample size is small. There is indeed sometimes strong serial correlation observed in these variables. A simple calculation shows that the first differences will almost always have much smaller serial correlations than the original series. The third recommendation is to use the Box-Jenkins methodology to model each macroeconomic variable individually and then search for relationships between the series by relating the residuals from each individual model. The idea here is that the Box-Jenkins process extracts the explained part of the series, leaving the residuals, which contain only what can’t be explained by the series’ own past behavior. This makes it easier to check whether these unexplained parts (residuals) are related across variables. 6. Conclusion Many econometrics textbooks warn about specification errors in regression models, but the problem still shows up in many published papers. Granger and Newbold (1974) highlighted the risk of spurious regressions, where you get a high paired with very low Durbin-Watson statistics. Using Python simulations, we showed some of the main causes of these spurious regressions, especially including variables that don’t belong in the model and are highly autocorrelated. We also demonstrated how these issues can completely distort hypothesis tests on the coefficients. Hopefully, this post will help reduce the risk of spurious regressions in future econometric analyses. 7. Appendice: Python code for simulation. #####################################################Simulation Code for table 1 ##################################################### import numpy as np import pandas as pd import statsmodels.api as sm import matplotlib.pyplot as plt np.random.seed(123) M = 100 n = 50 S = np.zeros(M) for i in range(M): #————————————————————— # Generate the data #————————————————————— espilon_y = np.random.normal(0, 1, n) espilon_x = np.random.normal(0, 1, n) Y = np.cumsum(espilon_y) X = np.cumsum(espilon_x) #————————————————————— # Fit the model #————————————————————— X = sm.add_constant(X) model = sm.OLS(Y, X).fit() #————————————————————— # Compute the statistic #—————————————————— S[i] = np.abs(model.params[1])/model.bse[1] #—————————————————— # Maximum value of S #—————————————————— S_max = int(np.ceil(max(S))) #—————————————————— # Create bins #—————————————————— bins = np.arange(0, S_max + 2, 1) #—————————————————— # Compute the histogram #—————————————————— frequency, bin_edges = np.histogram(S, bins=bins) #—————————————————— # Create a dataframe #—————————————————— df = pd.DataFrame({ “S Interval”: [f”{int(bin_edges[i])}-{int(bin_edges[i+1])}” for i in range(len(bin_edges)-1)], “Frequency”: frequency }) print(df) print(np.mean(S)) #####################################################Simulation Code for table 2 ##################################################### import numpy as np import pandas as pd import statsmodels.api as sm from statsmodels.stats.stattools import durbin_watson from tabulate import tabulate np.random.seed(1) # Pour rendre les résultats reproductibles #—————————————————— # Definition of functions #—————————————————— def generate_random_walk(T): “”” Génère une série de longueur T suivant un random walk : Y_t = Y_{t-1} + e_t, où e_t ~ N(0,1). “”” e = np.random.normal(0, 1, size=T) return np.cumsum(e) def generate_arima_0_1_1(T): “”” Génère un ARIMA(0,1,1) selon la méthode de Granger & Newbold : la série est obtenue en additionnant une marche aléatoire et un bruit blanc indépendant. “”” rw = generate_random_walk(T) wn = np.random.normal(0, 1, size=T) return rw + wn def difference(series): “”” Calcule la différence première d’une série unidimensionnelle. Retourne une série de longueur T-1. “”” return np.diff(series) #—————————————————— # Paramètres #—————————————————— T = 50 # longueur de chaque série n_sims = 100 # nombre de simulations Monte Carlo alpha = 0.05 # seuil de significativité #—————————————————— # Definition of function for simulation #—————————————————— def run_simulation_case(case_name, m_values=[1,2,3,4,5]): “”” case_name : un identifiant pour le type de génération : – ‘rw-levels’ : random walk (levels) – ‘rw-diffs’ : differences of RW (white noise) – ‘arima-levels’ : ARIMA(0,1,1) en niveaux – ‘arima-diffs’ : différences d’un ARIMA(0,1,1) = > MA(1) m_values : liste du nombre de régresseurs. Retourne un DataFrame avec pour chaque m : – % de rejets de H0 – Durbin-Watson moyen – R^2_adj moyen – % de R^2 > 0.1 “”” results = [] for m in m_values: count_reject = 0 dw_list = [] r2_adjusted_list = [] for _ in range(n_sims): #————————————– # 1) Generation of independents de Y_t and X_{j,t}. #—————————————- if case_name == ‘rw-levels’: Y = generate_random_walk(T) Xs = [generate_random_walk(T) for __ in range(m)] elif case_name == ‘rw-diffs’: # Y et X sont les différences d’un RW, i.e. ~ white noise Y_rw = generate_random_walk(T) Y = difference(Y_rw) Xs = [] for __ in range(m): X_rw = generate_random_walk(T) Xs.append(difference(X_rw)) # NB : maintenant Y et Xs ont longueur T-1 # = > ajuster T_effectif = T-1 # = > on prendra T_effectif points pour la régression elif case_name == ‘arima-levels’: Y = generate_arima_0_1_1(T) Xs = [generate_arima_0_1_1(T) for __ in range(m)] elif case_name == ‘arima-diffs’: # Différences d’un ARIMA(0,1,1) = > MA(1) Y_arima = generate_arima_0_1_1(T) Y = difference(Y_arima) Xs = [] for __ in range(m): X_arima = generate_arima_0_1_1(T) Xs.append(difference(X_arima)) # 2) Prépare les données pour la régression # Selon le cas, la longueur est T ou T-1 if case_name in [‘rw-levels’,’arima-levels’]: Y_reg = Y X_reg = np.column_stack(Xs) if m >0 else np.array([]) else: # dans les cas de différences, la longueur est T-1 Y_reg = Y X_reg = np.column_stack(Xs) if m >0 else np.array([]) # 3) Régression OLS X_with_const = sm.add_constant(X_reg) # Ajout de l’ordonnée à l’origine model = sm.OLS(Y_reg, X_with_const).fit() # 4) Test global F : H0 : tous les beta_j = 0 # On regarde si p-value < alpha if model.f_pvalue is not None and model.f_pvalue 0.7) results.append({ ‘m’: m, ‘Reject %’: reject_percent, ‘Mean DW’: dw_mean, ‘Mean R^2’: r2_mean, ‘% R^2_adj >0.7’: r2_above_0_7_percent }) return pd.DataFrame(results) #—————————————————— # Application of the simulation #—————————————————— cases = [‘rw-levels’, ‘rw-diffs’, ‘arima-levels’, ‘arima-diffs’] all_results = {} for c in cases: df_res = run_simulation_case(c, m_values=[1,2,3,4,5]) all_results[c] = df_res #—————————————————— # Store data in table #—————————————————— for case, df_res in all_results.items(): print(f”nn{case}”) print(tabulate(df_res, headers=’keys’, tablefmt=’fancy_grid’)) References Granger, Clive WJ, and Paul Newbold. 1974. “Spurious Regressions in Econometrics.” Journal of Econometrics 2 (2): 111–20. Knowles, EAG. 1954. “Exercises in Theoretical Statistics.” Oxford University Press.

1. Introduction

It’s pretty clear that most of our work will be automated by AI in the future. This will be possible because many researchers and professionals are working hard to make their work available online. These contributions not only help us understand fundamental concepts but also refine AI models, ultimately freeing up time to focus on other activities.

However, there is one concept that remains misunderstood, even among experts. It is spurious regression in time series analysis. This issue arises when regression models suggest strong relationships between variables, even when none exist. It is typically observed in time series regression equations that seem to have a high degree of fit — as indicated by a high R² (coefficient of multiple correlation) — but with an extremely low Durbin-Watson statistic (d), signaling strong autocorrelation in the error terms.

What is particularly surprising is that almost all econometric textbooks warn about the danger of autocorrelated errors, yet this issue persists in many published papers. Granger and Newbold (1974) identified several examples. For instance, they found published equations with R² = 0.997 and the Durbin-Watson statistic (d) equal to 0.53. The most extreme found is an equation with R² = 0.999 and d = 0.093.

It is especially problematic in economics and finance, where many key variables exhibit autocorrelation or serial correlation between adjacent values, particularly if the sampling interval is small, such as a week or a month, leading to misleading conclusions if not handled correctly. For example, today’s GDP is strongly correlated with the GDP of the previous quarter. Our post provides a detailed explanation of the results from Granger and Newbold (1974) and Python simulation (see section 7) replicating the key results presented in their article.

Whether you’re an economist, data scientist, or analyst working with time series data, understanding this issue is crucial to ensuring your models produce meaningful results.

To walk you through this paper, the next section will introduce the random walk and the ARIMA(0,1,1) process. In section 3, we will explain how Granger and Newbold (1974) describe the emergence of nonsense regressions, with examples illustrated in section 4. Finally, we’ll show how to avoid spurious regressions when working with time series data.

2. Simple presentation of a Random Walk and ARIMA(0,1,1) Process

2.1 Random Walk

Let 𝐗ₜ be a time series. We say that 𝐗ₜ follows a random walk if its representation is given by:

𝐗ₜ = 𝐗ₜ₋₁ + 𝜖ₜ. (1)

Where 𝜖ₜ is a white noise. It can be written as a sum of white noise, a useful form for simulation. It is a non-stationary time series because its variance depends on the time t.

2.2 ARIMA(0,1,1) Process

The ARIMA(0,1,1) process is given by:

𝐗ₜ = 𝐗ₜ₋₁ + 𝜖ₜ − 𝜃 𝜖ₜ₋₁. (2)

where 𝜖ₜ is a white noise. The ARIMA(0,1,1) process is non-stationary. It can be written as a sum of an independent random walk and white noise:

𝐗ₜ = 𝐗₀ + random walk + white noise. (3) This form is useful for simulation.

Those non-stationary series are often employed as benchmarks against which the forecasting performance of other models is judged.

3. Random walk can lead to Nonsense Regression

First, let’s recall the Linear Regression model. The linear regression model is given by:

𝐘 = 𝐗𝛽 + 𝜖. (4)

Where 𝐘 is a T × 1 vector of the dependent variable, 𝛽 is a K × 1 vector of the coefficients, 𝐗 is a T × K matrix of the independent variables containing a column of ones and (K−1) columns with T observations on each of the (K−1) independent variables, which are stochastic but distributed independently of the T × 1 vector of the errors 𝜖. It is generally assumed that:

𝐄(𝜖) = 0, (5)

and

𝐄(𝜖𝜖′) = 𝜎²𝐈. (6)

where 𝐈 is the identity matrix.

A test of the contribution of independent variables to the explanation of the dependent variable is the F-test. The null hypothesis of the test is given by:

𝐇₀: 𝛽₁ = 𝛽₂ = ⋯ = 𝛽ₖ₋₁ = 0, (7)

And the statistic of the test is given by:

𝐅 = (𝐑² / (𝐊−1)) / ((1−𝐑²) / (𝐓−𝐊)). (8)

where 𝐑² is the coefficient of determination.

If we want to construct the statistic of the test, let’s assume that the null hypothesis is true, and one tries to fit a regression of the form (Equation 4) to the levels of an economic time series. Suppose next that these series are not stationary or are highly autocorrelated. In such a situation, the test procedure is invalid since 𝐅 in (Equation 8) is not distributed as an F-distribution under the null hypothesis (Equation 7). In fact, under the null hypothesis, the errors or residuals from (Equation 4) are given by:

𝜖ₜ = 𝐘ₜ − 𝐗𝛽₀ ; t = 1, 2, …, T. (9)

And will have the same autocorrelation structure as the original series 𝐘.

Some idea of the distribution problem can arise in the situation when:

𝐘ₜ = 𝛽₀ + 𝐗ₜ𝛽₁ + 𝜖ₜ. (10)

Where 𝐘ₜ and 𝐗ₜ follow independent first-order autoregressive processes:

𝐘ₜ = 𝜌 𝐘ₜ₋₁ + 𝜂ₜ, and 𝐗ₜ = 𝜌* 𝐗ₜ₋₁ + 𝜈ₜ. (11)

Where 𝜂ₜ and 𝜈ₜ are white noise.

We know that in this case, 𝐑² is the square of the correlation between 𝐘ₜ and 𝐗ₜ. They use Kendall’s result from the article Knowles (1954), which expresses the variance of 𝐑:

𝐕𝐚𝐫(𝐑) = (1/T)* (1 + 𝜌𝜌*) / (1 − 𝜌𝜌*). (12)

Since 𝐑 is constrained to lie between -1 and 1, if its variance is greater than 1/3, the distribution of 𝐑 cannot have a mode at 0. This implies that 𝜌𝜌* > (T−1) / (T+1).

Thus, for example, if T = 20 and 𝜌 = 𝜌*, a distribution that is not unimodal at 0 will be obtained if 𝜌 > 0.86, and if 𝜌 = 0.9, 𝐕𝐚𝐫(𝐑) = 0.47. So the 𝐄(𝐑²) will be close to 0.47.

It has been shown that when 𝜌 is close to 1, 𝐑² can be very high, suggesting a strong relationship between 𝐘ₜ and 𝐗ₜ. However, in reality, the two series are completely independent. When 𝜌 is near 1, both series behave like random walks or near-random walks. On top of that, both series are highly autocorrelated, which causes the residuals from the regression to also be strongly autocorrelated. As a result, the Durbin-Watson statistic 𝐝 will be very low.

This is why a high 𝐑² in this context should never be taken as evidence of a true relationship between the two series.

To explore the possibility of obtaining a spurious regression when regressing two independent random walks, a series of simulations proposed by Granger and Newbold (1974) will be conducted in the next section.

4. Simulation results using Python.

In this section, we will show using simulations that using the regression model with independent random walks bias the estimation of the coefficients and the hypothesis tests of the coefficients are invalid. The Python code that will produce the results of the simulation will be presented in section 6.

A regression equation proposed by Granger and Newbold (1974) is given by:

𝐘ₜ = 𝛽₀ + 𝐗ₜ𝛽₁ + 𝜖ₜ

Where 𝐘ₜ and 𝐗ₜ were generated as independent random walks, each of length 50. The values 𝐒 = |𝛽̂₁| / √(𝐒𝐄̂(𝛽̂₁)), representing the statistic for testing the significance of 𝛽₁, for 100 simulations will be reported in the table below.

Table 1: Regressing two independent random walks

The null hypothesis of no relationship between 𝐘ₜ and 𝐗ₜ is rejected at the 5% level if 𝐒 > 2. This table shows that the null hypothesis (𝛽 = 0) is wrongly rejected in about a quarter (71 times) of all cases. This is awkward because the two variables are independent random walks, meaning there’s no actual relationship. Let’s break down why this happens.

If 𝛽̂₁ / 𝐒𝐄̂ follows a 𝐍(0,1), the expected value of 𝐒, its absolute value, should be √2 / π ≈ 0.8 (√2/π is the mean of the absolute value of a standard normal distribution). However, the simulation results show an average of 4.59, meaning the estimated 𝐒 is underestimated by a factor of:

4.59 / 0.8 = 5.7

In classical statistics, we usually use a t-test threshold of around 2 to check the significance of a coefficient. However, these results show that, in this case, you would need to use a threshold of 11.4 to properly test for significance:

2 × (4.59 / 0.8) = 11.4

Interpretation: We’ve just shown that including variables that don’t belong in the model — especially random walks — can lead to completely invalid significance tests for the coefficients.

To make their simulations even clearer, Granger and Newbold (1974) ran a series of regressions using variables that follow either a random walk or an ARIMA(0,1,1) process.

Here is how they set up their simulations:

They regressed a dependent series 𝐘ₜ on m series 𝐗ⱼ,ₜ (with j = 1, 2, …, m), varying m from 1 to 5. The dependent series 𝐘ₜ and the independent series 𝐗ⱼ,ₜ follow the same types of processes, and they tested four cases:

  • Case 1 (Levels): 𝐘ₜ and 𝐗ⱼ,ₜ follow random walks.
  • Case 2 (Differences): They use the first differences of the random walks, which are stationary.
  • Case 3 (Levels): 𝐘ₜ and 𝐗ⱼ,ₜ follow ARIMA(0,1,1).
  • Case 4 (Differences): They use the first differences of the previous ARIMA(0,1,1) processes, which are stationary.

Each series has a length of 50 observations, and they ran 100 simulations for each case.

All error terms are distributed as 𝐍(0,1), and the ARIMA(0,1,1) series are derived as the sum of the random walk and independent white noise. The simulation results, based on 100 replications with series of length 50, are summarized in the next table.

Table 2: Regressions of a series on m independent ‘explanatory’ series.

Interpretation of the results :

  • It is seen that the probability of not rejecting the null hypothesis of no relationship between 𝐘ₜ and 𝐗ⱼ,ₜ becomes very small when m ≥ 3 when regressions are made with random walk series (rw-levels). The 𝐑² and the mean Durbin-Watson increase. Similar results are obtained when the regressions are made with ARIMA(0,1,1) series (arima-levels).
  • When white noise series (rw-diffs) are used, classical regression analysis is valid since the error series will be white noise and least squares will be efficient.
  • However, when the regressions are made with the differences of ARIMA(0,1,1) series (arima-diffs) or first-order moving average series MA(1) process, the null hypothesis is rejected, on average:

(10 + 16 + 5 + 6 + 6) / 5 = 8.6

which is greater than 5% of the time.

If your variables are random walks or close to them, and you include unnecessary variables in your regression, you will often get fallacious results. High 𝐑² and low Durbin-Watson values do not confirm a true relationship but instead indicate a likely spurious one.

5. How to avoid spurious regression in time series

It’s really hard to come up with a complete list of ways to avoid spurious regressions. However, there are a few good practices you can follow to minimize the risk as much as possible.

If one performs a regression analysis with time series data and finds that the residuals are strongly autocorrelated, there is a serious problem when it comes to interpreting the coefficients of the equation. To check for autocorrelation in the residuals, one can use the Durbin-Watson test or the Portmanteau test.

Based on the study above, we can conclude that if a regression analysis performed with economical variables produces strongly autocorrelated residuals, meaning a low Durbin-Watson statistic, then the results of the analysis are likely to be spurious, whatever the value of the coefficient of determination R² observed.

In such cases, it is important to understand where the mis-specification comes from. According to the literature, misspecification usually falls into three categories : (i) the omission of a relevant variable, (ii) the inclusion of an irrelevant variable, or (iii) autocorrelation of the errors. Most of the time, mis-specification comes from a mix of these three sources.

To avoid spurious regression in a time series, several recommendations can be made:

  • The first recommendation is to select the right macroeconomic variables that are likely to explain the dependent variable. This can be done by reviewing the literature or consulting experts in the field.
  • The second recommendation is to stationarize the series by taking first differences. In most cases, the first differences of macroeconomic variables are stationary and still easy to interpret. For macroeconomic data, it’s strongly recommended to differentiate the series once to reduce the autocorrelation of the residuals, especially when the sample size is small. There is indeed sometimes strong serial correlation observed in these variables. A simple calculation shows that the first differences will almost always have much smaller serial correlations than the original series.
  • The third recommendation is to use the Box-Jenkins methodology to model each macroeconomic variable individually and then search for relationships between the series by relating the residuals from each individual model. The idea here is that the Box-Jenkins process extracts the explained part of the series, leaving the residuals, which contain only what can’t be explained by the series’ own past behavior. This makes it easier to check whether these unexplained parts (residuals) are related across variables.

6. Conclusion

Many econometrics textbooks warn about specification errors in regression models, but the problem still shows up in many published papers. Granger and Newbold (1974) highlighted the risk of spurious regressions, where you get a high paired with very low Durbin-Watson statistics.

Using Python simulations, we showed some of the main causes of these spurious regressions, especially including variables that don’t belong in the model and are highly autocorrelated. We also demonstrated how these issues can completely distort hypothesis tests on the coefficients.

Hopefully, this post will help reduce the risk of spurious regressions in future econometric analyses.

7. Appendice: Python code for simulation.

#####################################################Simulation Code for table 1 #####################################################

import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt

np.random.seed(123)
M = 100 
n = 50
S = np.zeros(M)
for i in range(M):
#---------------------------------------------------------------
# Generate the data
#---------------------------------------------------------------
    espilon_y = np.random.normal(0, 1, n)
    espilon_x = np.random.normal(0, 1, n)

    Y = np.cumsum(espilon_y)
    X = np.cumsum(espilon_x)
#---------------------------------------------------------------
# Fit the model
#---------------------------------------------------------------
    X = sm.add_constant(X)
    model = sm.OLS(Y, X).fit()
#---------------------------------------------------------------
# Compute the statistic
#------------------------------------------------------
    S[i] = np.abs(model.params[1])/model.bse[1]


#------------------------------------------------------ 
#              Maximum value of S
#------------------------------------------------------
S_max = int(np.ceil(max(S)))

#------------------------------------------------------ 
#                Create bins
#------------------------------------------------------
bins = np.arange(0, S_max + 2, 1)  

#------------------------------------------------------
#    Compute the histogram
#------------------------------------------------------
frequency, bin_edges = np.histogram(S, bins=bins)

#------------------------------------------------------
#    Create a dataframe
#------------------------------------------------------

df = pd.DataFrame({
    "S Interval": [f"{int(bin_edges[i])}-{int(bin_edges[i+1])}" for i in range(len(bin_edges)-1)],
    "Frequency": frequency
})
print(df)
print(np.mean(S))

#####################################################Simulation Code for table 2 #####################################################

import numpy as np
import pandas as pd
import statsmodels.api as sm
from statsmodels.stats.stattools import durbin_watson
from tabulate import tabulate

np.random.seed(1)  # Pour rendre les résultats reproductibles

#------------------------------------------------------
# Definition of functions
#------------------------------------------------------

def generate_random_walk(T):
    """
    Génère une série de longueur T suivant un random walk :
        Y_t = Y_{t-1} + e_t,
    où e_t ~ N(0,1).
    """
    e = np.random.normal(0, 1, size=T)
    return np.cumsum(e)

def generate_arima_0_1_1(T):
    """
    Génère un ARIMA(0,1,1) selon la méthode de Granger & Newbold :
    la série est obtenue en additionnant une marche aléatoire et un bruit blanc indépendant.
    """
    rw = generate_random_walk(T)
    wn = np.random.normal(0, 1, size=T)
    return rw + wn

def difference(series):
    """
    Calcule la différence première d'une série unidimensionnelle.
    Retourne une série de longueur T-1.
    """
    return np.diff(series)

#------------------------------------------------------
# Paramètres
#------------------------------------------------------

T = 50           # longueur de chaque série
n_sims = 100     # nombre de simulations Monte Carlo
alpha = 0.05     # seuil de significativité

#------------------------------------------------------
# Definition of function for simulation
#------------------------------------------------------

def run_simulation_case(case_name, m_values=[1,2,3,4,5]):
    """
    case_name : un identifiant pour le type de génération :
        - 'rw-levels' : random walk (levels)
        - 'rw-diffs'  : differences of RW (white noise)
        - 'arima-levels' : ARIMA(0,1,1) en niveaux
        - 'arima-diffs'  : différences d'un ARIMA(0,1,1) => MA(1)
    
    m_values : liste du nombre de régresseurs.
    
    Retourne un DataFrame avec pour chaque m :
        - % de rejets de H0
        - Durbin-Watson moyen
        - R^2_adj moyen
        - % de R^2 > 0.1
    """
    results = []
    
    for m in m_values:
        count_reject = 0
        dw_list = []
        r2_adjusted_list = []
        
        for _ in range(n_sims):
#--------------------------------------
# 1) Generation of independents de Y_t and X_{j,t}.
#----------------------------------------
            if case_name == 'rw-levels':
                Y = generate_random_walk(T)
                Xs = [generate_random_walk(T) for __ in range(m)]
            
            elif case_name == 'rw-diffs':
                # Y et X sont les différences d'un RW, i.e. ~ white noise
                Y_rw = generate_random_walk(T)
                Y = difference(Y_rw)
                Xs = []
                for __ in range(m):
                    X_rw = generate_random_walk(T)
                    Xs.append(difference(X_rw))
                # NB : maintenant Y et Xs ont longueur T-1
                # => ajuster T_effectif = T-1
                # => on prendra T_effectif points pour la régression
            
            elif case_name == 'arima-levels':
                Y = generate_arima_0_1_1(T)
                Xs = [generate_arima_0_1_1(T) for __ in range(m)]
            
            elif case_name == 'arima-diffs':
                # Différences d'un ARIMA(0,1,1) => MA(1)
                Y_arima = generate_arima_0_1_1(T)
                Y = difference(Y_arima)
                Xs = []
                for __ in range(m):
                    X_arima = generate_arima_0_1_1(T)
                    Xs.append(difference(X_arima))
            
            # 2) Prépare les données pour la régression
            #    Selon le cas, la longueur est T ou T-1
            if case_name in ['rw-levels','arima-levels']:
                Y_reg = Y
                X_reg = np.column_stack(Xs) if m>0 else np.array([])
            else:
                # dans les cas de différences, la longueur est T-1
                Y_reg = Y
                X_reg = np.column_stack(Xs) if m>0 else np.array([])
            
            # 3) Régression OLS
            X_with_const = sm.add_constant(X_reg)  # Ajout de l'ordonnée à l'origine
            model = sm.OLS(Y_reg, X_with_const).fit()
            
            # 4) Test global F : H0 : tous les beta_j = 0
            #    On regarde si p-value < alpha
            if model.f_pvalue is not None and model.f_pvalue  0.7)
        
        results.append({
            'm': m,
            'Reject %': reject_percent,
            'Mean DW': dw_mean,
            'Mean R^2': r2_mean,
            '% R^2_adj>0.7': r2_above_0_7_percent
        })
    
    return pd.DataFrame(results)
    
#------------------------------------------------------
# Application of the simulation
#------------------------------------------------------       

cases = ['rw-levels', 'rw-diffs', 'arima-levels', 'arima-diffs']
all_results = {}

for c in cases:
    df_res = run_simulation_case(c, m_values=[1,2,3,4,5])
    all_results[c] = df_res

#------------------------------------------------------
# Store data in table
#------------------------------------------------------

for case, df_res in all_results.items():
    print(f"nn{case}")
    print(tabulate(df_res, headers='keys', tablefmt='fancy_grid'))

References

  • Granger, Clive WJ, and Paul Newbold. 1974. “Spurious Regressions in Econometrics.” Journal of Econometrics 2 (2): 111–20.
  • Knowles, EAG. 1954. “Exercises in Theoretical Statistics.” Oxford University Press.
Shape
Shape
Stay Ahead

Explore More Insights

Stay ahead with more perspectives on cutting-edge power, infrastructure, energy,  bitcoin and AI solutions. Explore these articles to uncover strategies and insights shaping the future of industries.

Shape

INPEX sanctions Ichthys LNG expansion project, awards contract

The INPEX Corp.-led Ichthys Joint Venture has sanctioned the next phase of development of the Ichthys LNG project offshore Western Australia and awarded a contract to a consortium of McDermott and Baker Hughes for part of the work. The development, designated Phase 2c Part 1, includes five production wells targeting

Read More »

IEA sees potential for looser oil, LNG markets after Hormuz crisis

Global oil and natural gas markets could face a period of more abundant supply and lower prices once disruptions in the Middle East ease, although the timing and extent of any recovery remain uncertain, according to the International Energy Agency’s (IEA) World Energy Outlook 2026. The outlook points to sharply different demand trajectories depending on how governments respond to the 2026 energy crisis. Under IEA’s Stated Policies Scenario (STEPS), global oil demand levels off at around 103 million b/d by 2030 before declining gradually, while natural gas demand continues growing until the mid-2030s. Under the Current Policies Scenario (CPS), consumption of both fuels continues to increase. The projections assume that constraints on Gulf oil and gas exports will ease into 2027 and that conditions will allow unimpeded transit through the Strait of Hormuz by end-2027. IEA cautioned that the scenarios are not forecasts and that considerable uncertainty remains over the duration of the crisis and the future of Middle East exports. The disruption to energy flows through the Strait of Hormuz has resulted in cumulative losses of 2.8 billion bbl of oil and 60 billion cu m (bcm) of natural gas supply from the Gulf. At its peak, global oil supply fell by 16.1 million b/d from pre-conflict levels. Ample inventories, increased production and exports from non-Gulf suppliers, alternative export routes, and reduced Chinese crude imports helped cushion the initial impact. IEA member countries also coordinated the release of 400 million bbl of emergency oil stocks, announced Mar. 11, 2026. As inventories declined and supply disruptions continued, however, higher prices increasingly weighed on consumption. Diesel markets have been particularly strained, with prices exceeding $200/bbl amid disruptions to Middle Eastern product exports and reduced Russian supplies. IEA’s November 2026 Oil Market Report expects global oil demand to decline by around 2.7

Read More »

Phelan Green advances development of South African SAF complex

Privately held Phelan Green Group’s clean-fuels subsidiary Phelan eFuels has let a contract to Honeywell Technologies to deliver a suite of services to support the first-phase of its multi-phased electro sustainable aviation fuel (eSAF) production complex under development as part of the operator’s broader $2.5-billion Phelan Green Hydrogen project in Saldanha Bay, Western Cape, South Africa. As part of the contract, Honeywell Technologies will provide licensing for its proprietary Advanced Solvent Carbon Capture (ASCC) technology—a co-development of Honeywell UOP and the University of Texas (UT) at Austin—alongside modular engineering services for the Phase 1 eSAF plant on which Phelan Green recently took positive final investment decision to proceed, the service provider said in a release Nov. 18. The ASCC technology will enable the plant to capture carbon dioxide (CO2) from on-site flue gas which—under either scenario—will contribute to reduced CO2 emissions from the plant. This latest contract for the Phase 1 eSAF plant follows Phelan Green’s previous award earlier in the year to Honeywell UOP for licensing of its proprietary Fischer-Tropsch (FT) Unicracking process technology, which uses FT liquids and waxes made from CO2 to create eSAF that complies with aviation industry standards. Designed with a nameplate e-SAF production capacity of 35,000 tonnes/year (tpy) and scheduled to enter operation in third-quarter 2028, the Phase 1 plant supports Phelan Green’s goal of helping to establish South Africa as a competitive producer and exporter of lower-carbon aviation fuel in a process further streamlined by Honeywell Technologies’ integrated suite of technology, engineering, and equipment offerings, said Blair Phelan, Phelan Green’s managing director. The Phase 1 plant will also use Johnson Matthey PLC (JM) subsidiary JM Catalyst Technologies’ (JM CT) proprietary HyCOgen technology to catalytically convert CO2 and electrolytic (green) hydrogen into carbon monoxide (CO) that will be combined with hydrogen to form synthesis

Read More »

Grid operators need data center ride-through rules that protect reliability

Listen to the article 7 min This audio is auto-generated. Please let us know if you have feedback. Alan Cooper is CEO of ON.energy, a developer of power infrastructure.  On July 22, a mechanical failure forced a 230-kV transmission line in Northern Virginia offline. Within milliseconds, grid protection systems cleared the fault, as they are designed to do. The event would not have been a particularly noteworthy story, had it stopped there. It’s what happened next, though, that pointed to a growing challenge for grid reliability. Optional Caption Permission granted by Alan Cooper In response to the fault, data centers that had been drawing nearly 4 GW of energy, close to 4% of PJM’s load, collectively disconnected and switched to their on-site backup generation. The move sent voltage and frequency spiking across the system, a technically wonky but threatening result that caused power flickers as far away as Chicago. Thankfully, the prevailing grid conditions were strong enough to absorb the shock of the fault, so the grid held. But if the same event had hit Texas or even Northern Virginia at a time of higher demand, it could have caused a widespread outage. In West Texas, the Electric Reliability Counil of Texas estimates that a loss of as little as 3.2 GW could threaten grid reliability. The power sector is not unfamiliar with the need for things hooked up to the electricity grid to “ride through” disturbance events. In 2023, the Federal Energy Regulatory Commission told the North American Electricity Reliability Corp. to develop requirements that inverter-based supply resources like wind and solar facilities must follow to ensure they “ride through,” i.e., stay online, during certain disturbances. And this time around, NERC, FERC and several grid regions have been quick to identify a similar need for large loads to adhere

Read More »

Fuel constraints could challenge grid this winter, despite resource improvements: NERC

Listen to the article 3 min This audio is auto-generated. Please let us know if you have feedback. Dive Brief: The bulk power system’s winter reliability outlook has improved since last season, but the El Niño weather pattern and potential for extreme cold can present challenges that grid operators must remain aware of, the North American Electric Reliability Corp. said Wednesday in its annual seasonal outlook. On-peak capacity resources grew 26.2 GW over the 2025-26 winter, a 2.4% increase led by batteries and gas-fired generation, according to the 2026–2027 Winter Reliability Assessment. A slowdown in generator retirements is also contributing to the improved resource outlook, NERC said. The reliability watchdog’s outlook identified no areas facing a high risk of resource shortfalls this winter, but it found some areas in the Midwest, Southeast and New England where transmission system constraints, pipeline constraints and fuel risk scenarios may present problems during periods of extreme weather. Dive Insight: Electric demand is growing, but “there has also been strong growth in resources,” Mark Olson, manager of reliability assessments, said Wednesday in a call discussing the outlook. “In many ways, the industry is on better footing going into winter this year,” he said. However, Olson added that “historically, winter is the toughest operating period for the North American electric grid.”  While some models show the El Niño weather pattern can bring milder-than-average conditions that are a help to a constrained power system, “the reality is that extreme storms with ice and abnormal cold temperatures can lie within those averages,” Olson said. Retrieved from North American Electric Reliability Corp. “We’re talking to our industry stakeholders about the need for extra vigilance. El Niño is a variable that adds to uncertainty for the winter,” he said. Peak winter electric heating demand often coincides with higher residential gas usage,

Read More »

Energy Secretary Acts to Protect Mid-Atlantic Grid

WASHINGTON—U.S. Secretary of Energy Chris Wright issued an emergency order to reduce the risk of power outages and strengthen grid reliability across the mid-Atlantic region. The order permits PJM Interconnection, L.L.C. (PJM), in coordination with Talen Energy Corporation, to run Unit 4 at the Wagner Generating Station in Anne Arundel County, Maryland, to meet anticipated electricity demand. The order follows an application from PJM submitted on October 23, 2026.  “Energy shortfalls for 65 million Americans are simply unacceptable,” said U.S. Secretary of Energy Chris Wright.  “Previous orders in the mid-Atlantic region have helped keep the lights on and I’m confident this order will help do so in the coming months as well. Today’s order helps temporarily address the urgent need to strengthen grid reliability and ensure Americans across the mid-Atlantic region have access to affordable, reliable, and secure power, regardless of whether the sun shines or the wind blows.”   Secretary Wright previously granted PJM’s requests to operate Wagner Unit 4 beyond its operating limit to help maintain grid reliability.  PJM relied on Wagner Unit 4 during periods of unusually high temperatures earlier this summer, using the unit’s remaining operating hours. PJM has since reaffirmed to the Department of Energy (DOE) the continued need for Wagner Unit 4 to meet high electricity demand, strengthen grid reliability, and reduce the risk of power outages across the mid-Atlantic region.   The renewed order is effective from November 18, 2026, through December 31, 2026. 

Read More »

EIA: US crude oil inventories up 3.4 million bbl

US crude oil inventories for the week ended Nov. 13, excluding the Strategic Petroleum Reserve, increased by 3.4 million bbl from the previous week, according to data from the US Energy Information Administration (EIA). At 417.8 million bbl, US crude oil inventories are 3% below the 5-year average for this time of year, the EIA report indicated. Gasoline inventories increased 5.2 million bbl, 3% above the 5-year average. Propane-propylene inventories remained unchanged, 12% above the 5-year average. Total commercial petroleum inventories increased by 9.2 million bbl for the week. Distillate inventories increased 1.6 million bbl, 7% below the 5-year average. US crude oil refinery inputs averaged 16.9 million b/d for the week ended Nov. 13, which was 129,000 b/d less than the previous week’s average. Refineries operated at 95.7% of capacity. Gasoline output averaged 9.4 million b/d, and distillate production increased to 5.2 million b/d. Crude oil imports decreased 1.1 million b/d to 5.6 million b/d. The 4-week average of 6.3 million b/d is 13% above the year-ago level. Gasoline imports increased and averaged 533,000 b/d; distillate imports also increased and averaged 133,000 b/d. Over the past four weeks, total product supplied averaged 20.6 million b/d, down 0.3% year over year. The 4-week average for gasoline product supplied decreased 1.8% year over year to 8.7 million b/d, while the 4-week average for distillate product supplied increased 4.3% to 4.0 million b/d. The 4-week average for jet fuel product supplied increased 0.3% year over year.

Read More »

Dell and Microsoft drive private AI as enterprises seek greater data control

Dell Technologies and Microsoft are expanding their private cloud partnership with a new set of infrastructure offerings designed to let enterprises run AI models closer to their data, reduce reliance on public cloud services and maintain control over sensitive information. The announcement, made during Microsoft’s Ignite conference, extends Dell’s AI Factory with Nvidia initiative to Dell Private Cloud environments. It also reflects a growing industry push toward internally deployed AI inferencing, particularly among organizations dealing with sensitive corporate information, regulatory requirements and data residency restrictions, Dell stated. The expanded offering combines Dell Private Cloud, Microsoft Azure Local, Microsoft Foundry Local and Nvidia acceleration. The objective is to provide organizations with an integrated infrastructure for deploying heavy AI workloads inside their own data centers and firewalls, the companies said.

Read More »

StarlingX 13.0 updates the network stack for distributed edge deployments

Edge sites do not always come with the network gear that a data center takes for granted. That’s a challenge the open-source StarlingX project from the OpenInfra Foundation is now tackling in its 13.0 update. StarlingX is an open-source distributed cloud platform that combines Linux, Kubernetes, OpenStack, and Ceph. The OpenInfra Foundation hosts the project, and operators including Verizon, Vodafone, T-Systems and KDDI use it for 5G and Open Radio Access Network (O-RAN) infrastructure. Earlier this year, the StarlingX 12.0 update added precision timing capabilities to help networking stay on time. The 13.0 update tackles a different set of issues. Key features include: Gateway API support. Calico-based support is available to hosted applications, while platform services still use Ingress NGINX Optional certificate authority. The Enterprise Java Beans Certificate Authority (EJBCA) runs on the platform and serves certificates to hosted applications. Subcloud restore. A backup restores directly onto a factory-installed node at a subcloud, a remote edge site, with no operating system reinstall. Direct upgrades. Operators can upgrade from two releases back with a single reboot per host. Layer 2 switching. A new Open vSwitch application lets servers at sites with no router next to them bridge traffic and connect directly to aggregation routers. “The OpenvSwitch application adds Layer 2 switching to the K8S platform,” Greg Waines, a senior member of technical staff at Wind River and a StarlingX Technical Steering Committee member, told Network World. “A server can bridge traffic between its SR-IOV network interfaces and other servers, and it can forward traffic to the site routers.”

Read More »

Sovereign AI is about control, not just location

Guetari was open about the term itself being fuzzy. “Sovereignty is very vague and not always clear,” he told me. “It’s about control, auditability, observability.” He drew a line between data locality, meaning the data stays in-country, and sovereignty, which he described as being “more about control, control of what happens to your data and your workload.” He then made the case for why F5 belongs in that conversation: “This is what we have been doing for two decades.” F5 has always sat in the data path, deciding what an application can communicate with, enforcing security rules, and logging what happened. For a regulated bank or government agency, the ability to reconstruct exactly what happened after an incident is worth more than a press release about a data center’s location. The token is the new unit of economics The most interesting part of my conversation with Guetari wasn’t about security. It was about business models. He argued that every successful technology business has found the right unit to monetize. “The cloud is selling the unit of the cloud, which is the CPU core, very well. SaaS is the seat,” he said. “Telcos have been very successful early on using the voice minute, and then they moved to the gigabyte. Sooner or later, that unit will be the token, and most telcos will have to move to the token. And the ones who will not go to token, unfortunately, will be just pipes.”

Read More »

Palo Alto Networks bets $4.5B on enterprise AI with Applied Compute acquisition

GPUs and data centers To train and host those models, Palo Alto plans to build its own compute capacity, CFO Dipak Golechha said on the call. The spending “will mostly go to GPUs and servers,” with a smaller share for the facilities that house them, he said. Palo Alto will “continue to source from our cloud partners” and add dedicated capacity over time, Golechha said. Arora said the company aims for more than100 megawatts of compute in the next three to five years. Gogia said Palo Alto has sovereignty credentials for security data, including customer-held keys and a European Cortex service governed by Deutsche Telekom. “None of that yet extends to GPUs,” he said, adding that a US-headquartered operator faces the same jurisdictional questions as the hyperscalers it rents capacity from. “A sovereign hosting offer with no named region, operator or date is a promise looking for an address,” Gogia said. Compliance officers should ask Palo Alto for a region list, the operator, and a dated certification plan, he said.

Read More »

App developers turn to QVAC for AI tooling without the cloud  

In most instances today, adding AI to an app means sending user data to a third-party API, paying per request, and hoping the connection holds.    Tether’sQVAC is built on the opposite premise. It runs AI models directly on the user’s device. Data never leaves the hardware, there are no per-request costs, no rate limits, and no need for an internet connection once a model is downloaded.   QVAC SDK: the next milestone in cloud-free AI  While open-source tools and standards like llama.cpp, GGML, and ONNX have made local AI much more accessible in recent years, creating new possibilities for consumer devices, the edge AI ecosystem remains fragmented.    Against this backdrop, in 2026, Tether released a software development kit (SDK) that pulls together the best open tooling from across different domains of machine learning.    The centerpiece of Tether’sQVACSDK isFabric. Built on llama.cpp, it expands the engine to enable on-device fine tuning and additional AI capabilities—opening access to memory-intensive workloads previously only possible with expensive hardware or cloud services.    Building on GGML’s modular framework, QVAC Fabric is primarily oriented toward three backends: Vulkan, which covers most consumer-grade desktop and mobile chips; Metal and Core ML for Apple Silicon; and CUDA for higher-end NVIDIA GPU setups.   The QVAC SDK empowers developers with a unified interface for building multiplatformAIappsthatrunon-device. But although significant progress has been made in bringing inference to consumer devices, text generation alone is only one part of the pipeline that makes modern AI useful.    Bringing search to edge AI   RAG (retrieval-augmented generation) is what equips AI to import and process contextual information to enhance its output. Instead of language models relying solely on their training  data to answer questions, RAG lets them answer using a specific set of documents.    Implementing RAG into an AI app incurs significant memory overhead versus running a

Read More »

DCF Video: CoolIT’s 3 MW CDU Brings Smart Monitoring and Fanless Cooling to AI Infrastructure

As AI infrastructure moves toward higher rack densities and increasingly liquid-cooled architectures, the next challenge is not simply removing more heat. It’s ensuring that the cooling systems themselves remain reliable, efficient, and capable of protecting increasingly valuable computing hardware. At the 2026 OCP Global Summit in San Jose, CoolIT Systems Director of Marketing Charles Robison gave Data Center Frontier a close-up look at several technologies addressing that challenge, including the company’s new 3-megawatt CHx 3000 coolant distribution unit (CDU), integrated coolant health monitoring, and a fifth-generation cold plate loop designed for fanless servers. The demonstration comes at a significant moment for CoolIT, which was acquired by Ecolab for approximately $4.75 billion in July 2026. On November 16, Ecolab formally introduced its new integrated AI cooling platform at SC26 in Chicago, placing the CHx 3000 and its embedded 3D TRASAR technology at the center of a broader strategy combining liquid cooling hardware, fluid management, and digital monitoring. Looking Inside the CHx 3000 The CoolIT CHx 3000 is a liquid-to-liquid CDU rated for 3 MW of cooling capacity at a 3°C approach temperature difference, with a specified flow rate of 4,500 liters per minute. Designed for high-density AI deployments, the unit incorporates Ecolab’s 3D TRASAR technology to continuously monitor the condition of the circulating coolant. In the video, Robison points directly to the sensor array that makes the integration possible. “3D TRASAR, and you can see the array of sensors here, allows coolant health monitoring,” Robison explains. “It allows people to see real-time what’s going on with the coolant. So before it goes out of spec, they know what’s happening and they can address it.” The distinction is important. Traditional cooling monitoring typically emphasizes temperature, pressure, and flow. Ecolab’s technology adds visibility into fluid chemistry and contamination, allowing operators to identify deteriorating

Read More »

Microsoft will invest $80B in AI data centers in fiscal 2025

And Microsoft isn’t the only one that is ramping up its investments into AI-enabled data centers. Rival cloud service providers are all investing in either upgrading or opening new data centers to capture a larger chunk of business from developers and users of large language models (LLMs).  In a report published in October 2024, Bloomberg Intelligence estimated that demand for generative AI would push Microsoft, AWS, Google, Oracle, Meta, and Apple would between them devote $200 billion to capex in 2025, up from $110 billion in 2023. Microsoft is one of the biggest spenders, followed closely by Google and AWS, Bloomberg Intelligence said. Its estimate of Microsoft’s capital spending on AI, at $62.4 billion for calendar 2025, is lower than Smith’s claim that the company will invest $80 billion in the fiscal year to June 30, 2025. Both figures, though, are way higher than Microsoft’s 2020 capital expenditure of “just” $17.6 billion. The majority of the increased spending is tied to cloud services and the expansion of AI infrastructure needed to provide compute capacity for OpenAI workloads. Separately, last October Amazon CEO Andy Jassy said his company planned total capex spend of $75 billion in 2024 and even more in 2025, with much of it going to AWS, its cloud computing division.

Read More »

John Deere unveils more autonomous farm machines to address skill labor shortage

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Self-driving tractors might be the path to self-driving cars. John Deere has revealed a new line of autonomous machines and tech across agriculture, construction and commercial landscaping. The Moline, Illinois-based John Deere has been in business for 187 years, yet it’s been a regular as a non-tech company showing off technology at the big tech trade show in Las Vegas and is back at CES 2025 with more autonomous tractors and other vehicles. This is not something we usually cover, but John Deere has a lot of data that is interesting in the big picture of tech. The message from the company is that there aren’t enough skilled farm laborers to do the work that its customers need. It’s been a challenge for most of the last two decades, said Jahmy Hindman, CTO at John Deere, in a briefing. Much of the tech will come this fall and after that. He noted that the average farmer in the U.S. is over 58 and works 12 to 18 hours a day to grow food for us. And he said the American Farm Bureau Federation estimates there are roughly 2.4 million farm jobs that need to be filled annually; and the agricultural work force continues to shrink. (This is my hint to the anti-immigration crowd). John Deere’s autonomous 9RX Tractor. Farmers can oversee it using an app. While each of these industries experiences their own set of challenges, a commonality across all is skilled labor availability. In construction, about 80% percent of contractors struggle to find skilled labor. And in commercial landscaping, 86% of landscaping business owners can’t find labor to fill open positions, he said. “They have to figure out how to do

Read More »

2025 playbook for enterprise AI success, from agents to evals

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More 2025 is poised to be a pivotal year for enterprise AI. The past year has seen rapid innovation, and this year will see the same. This has made it more critical than ever to revisit your AI strategy to stay competitive and create value for your customers. From scaling AI agents to optimizing costs, here are the five critical areas enterprises should prioritize for their AI strategy this year. 1. Agents: the next generation of automation AI agents are no longer theoretical. In 2025, they’re indispensable tools for enterprises looking to streamline operations and enhance customer interactions. Unlike traditional software, agents powered by large language models (LLMs) can make nuanced decisions, navigate complex multi-step tasks, and integrate seamlessly with tools and APIs. At the start of 2024, agents were not ready for prime time, making frustrating mistakes like hallucinating URLs. They started getting better as frontier large language models themselves improved. “Let me put it this way,” said Sam Witteveen, cofounder of Red Dragon, a company that develops agents for companies, and that recently reviewed the 48 agents it built last year. “Interestingly, the ones that we built at the start of the year, a lot of those worked way better at the end of the year just because the models got better.” Witteveen shared this in the video podcast we filmed to discuss these five big trends in detail. Models are getting better and hallucinating less, and they’re also being trained to do agentic tasks. Another feature that the model providers are researching is a way to use the LLM as a judge, and as models get cheaper (something we’ll cover below), companies can use three or more models to

Read More »

OpenAI’s red teaming innovations define new essentials for security leaders in the AI era

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More OpenAI has taken a more aggressive approach to red teaming than its AI competitors, demonstrating its security teams’ advanced capabilities in two areas: multi-step reinforcement and external red teaming. OpenAI recently released two papers that set a new competitive standard for improving the quality, reliability and safety of AI models in these two techniques and more. The first paper, “OpenAI’s Approach to External Red Teaming for AI Models and Systems,” reports that specialized teams outside the company have proven effective in uncovering vulnerabilities that might otherwise have made it into a released model because in-house testing techniques may have missed them. In the second paper, “Diverse and Effective Red Teaming with Auto-Generated Rewards and Multi-Step Reinforcement Learning,” OpenAI introduces an automated framework that relies on iterative reinforcement learning to generate a broad spectrum of novel, wide-ranging attacks. Going all-in on red teaming pays practical, competitive dividends It’s encouraging to see competitive intensity in red teaming growing among AI companies. When Anthropic released its AI red team guidelines in June of last year, it joined AI providers including Google, Microsoft, Nvidia, OpenAI, and even the U.S.’s National Institute of Standards and Technology (NIST), which all had released red teaming frameworks. Investing heavily in red teaming yields tangible benefits for security leaders in any organization. OpenAI’s paper on external red teaming provides a detailed analysis of how the company strives to create specialized external teams that include cybersecurity and subject matter experts. The goal is to see if knowledgeable external teams can defeat models’ security perimeters and find gaps in their security, biases and controls that prompt-based testing couldn’t find. What makes OpenAI’s recent papers noteworthy is how well they define using human-in-the-middle

Read More »

Introducing Gemini 4 Neon

Delivers across enterprise, analytical, and scientific workEnterprise work lives in spreadsheets, PDFs, charts, audio, and video. Neon can synthesize across all of these inputs in

Read More »