It is a simple but powerful technique for time series analysis that provides highly interpretable and effective predictions if your data contains correlations across the time steps. The correlation across time steps is called autocorrelation because it is a measure of how much a value correlates with itself. A purely linear process will autocorrelate perfectly with itself across the time series, making it possible to predict the next value exactly from previous values using an autoregressive process. A completely stochastic process such as white noise will have no autocorrelation since we cannot predict the current or future values by using the past values.

A time series is a sequence of measurements of the same variable or group of variables made over time. The measurements are typically made at evenly spaced times, for instance hourly, monthly or yearly. As an example, we might have values that measure the number of airline passengers in a country, with measurements observed each month. In this case, y represents the measured passenger counts and emphasizes the existence of measured values over time. The value of t is applied as a subscript rather than the usual i to indicate that y_t represents the value of y at any time.

An autoregressive model is when we regress a value from a time series on previous values from that same time series. For example, y_t regressed on y_t-1 uses the previous value of y, called a lagged value, to predict the current value of y. In this simple regression model, the dependent variable in the previous time period has become the predictor. The errors represent all the usual assumptions about errors in a simple linear regression model. We often look at the order of an autoregression as the number of preceding values in the series used to predict the value now. So, y_t regressed on y_t-1 is a first-order autoregression, which is written as AR(1).

In a multiple linear regression, the output of the regression is a linear combination of multiple input variables. In autoregression models, the output is the future data point expressed as a linear combination of the past p data points. p is the number of lags included in the equation. An AR(1) model is defined mathematically as:

 $x_t=\delta +{\varphi }_1{x}_{t-1}+{\alpha }_t$

x_t-1 is the past series value from one lag back

ϕ is the calculated coefficient for that lag

Alpha_t is white noise (such as randomness)

Delta is defined as

 $\delta =(1-\sum _p^{i=1}{\varphi }_i)\mu$

for an autoregressive model of order p, where p is the total number of covariates calculated for lags and μ is the process mean.

When more lags are added to the model, we add more coefficients and lag variables to the equation:

 $x_t=\delta +{\varphi }_1{x}_{t-1}+{\varphi }_2{x}_{t-2}+{\alpha }_t$

The preceding model is a second-order autoregression since it contains two lags.

The general form of an autoregressive equation for an order p is

 $x_t=\delta +{\varphi }_1{x}_{t-1}...{\varphi }_p{x}_{t-p}+{\alpha }_t$

To use autoregressive models for time-series forecasting we use the current time value and any historical data to predict the next time step. For instance, an AR model with 2 lags might predict a single time step forward like so:

 $x_{t+1}=\delta +{\varphi }_1{x}_t+{\varphi }_2{x}_{t-1}+{\alpha }_{t+1}$

The most common approaches to calculating the coefficients for each lag are either maximum likelihood estimation (MLE) or estimation that uses least squares (OLS). The same limitations that these approaches have when fitting a regression of a linear model are present when fitting autoregressive models as well. Depending on whether you're using Python or R and the library, you might be able to use the Yule-Walker or Burg methods in addition to MLE or OLS.

Many libraries allow users to select which criteria to use when selecting models from all the candidate models. For instance, you might want to use the model coefficients to minimize the Aikike Information Criterion or the Bayesian Information Criteria depending on your use case and data.

Autocorrelation calculates the correlation between a time series and a lagged version of itself. The lag is the number of time units to shift the time series. A lag of 1 compares the series with one previous time step. A lag of 2 compares it with the time step before that one. The degree of autocorrelation at a particular lag shows the temporal dependence of the data. Where the autocorrelation is high, there is a strong relationship between the current value and the value at that lag. Where the autocorrelation is low or close to zero it suggests a weak relationship or no relationship at all.

A common approach to visualize autocorrelation is by calculating the autocorrelation function (ACF) or ACF plot that displays the autocorrelation coefficients at different lags. 

The horizontal axis represents the lag, and the vertical axis represents the autocorrelation values. Significant peaks or patterns in the ACF plot can reveal the underlying temporal structure of the data. The selection of the lag order (p) in the AR model often relies on the analysis of the ACF plot. In an AR(p) model, the current value of the time series is expressed as a linear combination of its past p values, with coefficients determined through OLS or MLE. Autocorrelation is also used to assess whether a time series is stationary. For a stationary time series, the autocorrelation should gradually decrease as the lag increases but if the ACF plot doesn't indicate a decrease, the data might contain nonstationarity. You can learn more about autocorrelation here.