Time Series Processes¶

Time Series¶

Observation of random variable ordered by time

Time series variable can be

• Time series at level (absolute value)
• Difference series (relative value)
  • First order difference \(\Delta y_t = y_t - y_{t-1}\)
  • Called as ‘returns’ in finance
  • Second order difference \((\Delta y_t)_2 = \Delta y_t - \Delta y_{t-1}\)

Univariate Time Series¶

Basic model only using a variable’s own properties like lagged values, trend, seasonality, etc

Why do we use different techniques for time series?¶

This is due to

• behavioral effect
• history/memory effect
  • Medical industry always looks at the records of your medical history
• Inertia of change
• Limited data

Components of Time Series Processes¶

Auto-correlation¶

Sometimes just auto-correlation is enough to learn the values of a value

If we take \(j\) lags,

Generally, \(i>j \implies \beta_i < \beta_j\)

Impact of earlier lags is lower than impact of recent lags

Shock¶

‘Shock’ is an abrupt/unexpected deviation(inc/dec) of the value of a variable from its expected value

This incorporates influence of previous disturbance

They cause a structural change in our model equation. Hence, we need to incorporate their effect.

Basically, shock is basically \(u_t\) but it is fancily called as a shock, because they are large \(u\)

Can be

Model becomes

Structural Breaks¶

Permanent change in the variable causes permanent change in relationship

We can either use

• different models before and after structural break
• binary ‘structural dummy variable’ to capture this effect

For eg, long-term injury

Trend¶

Tendency of time series to change at a certain expected rate.

Trend can be

• deterministic/systematic (measurable)
• random/stochastic (not measurable \(\implies\) cannot be incorporated)

For eg: as age increases, humans have a trend of

• growing at a certain till the age of 20 or so
• reducing heart rate

Model becomes

Seasonality/Periodicity¶

Tendency of a variable to change in a certain manner at regular intervals.

For eg

• demand for woolen clothes is high every winter
• demand for ice cream is high every summer

Finance industry has ‘anomalies’

Two ways to encode

Type	Smooth	Example	\(S\)
Binary	❌	Dummy	\(\{0, 1\}\)
Continuous	✅	Cyclic Linear Basis	\(\exp{\left[\dfrac{- 1}{2 \alpha^2} (x_i - \text{pivot})^2\right]}\) - Pivot is the center of the curve	Preferred, as more control over amplitude and bandwidth
	✅	Fourier series	\(\alpha \cos \left(\dfrac{2 \pi}{\nu} t + \phi \right) + \beta \sin \left( \dfrac{2 \pi}{\nu} t + \phi\right)\) Usually \(\alpha, \beta = 1\) \(\nu =\) Frequency of seasonality - Quarterly = 4 - Monthly = 12 - Daily = 365.25 \(\phi\) is the offset

Volatility¶

Annualized standard deviation of change of a random variable

Measure of variation of a variable from its expected value

If the variance is heteroskedastic (changes over time), the variable is volatile

Lag Terms¶

Here, \(\rho_1\) and \(\rho_2\) are partial-autocorrelation coefficient of \(y_{t-1}\) and \(y_{t-2}\) on \(y_t\)

Here, \(\rho_1\) is total autocorrelation coefficient of \(y_{t-2}\) on \(y_t\)

We choose the number of lags by trial-and-error and checking which coefficients are significant (\(\ne 0\))

Interactions¶

• trend-level
• trend-autocorrelation
• trend-seasonality
• seasonality-level
• seasonality-autocorrelation

Stochastic Data-Generating Processes¶

Stochastic process is a sequence of random observations indexed by time

Markov Chain¶

Stochastic process where effect of past on future is summarized only by current state $$ P(y_{t+1} = a \vert x_0, x_1, \dots x_t) = P(x_{t+1} = a \vert x_t) $$ If possible values of \(x_i\) is a finite set, MC can be represented as a transition probability matrix

Martingale¶

Stochastic processes which are a “fair” game $$ E[y_{t+1} \vert y_t] = y_t $$ Follow optimal stopping theorem

Subordinated¶

Stationarity¶

Distribution of \(y_t\) can be inspected with rolling statistics

Types of Stochastic Processes¶

Consider \(u_t = N(0, \sigma^2)\)

Impulse Response Function of covariance stationary process \(y_t\) is $$ \begin{aligned} \text{IR}(j) &= \dfrac{\partial y_t}{\partial \eta_{t-j}} \ &= \psi_j \ \implies \sum \text{IR}(j) &= \phi(L), \text{with L=}1 \ &\text{ (L is lag operator)} \end{aligned} $$

Differentiation¶

When converting a non-stationary series \(y_t\) into a stationary series \(y'_t\), we want

• Obtain stationarity: ADF Stat at 95% CL as \(-2.8623\)
• Retain memory: Similarity to original series; High correlation b/w original series and differentiated series

Fractional Differencing/Un-differencing¶

To un-difference and approximately recover the original data:

Pre-Requisites¶

1. A fixed-sized window is used for differencing
   1. Fractional differencing using a fixed window only considers a certain number (say, w) of past data points to compute the differenced value at each time step
   2. Unlike standard integer differencing, fractional differencing uses a weighted sum of previous values, with the weights decaying over history but not reaching zero until you hit a threshold.
2. The first window of original data points is saved
   1. This means that to reconstruct (un-difference) a value at time t, you need the w most recent original (pre-differenced) or already reconstructed data points, starting with the first window.

Method¶

Practical Steps: How Fractional Un-differencing Can Be Done¶

1. During differencing:
   • Apply fractional differencing with a fixed window length w; store the first w-1 original values (these are the boundary values you'll need later).
2. To un-difference:
   • For each point after the initial window, set up the equation represented by the differencing operation (i.e., the original is a weighted sum of the previous w-1 originals and the differenced value).
   • Solve this recursively using the saved initial values.
   • This is analogous to "unsmoothing", where a moving-average (or filter) can be inverted as long as enough initial data is available to resolve the system

import numpy as np
from scipy.signal import lfilter
from fracdiff import fdiff

def is_integer(d, tol=1e-10):
    """Check if d is effectively an integer within floating point tolerance."""
    return abs(d - round(d)) < tol

def fracdiff_weights(d, N):
    """
    Compute fractional differencing weights for given order d and window size N.
    Correctly handles all real d, including negative and integer.
    """
    if is_integer(d):
        w = np.zeros(N)
        w[round(d)] = (-1)**round(d)
        return w
    else:
        k = np.arange(1, N)
        mult = -(d - k + 1) / k
        w = np.empty(N)
        w[0] = 1.0
        w[1:] = np.cumprod(mult)
        return w


def unfracdiff(y, d, x0=None, window=10):
    """
    Reconstruct the original series from a fractionally (or integer) differenced series y.

    Parameters:
        y: differenced series
        d: differencing order (can be int or float)
        x0: initial value(s) (scalar or array of size d for integer or window-1 for fractional)
        window: used for fractional differencing only

    Returns:
        Reconstructed series x
    """
    N = len(y)

    if is_integer(d):
        d = int(round(d))
        if x0 is None:
            x0 = np.zeros(d)
        x = y.copy()
        for j in range(d):
            x = np.cumsum(x)
        return x

    else:
        # Fractional case (non-integer)
        if x0 is None:
            raise ValueError("Initial values x0 (length=window-1) must be provided for fractional d.")
        w = fracdiff_weights(d, window)
        a = w
        b = [1.0]
        # Pad y with initial values to simulate correct lfilter history
        y_padded = y.copy()
        # Apply inverse filter
        x_reconstructed = lfilter(b, a, y_padded)
        # Replace the initial window values with original x0
        x_reconstructed[:window-1] = x0
        return x_reconstructed


# Example full working test
import numpy as np
from fracdiff import fdiff

x = np.random.random(10)
d = 0.5                     # Can be integer (1, 2, ...) or fractional (0.5, 0.25, ...)
window = 10

# Differencing
if is_integer(d):
    x_shifted = x.copy()
    for i in range(d):
      x_shifted = np.diff(x_shifted, n=1, prepend=[0])
    y = x_shifted.copy()
    x0 = x[:int(d)]
else:
    y = fdiff(x, n=d, window=window)
    x0 = x[:window-1]  # Save initial window for un-fractional differencing

# Un-differencing
x_reconstructed = unfracdiff(y, d, x0=x0, window=10)

print("Original x:      ", x.round(1))

temp = np.ceil(d).astype(int)
print("Diff: ", np.concatenate(
    ([np.nan]*temp, y.round(1)[temp:])
))
print("Reconstructed x:", x_reconstructed.round(1))
print()
print("Difference:", (x - x_reconstructed).round(1))

Integrated/DS Process¶

Difference Stationary Process

A non-stationary series is said to be integrated of order \(k\), if mean and variance of \(k^\text{th}\)-difference are time-invariant

If the first-difference is non-stationary, we take second-difference, and so on

Pure random walk is DS¶

Random walk w/ drift is DS¶

TS Process¶

Trend Stationary Process

A non-stationary series is said to be …, if mean and variance of de-trended series are time-invariant

Assume a process is given by

where trend is deterministic/stochastic

Then

• Time-varying mean
• Constant variance ???

We perform de-trending \(\implies\) subtract \((\beta_0 + \beta_1 t)\) from \(y_t\)

If

• \(\beta_2 = 0\), the de-trended series is white noise process
• \(\beta_2 \ne 0\), the de-trended series is a stationary process

Note Let’s say \(y_t = f(x_t)\)

If both \(x_t\) and \(y_t\) have equal trends, then no need to de-trend, as both the trends will cancel each other

Unit Root Test for Process Identification¶

Augmented Dicky-Fuller Test¶

• \(H_0: \beta_1=1\)
• \(H_0: \beta_1 \ne 1\)

Alternatively, subtract \(y_{t-1}\) on both sides of main equation

• \(H_0: \gamma=1\) (Non-Stationary)
• \(H_1: \gamma \ne 1\) (Stationary)

If p value \(\le 0.05\)

• we reject null hypothesis and accept alternate hypothesis
• Hence, process is stationary

We test the hypothesis using Dicky-Fuller distribution, to generate the critical region

Long memory series¶

Earlier past is as important as recent past

Q Statistic¶

Test statistic like \(z\) and \(t\) distribution, which is used to test ‘joint hypothesis’

Inertia of Time Series Variable¶

Persistance of value due to Autocorrelation

Today’s exchange rate is basically yesterday’s exchange rate, plus-minus something

Comments