Predicting multidimensional time series using a hybrid method

Introduction

In time series analysis and forecasting, the classical problem remains the separation of the observed signal into deterministic and stochastic components. If the deterministic part allows for an accurate parametric description, then the remainder, as a rule, contains a correlated structure that is not reduced to simple noise, but is also not described by linear methods. Hybrid schemes combining classical decomposition with neural network modeling of the remainder make it possible to use the strengths of both methods.

In this paper, we consider a hybrid method for predicting multidimensional time series, which includes three stages:

1. Least Squares Linear trend Removal (OLS);
2. Selection of two dominant sinusoidal components with frequency optimization by the Nelder-Mead method;
3. Prediction of the remainder by the recurrent neural network LSTM.

The quality of the forecast is assessed using RMSE and MAE metrics, and is also compared with a model without a neural network block.

Let's consider some definitions used in this work.:

• A trend is a long–term, monotonous component of a time series that reflects the general direction of its change: growth, decline, or stagnation.
• Seasonality is a periodic component of a series that repeats at approximately equal time intervals with a constant amplitude and phase.
• LSTM (Long Short-Term Memory) is a type of recurrent neural network capable of storing information about previous states of a sequence at long intervals. Unlike the classical recurrent neural network, LSTM uses a mechanism that controls the writing, reading and erasing of information in a hidden state, which allows it to effectively model long-term dependencies in time series.
• Residual energy is the sum of the squares of the discrepancies between the signal and its model.: . This is a standard OLS objective function; minimizing it means finding the model parameters at which it best describes the signal. In this paper, the residual energy is used as a criterion for optimizing the frequencies of sinusoids.
• RMS (Root Mean Square, RMS value) – the root of the average square of the value: ​​. For the remainder of the series, RMS shows the typical amplitude of the discrepancy in the same units as the signal. The RMS of the remainder in the training area characterizes the "unexplained" part of the series, which is then transmitted to the LSTM input.
• RMSE (Root Mean Square Error, root of the mean square error) is a metric of forecast quality: ​. It is measured in signal units and penalizes large misses more strongly than MAE.
• MAE (Mean Absolute Error) – metric: . It shows the average "typical" forecast error in signal units and is less sensitive to outliers than RMSE.
• MNC (least squares method) is a method for estimating the parameters of a linear model that minimizes the sum of squared residuals. It is used to estimate the linear trend and the amplitudes of sinusoids.
• The Nelder-Mead method is an unconditional optimization algorithm that does not require derivatives; it is used to refine a pair of sinusoid frequencies by the criterion of residual energy.

Joining libraries

Connecting the necessary libraries:

Flux – neural network training and application;

Optim – optimization of sinusoid frequencies;

Statistics– calculation of remainder statistics for normalization;

LinearAlgebra – solution of MNC systems for calculating linear trend and sinusoid amplitudes;

Random – for reproducibility of learning.

Data generation

We will generate three time series , , 5000 points long.

Each row contains a general trend and the seasonal component with phase shift and constant . For and The amplitude of the overall trend is scaling, and the phase of the seasonal component is shifting.

Let's plot all three rows.

Splitting into training and test samples

Let's split the data into a training sample (the first 4000 points) and a test sample (the last 1000 points), and form a common array. данные_все (5000×3) and the time vector tt. Let's highlight the time axis separately. t_обуч for the training area.

Removing a linear trend

Let's estimate the linear trend using the least squares (OLS) training sample for each row, plot the trend along its entire length and subtract it from the source data. As a result, we get the detrended data. детренд_все and детренд_обуч.

Residual energy function

Define the function остаточная_энергия(частоты, сигнал, t), which, based on a given pair of frequencies, builds a basis of constants, cosines and sines, selects the coefficients of the OLS and returns the sum of the squares of the residuals. This function will be used as a target function for frequency optimization.

Two sinusoids search function

Define the function синусоиды(сигнал, t), which searches for two dominant frequencies in the signal: first, it finds the first frequency by brute force searching through the grid, then, by subtracting its contribution, the second, and then clarifies both frequencies using the NelderMead method. Returns the found pair (f1, f2).

Selection of sinusoids by series

Apply the function синусоиды for each of the three determined series, compare the found frequencies with the true values. and . we will perform the final MNC selection of amplitudes in the training area and construct sinusoidal components along the entire length of the row. We will output RMS for each row.

Normalization of the balance

Calculate the average and the standard deviation the remainder of the training sample, let's replace the ones that are too small by 1.0 , we get the normalized remainder . остаток_норм with zero mean and unit variance for each row.

Creating windows for LSTM

Let's create training sequences for LSTM: the window length is L = 100, for each of the three channels. Getting arrays X_посл forms (3, 100, 3900) – entrance windows, and Y_посл The forms (3, 3900) are the target values.

LSTM Training

Let's define an LSTM model (input 3 → hidden state 32 → output 3) with a fully connected layer at the output. Let's set up the Adam optimizer and train the model with 10 epochs in mini-batches of 64 examples each, minimizing the MSE between the prediction at the last step of the window and the true value. We will output the value of the loss function for each epoch.

Balance forecast

Define the function прогноз_остатка, which recursively predicts the remainder by N_тест steps forward: at each step, the last window of length is taken L, is fed to the LSTM, the received prediction is added to the buffer and used to form the next window. Then we denormalize the forecast back to the original scale using and . As a result, we get остаток_полн the dimension is 5000×3.

Final forecast

Let's add up the three components: a linear trend тренд_все, the sinusoidal componentсинус_все and the forecast of the balance остаток_полн. We will get the full forecast y_полн and let's highlight the test part y_прогноз. We will save the true values for comparison in y_истина.

Quality metrics

Let's calculate the RMSE and MAE metrics for each of the three rows in the test area. Additionally, we will calculate the forecast without taking into account the LSTM for comparison. It will show how much LSTM improves the quality of the forecast.

Visualization of the forecast

Let's build three subgraphs (, , ), on each training area in blue, the true values on the test are dotted in orange and the forecast is red.

Conclusion

In this paper, a hybrid method for predicting multidimensional time series is constructed and investigated, combining OLS decomposition with LSTM modeling of the remainder. The proposed method showed high quality: the prediction error in the test section was ≈ 2% of the signal range (RMSE ≈ 0.38–0.52 with a range of ≈ 25 units), and compared with the model without a neural network unit, the accuracy improved 3.5–4.5 times.

Hybrid schemes are in demand in a wide range of applications: forecasting energy consumption and load of power grids, demand and sales, financial and product ranges, meteorological and climatic indicators, parameters of technological processes and telemetry, traffic and logistics. Classical decomposition provides interpretability and stability to small samples, while LSTM takes into account non–linear and long-term dependencies in the remainder.