Engee documentation
Notebook

Time domain analysis and processing with EngeeDSP

We are getting acquainted with the functionality of EngeeDSP using the example of time domain signal analysis and processing.

Time domain signal analysis and processing is an approach to working with signals in which we consider its basic parameters that change over time. The analysis helps to extract important numerical characteristics from the "raw" signal.:

  1. Amplitude analysis: peak value, average value (constant component), RMS value, etc.

  2. Time analysis: signal duration, pulse repetition rate, front rise rate.

  3. Waveform: distortion analysis, presence of "spikes" (outliers), attenuation, etc.

Processing allows you to extract information from a signal, change its characteristics and statistical indicators, scale it by level or smooth its shape.

The purpose of this example is to visualize the signal recorded from a real sensor, perform preprocessing, evaluate statistical metrics, local extremes, and cut off unwanted outliers.

Signal import and visualization

The data from the sensor's ADC is stored in a file data.txt. We consider them as a matrix in a variable datamat:

In [ ]: 
using DelimitedFiles
datamat = readdlm("data.txt")
Out[0]:
10000×2 Matrix{Float64}:
 0.0     0.0
 0.001   0.0
 0.002   0.0
 0.003   0.0
 0.004   0.0
 0.005   0.0
 0.006   0.0
 0.007   0.0
 0.008   0.0
 0.009   0.0
 0.01    0.0
 0.011   0.0
 0.012   0.0
 ⋮      
 9.988  -0.769358
 9.989  -0.764874
 9.99   -0.761144
 9.991  -0.76214
 9.992  -0.764418
 9.993  -0.763871
 9.994  -0.766009
 9.995  -0.763577
 9.996  -0.763095
 9.997  -0.764831
 9.998  -0.766074
 9.999  -0.767068

Let's extract separate vectors for time and "raw" values from it, and display them on a graph in the time domain.:

In [ ]: 
t = datamat[:,1];
original = datamat[:,2];
plot(t, original, xguide = "Время (с)", title = "Исходный сигнал", legend = false)
Out[0]:

Removing the permanent component

The analysis of statistical metrics can be hindered by the "drift" of the signal - a smooth change in its constant component with relatively fast oscillations around it.

Let's get rid of the constant component using the function detrend from the set of functions EngeeDSP. Under the hood of the function, a method is used to approximate the signal values by a polynomial of an arbitrary order, and then the calculated polynomial is subtracted from the signal. In our case, a 12th-order polynomial is suitable.:

In [ ]: 
sig = EngeeDSP.Functions.detrend(original, 12)
plot(t, sig, xguide = "Время (с)", title = "Сигнал без постоянной составляющей", legend = false)
Out[0]:

Signal Statistics

Now that the signal is oscillating around zero, we can calculate the statistics we are interested in.

Consider the EngeeDSP functions for determining such "popular" signal metrics as extremes, RMS, and variance. But first, let's estimate the sampling period of the signal.:

In [ ]: 
dt = EngeeDSP.Functions.mean(diff(t))
Out[0]:
0.001

And the sampling rate of the signal:

In [ ]: 
fs = 1/dt
Out[0]:
1000.0

Let's define the duration of the discrete signal - the number of samples in the vector sig:

In [ ]: 
nsamples = length(sig)
Out[0]:
10000

Let's find the maximum value and the ordinal number of the maximum reference in the numerical vector:

In [ ]: 
maxval, maxidx = EngeeDSP.Functions.max(sig)
Out[0]:
(M = 0.21000776480728567, I = 3256)

The same is true for the minimum value. These are the absolute extremes of the signal:

In [ ]: 
minval, minidx = EngeeDSP.Functions.min(sig)
Out[0]:
(M = -0.18815908552773913, I = 3254)

Knowing them, you can determine the magnitude of the signal.:

In [ ]: 
range = abs(maxval - minval)
Out[0]:
0.3981668503350248

Now let's calculate the arithmetic mean (note that it is close to zero in value):

In [ ]: 
EngeeDSP.Functions.mean(sig)
Out[0]:
1.7684215789515178e-10

Let's find the RMS value:

In [ ]: 
rmsval = EngeeDSP.Functions.rms(sig)
Out[0]:
0.021745540426125472

And the variance:

In [ ]: 
EngeeDSP.Functions.var(sig)
Out[0]:
(V = 0.0004729158200062578, M = 1.7684215789515178e-10)

Signal peak analysis

It is often useful to be able to find local signal extremes, or so-called peaks. To do this, use the function findpeaks. The function returns the values and indices of peaks, as well as (optionally) their width and severity. As additional input arguments, specify the minimum peak height and the minimum distance between neighboring peaks.:

In [ ]: 
pks, locs, w, p = EngeeDSP.Functions.findpeaks(sig, out=:data, 
                                            MinPeakHeight = 0.08,
                                            MinPeakDistance = 500);
plot(t,sig, xguide = "Время (с)", label = false)
scatter!(t[locs], pks, label = "Пики")
Out[0]:

We will output the amplitudes and their corresponding time points programmatically. We will also output the width and severity:

In [ ]: 
hcat(t[locs], pks, w, p)
Out[0]:
6×4 Matrix{Float64}:
 3.255  0.210008   2.29612  0.348882
 4.661  0.107894   3.94602  0.237671
 6.835  0.143331   2.1476   0.277283
 8.363  0.0834394  3.75173  0.176231
 8.909  0.11189    3.10714  0.208709
 9.651  0.0939499  5.35202  0.205565

Running Median Filter

In the time domain, you can apply basic operations of averaging or trimming the signal using simple sliding window filters - averaging, median, and so on.

Let's get rid of the peaks using a nonlinear filter, namely, a running median with a window of fifteen samples. To do this, use the function movmedian:

In [ ]: 
nospikes = EngeeDSP.Functions.movmedian(sig, 15);
plot(t,sig, xguide = "Время (с)", label = "До фильтра")
plot!(t, nospikes, linewidth = 2.5, label = "После фильтра")
Out[0]:

Let's check the statistical metrics of the filtered signal:

In [ ]: 
EngeeDSP.Functions.rms(nospikes)
Out[0]:
0.019490707144526026
In [ ]: 
EngeeDSP.Functions.var(nospikes)
Out[0]:
(V = 0.00037992459672715685, M = -3.256879171607716e-5)
In [ ]: 
newmax, maxidx = EngeeDSP.Functions.max(nospikes);
newmin, minidx = EngeeDSP.Functions.min(nospikes);
new_range = abs(newmax - newmin)
Out[0]:
0.12617180148957607

Conclusion

We got acquainted with the EngeeDSP functionality for tasks of visualization, analysis and processing of signals in the time domain. Using the described functions, you can:

  • Measure the basic parameters of the signal (amplitude, duration).

  • Visualize the waveform.

  • Clean the signal from noise and interference using simple operations (averaging, filtering).

  • Prepare the signal for deeper analysis, for example, to transition to the frequency domain using the Fourier transform.