Detection performance analysis using detection characteristics (Part 1)

The example demonstrates how the performance of coherent and incoherent systems can be evaluated using plots of receiver detection curves (ROC curves). The example assumes that the detector operates under additive complex white Gaussian noise.

ROC curves are often used to evaluate the performance of a radar or sonar detector. ROC curves are plots of the probability of detection (Pd) versus the probability of false alarm (Pfa) for a given signal-to-noise ratio (SNR).

Functions used

Pkg.add(["Statistics"])

using SpecialFunctions,Statistics

Introduction

Pd is the probability of stating that 1 is true if event 1 has occurred. Pfa is the probability of stating that 1 is true if event 0 has occurred. In applications such as sonar and radar, event 1 means that a target is present and event 0 means that no target is present.

Single Pulse Detection

Given an SNR value, you can calculate the Pd and Pfa values that a linear or quadratic detector can achieve with a single pulse. Set the Pd value assuming an SNR of 8 dB and requirements dictate a Pfa value of no more than 1%. You can use the rocsnr function to calculate the Pd and Pfa values, and then determine which Pd value corresponds to Pfa = 0.01. Note that by default the rocsnr function assumes coherent detection.

Pd,Pfa = rocsnr(8,"Pd&Pfa")

# нахождения номера элемента для Pfa=0.01
idx = findfirst(round.(Pfa;sigdigits=16).==0.01) 
print("idx = $(idx)")

idx = 81

Using the number defined above, the Pd value corresponding to Pfa = 0.01 can be found.

Pd[idx]
print("Pd[idx] = $(round(Pd[idx];sigdigits=5))")

Pd[idx] = 0.8899

One of the features of the rocsnr function is that you can specify a vector of SNR values and the function will calculate a ROC curve for each of these SNR values. Instead of calculating the Pd and Pfa values for a given SNR separately, you can view the results as a graph of the ROC curves. The rocsnr function plots ROC curves by default if no output arguments are specified. Calling the rocsnr function with an input vector of four SNR values and no output arguments produces a ROC curve plot.

SNRvals = [2 4 8 9.4]
rocsnr(SNRvals)

In the plot, select with the cursor the curve SNR = 8 dB at the point where Pd = 0.9. You can see that Pfa is equal to the reported value and is approximately 0.01.

Detecting multiple pulses

One way to improve detector performance is to average over multiple pulses. This is particularly useful when the signal of interest is known and occurs in additive complex white noise. Although this method is applicable to both linear and quadratic detectors, the result for quadratic detectors can be biased by about 0.2 dB. Analyse the performance by taking the SNR as 8 dB and averaging over the two pulses.

rocsnr(8;NumPulses=2)

The graph shows that averaging over the two pulses resulted in a higher probability of detection for a given false alarm rate. With an SNR of 8 dB and averaging over two pulses, you can limit the false alarm probability to no more than 0.0001 and get a probability of detection of 0.9. Recall that for a single pulse, the false alarm probability had to be 1% to achieve the same detection probability.

Non-coherent detection

At this point, the example considers a known signal in complex white Gaussian noise. The default rocsnr function assumes a coherent detector. To analyse the detector performance for the case where the signal is known except for phase, an incoherent detector can be specified. Using the same SNR values as before, analyse the performance of the incoherent detector.

rocsnr(SNRvals,SignalType="NonfluctuatingNoncoherent")

Focus on the ROC curve corresponding to an SNR of 8 dB. By inspecting the graph with the data cursor, you will see that to achieve a detection probability of 0.9, you must allow the false alarm probability to be as low as 0.05. Without using phase information, you will need a higher SNR to achieve the same Pd for a given Pfa. For incoherent linear detectors, use the Albersham equation to determine what SNR value will provide the desired Pd and Pfa.

SNR_valdB = albersheim(0.9,.01) # Pd=0.9 and Pfa=0.01
print("SNR_valdB = $(round(SNR_valdB;sigdigits=5)) дБ")

SNR_valdB = 9.5027 дБ

By plotting the ROC curve for the SNR value approximated by the Albersham equation, it can be seen that the detector will achieve Pd = 0.9 and Pfa = 0.01.Note that the Albersham technique is only applicable to incoherent detectors.

rocsnr(SNR_valdB,SignalType="NonfluctuatingNoncoherent")

Detection of fluctuating targets

In all of the above reasoning, it is assumed that the target does not fluctuate, that is, the statistical characteristics of the target do not change over time.However, in real-world scenarios, targets can accelerate and decelerate, as well as roll and tilt. These factors cause the radar cross section (RCS) of the target to change with time. A set of statistical models called Sverling models are often used to describe the random variation in a target's RCS.

There are four Drilling models, namely Drilling 1 - 4. A non-vibrating target is often referred to as Drilling 0 or Drilling 5. Each Drilling model describes how the target's RCS changes over time and the probability distribution of that change.

Because a target's RCS changes, the ROC curves for fluctuating targets are not the same as those for non-fluctuating targets. In addition, because Drilling targets add random phase to the received signal, it is more difficult to use a coherent detector for Drilling targets. Therefore, incoherent detection methods are often used for Drilling targets.

Now compare the ROC curves for an incoherent target and a Swerling 1 target. In particular, you want to find out what the SNR requirements are for both situations if you want to achieve the same Pd and Pfa. For this comparison, it is often convenient to plot the ROC curve as a function of Pd versus SNR while varying Pfa. We can use the rocpfa function to plot the ROC curve in this form.

Suppose the detection is incoherent, with 10 integrated pulses, and the desired value of Pfa is no more than 1e-8. First plot the ROC curve for the incoherent target.

rocpfa(1e-8,NumPulses=10,SignalType="NonfluctuatingNoncoherent")

Then plot the ROC curve for the Swerling 1 target for comparison.

rocpfa(1e-8,NumPulses=10,SignalType="Swerling1")

You can see from the figures that a Pd of 0.9 requires an SNR of about 6 dB if the target does not oscillate. However, if the target is a Swerling case 1 model, the required SNR increases to over 14 dB, which is a difference of 8 dB. This will greatly affect the design of the system.

As with non-swerling targets, there are approximation equations to determine the required SNR without having to plot all the curves. The equation used for fluctuating targets is the Schnidman equation. For the scenario used to construct the ROC curves, the SNR requirements can be derived using the shnidman function.

snr_sw1_db = shnidman(0.9,1e-8,10,1)  # Pd=0.9, Pfa=1e-8, 10 pulses
print("snr_sw1_db = $(round(snr_sw1_db;sigdigits=5)) дБ")

snr_sw1_db = 14.713 дБ

The calculated SNR value corresponds to the value obtained from the curve.

Conclusion

ROC curves are useful for analysing detector efficiency for both coherent and incoherent systems. In this example, the rocsnr function was used to analyse the performance of a linear detector at different SNR values. The improvement in detector performance achieved by averaging multiple samples was also examined. Finally, the example shows how the rocsnr and rocpfa functions can be used to analyse detector performance when using an incoherent detector for incoherent and fluctuating targets.