Filtering functions

Functions

# ImageFiltering.imfilter — Function

imfilter([T], img, kernel, [border="replicate"], [alg]) --> imgfilt
imfilter([r], img, kernel, [border="replicate"], [alg]) --> imgfilt
imfilter(r, T, img, kernel, [border="replicate"], [alg]) --> imgfilt

Filter a one, two or multidimensional array img with a kernel by computing their correlation.

Extended help

Choices for r

Optionally, you can dispatch to different implementations by passing in a resource r as defined by the ComputationalResources package.

For example:

imfilter(ArrayFireLibs(), img, kernel)

would request that the computation be performed on the GPU using the ArrayFire libraries.

Choices for T

Optionally, you can control the element type of the output image by passing in a type T as the first argument.

Choices for img

You can specify a one, two, or multidimensional array defining your image.

Choices for kernel

The kernel[0, 0,..] parameter corresponds to the origin (zero displacement) of the kernel; you can use centered to place the origin at the array center, or use the OffsetArrays package to set kernel's indices manually. For example, to filter with a random centered 3x3 kernel, you could use either of the following:

kernel = centered(rand(3,3))
kernel = OffsetArray(rand(3,3), -1:1, -1:1)

The kernel parameter can be specified as an array or as a "factored kernel", a tuple (filt1, filt2, ...) of filters to apply along each axis of the image. In cases where you know your kernel is separable, this format can speed processing. Each of these should have the same dimensionality as the image itself, and be shaped in a manner that indicates the filtering axis, e.g., a 3x1 filter for filtering the first dimension and a 1x3 filter for filtering the second dimension. In two dimensions, any kernel passed as a single matrix is checked for separability; if you want to eliminate that check, pass the kernel as a single-element tuple, (kernel,).

Choices for border

At the image edge, border is used to specify the padding which will be used to extrapolate the image beyond its original bounds.

"replicate" (default)

The border pixels extend beyond the image boundaries.

   ╭────┏━━━━━━┓────╮
   │aaaa┃abcdef┃ffff│
   ╰────┗━━━━━━┛────╯

"circular"

The border pixels wrap around. For instance, indexing beyond the left border returns values starting from the right border.

   ╭────┏━━━━━━┓────╮
   │cdef┃abcdef┃abcd│
   ╰────┗━━━━━━┛────╯

"reflect"

The border pixels reflect relative to a position between pixels. That is, the border pixel is omitted when mirroring.

   ╭────┏━━━━━━┓────╮
   │dcba┃abcdef┃fedc│
   ╰────┗━━━━━━┛────╯

"symmetric"

The border pixels reflect relative to the edge itself.

   ╭────┏━━━━━━┓────╮
   │edcb┃abcdef┃edcb│
   ╰────┗━━━━━━┛────╯

Fill(m)

The border pixels are filled with a specified value .

   ╭────┏━━━━━━┓────╮
   │mmmm┃abcdef┃mmmm│
   ╰────┗━━━━━━┛────╯

Inner()

Indicate that edges are to be discarded in filtering, only the interior of the result is to be returned.

NA()

Choose filtering using "NA" (Not Available) boundary conditions. This is most appropriate for filters that have only positive weights, such as blurring filters.

See also: Pad, padarray, Inner, NA and NoPad

Choices for alg

The alg parameter allows you to choose the particular algorithm: Algorithm.FIR() (finite impulse response, aka traditional digital filtering) or Algorithm.FFT() (Fourier-based filtering). If no choice is specified, one will be chosen based on the size of the image and kernel in a way that strives to deliver good performance. Alternatively you can use a custom filter type, like KernelFactors.IIRGaussian.

# ImageFiltering.imfilter! — Function

imfilter!(imgfilt, img, kernel, [border="replicate"], [alg])
imfilter!(r, imgfilt, img, kernel, border::Pad)
imfilter!(r, imgfilt, img, kernel, border::NoPad, [inds=axes(imgfilt)])

Filter an array img with kernel kernel by computing their correlation, storing the result in imgfilt.

The indices of imgfilt determine the region over which the filtered image is computed---you can use this fact to select just a specific region of interest, although be aware that the input img might still get padded. Alteratively, explicitly provide the indices inds of imgfilt that you want to calculate, and use NoPad boundary conditions. In such cases, you are responsible for supplying appropriate padding: img must be indexable for all of the locations needed for calculating the output. This syntax is best-supported for FIR filtering; in particular, that that IIR filtering can lead to results that are inconsistent with respect to filtering the entire array.

Kernel

# ImageFiltering.Kernel — Module

Kernel is a module implementing filtering (correlation) kernels of full dimensionality. The following kernels are supported:

sobel
prewitt
ando3, ando4, and ando5
scharr
bickley
gaussian
DoG (Difference-of-Gaussian)
LoG (Laplacian-of-Gaussian)
Laplacian
gabor
moffat

KernelFactors

# ImageFiltering.KernelFactors — Module

KernelFactors is a module implementing separable filtering kernels, each stored in terms of their factors. The following kernels are supported:

box
sobel
prewitt
ando3, ando4, and ando5 (the latter in 2d only)
scharr
bickley
gaussian
IIRGaussian (approximate gaussian filtering, fast even for large σ)

Kernel utilities

# OffsetArrays.center — Function

center(A, [r::RoundingMode=RoundDown])::Dims

Return the center coordinate of given array A. If size(A, k) is even, a rounding procedure will be applied with mode r.

Compatibility: OffsetArrays 1.9

This method requires at least OffsetArrays 1.9.

Examples

julia> A = reshape(collect(1:9), 3, 3)
3×3 Matrix{Int64}:
 1  4  7
 2  5  8
 3  6  9

julia> c = OffsetArrays.center(A)
(2, 2)

julia> A[c...]
5

julia> Ao = OffsetArray(A, -2, -2); # axes (-1:1, -1:1)

julia> c = OffsetArrays.center(Ao)
(0, 0)

julia> Ao[c...]
5

To shift the center coordinate of the given array to (0, 0, ...), you can use centered.

# OffsetArrays.centered — Function

centered(A, cp=center(A)) -> Ao

Shift the center coordinate/point cp of array A to (0, 0, ..., 0). Internally, this is equivalent to OffsetArray(A, .-cp).

Compatibility: OffsetArrays 1.9

This method requires at least OffsetArrays 1.9.

Examples

julia> A = reshape(collect(1:9), 3, 3)
3×3 Matrix{Int64}:
 1  4  7
 2  5  8
 3  6  9

julia> Ao = OffsetArrays.centered(A); # axes (-1:1, -1:1)

julia> Ao[0, 0]
5

julia> Ao = OffsetArray(A, OffsetArrays.Origin(0)); # axes (0:2, 0:2)

julia> Aoo = OffsetArrays.centered(Ao); # axes (-1:1, -1:1)

julia> Aoo[0, 0]
5

Users are allowed to pass cp to change how "center point" is interpreted, but the meaning of the output array should be reinterpreted as well. For instance, if cp = map(last, axes(A)) then this function no longer shifts the center point but instead the bottom-right point to (0, 0, ..., 0). A commonly usage of cp is to change the rounding behavior when the array is of even size at some dimension:

julia> A = reshape(collect(1:4), 2, 2) # Ideally the center should be (1.5, 1.5) but OffsetArrays only support integer offsets
2×2 Matrix{Int64}:
 1  3
 2  4

julia> OffsetArrays.centered(A, OffsetArrays.center(A, RoundUp)) # set (2, 2) as the center point
2×2 OffsetArray(::Matrix{Int64}, -1:0, -1:0) with eltype Int64 with indices -1:0×-1:0:
 1  3
 2  4

julia> OffsetArrays.centered(A, OffsetArrays.center(A, RoundDown)) # set (1, 1) as the center point
2×2 OffsetArray(::Matrix{Int64}, 0:1, 0:1) with eltype Int64 with indices 0:1×0:1:
 1  3
 2  4

Boundaries and padding

# ImageFiltering.BorderArray — Type

BorderArray(inner::AbstractArray, border::AbstractBorder) <: AbstractArray

Construct a thin wrapper around the array inner, with given border. No data is copied in the constructor, instead border values are computed on the fly in getindex calls. Useful for stencil computations. See also padarray.

Examples

julia> using ImageFiltering

julia> arr = reshape(1:6, (2,3))
2×3 reshape(::UnitRange{Int64}, 2, 3) with eltype Int64:
 1  3  5
 2  4  6

julia> BorderArray(arr, Pad((1,1)))
BorderArray{Int64,2,Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}},Pad{2}} with indices 0:3×0:4:
 1  1  3  5  5
 1  1  3  5  5
 2  2  4  6  6
 2  2  4  6  6

julia> BorderArray(arr, Fill(10, (2,1)))
BorderArray{Int64,2,Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}},Fill{Int64,2}} with indices -1:4×0:4:
 10  10  10  10  10
 10  10  10  10  10
 10   1   3   5  10
 10   2   4   6  10
 10  10  10  10  10
 10  10  10  10  10

# ImageFiltering.Fill — Type

struct Fill{T,N} <: AbstractBorder
    value::T
    lo::Dims{N}
    hi::Dims{N}
end

Fill is a type that designates a particular value which will be used to extrapolate pixels beyond the boundary of an image.

Output

The type Fill specifying the value with which the boundary of the image should be padded.

Details

When representing a two-dimensional spatial image filtering operation as a discrete convolution between an image and a filter, the results are undefined for pixels closer than pixels from the border of the image. To define the operation near and at the border, you need a scheme for extrapolating pixels beyond the edge. The Fill type allows you to specify a particular value which will be used in the extrapolation. For more elaborate extrapolation schemes, see Pad.

The type facilitates the padding of one, two, or multi-dimensional images.

You can specify a different amount of padding at the lower and upper borders of each dimension of the image (top, left, bottom, and right in two dimensions).

Example

To illustrate this, consider an image consisting of a row of six pixels which are specified alphabetically:

    ┏━━━━━━┓
    ┃abcdef┃
    ┗━━━━━━┛

Padding with a constant value m only on the left and right border looks like this:

╭────┏━━━━━━┓────╮
│mmmm┃abcdef┃mmmm│
╰────┗━━━━━━┛────╯

(Analogous consequences hold for the top and bottom border.)

See also: Pad, padarray, Inner and NoPad

# ImageFiltering.Inner — Type

Inner()
Inner(lo, hi)

Indicate that edges are to be discarded in filtering, only the interior of the result is to be returned.

Example:

imfilter(img, kernel, Inner())

# ImageFiltering.NA — Type

NA(na=isnan)

Choose filtering using "NA" (Not Available) boundary conditions. This is most appropriate for filters that have only positive weights, such as blurring filters. Effectively, the output value is normalized in the following way:

          filtered array with Fill(0) boundary conditions
output =  -----------------------------------------------
          filtered 1     with Fill(0) boundary conditions

Array elements for which na returns true are also considered outside array boundaries.

# ImageFiltering.NoPad — Type

NoPad()
NoPad(border)

Indicates that no padding should be applied to the input array, or that you have already pre-padded the input image. Passing a border object allows you to preserve "memory" of a border choice; it can be retrieved by indexing with [].

Example

The commands

np = NoPad(Pad(:replicate))
imfilter!(out, img, kernel, np)

run filtering directly, skipping any padding steps. Every entry of out must be computable using in-bounds operations on img and kernel.

# ImageFiltering.Pad — Type

    struct Pad{N} <: AbstractBorder
        style::Symbol
        lo::Dims{N}    # number to extend by on the lower edge for each dimension
        hi::Dims{N}    # number to extend by on the upper edge for each dimension
    end

Pad is a type that designates the form of padding which should be used to extrapolate pixels beyond the boundary of an image. Instances must set style, a Symbol specifying the boundary conditions of the image.

Output

The type Pad specifying how the boundary of an image should be padded.

Extended help

When representing a spatial two-dimensional image filtering operation as a discrete convolution between the image and a filter, the results are undefined for pixels closer than pixels from the border of the image. To define the operation near and at the border, one needs a scheme for extrapolating pixels beyond the edge. The Pad type allows one to specify the necessary extrapolation scheme.

The type facilitates the padding of one, two or multi-dimensional images.

You can specify a different amount of padding at the lower and upper borders of each dimension of the image (top, left, bottom and right in two dimensions).

Options

Some valid style options are described below. As an indicative example of each option the results of the padding are illustrated on an image consisting of a row of six pixels which are specified alphabetically:

        ┏━━━━━━┓
        ┃abcdef┃
        ┗━━━━━━┛

We show the effects of padding only on the left and right border, but analogous consequences hold for the top and bottom border.

:replicate (Default)

The border pixels extend beyond the image boundaries.

   ╭────┏━━━━━━┓────╮
   │aaaa┃abcdef┃ffff│
   ╰────┗━━━━━━┛────╯

See also: Fill, padarray, Inner and NoPad

:circular

The border pixels wrap around. For instance, indexing beyond the left border returns values starting from the right border.

   ╭────┏━━━━━━┓────╮
   │cdef┃abcdef┃abcd│
   ╰────┗━━━━━━┛────╯

See also: Fill, padarray, Inner and NoPad

:symmetric

The border pixels reflect relative to a position between pixels. That is, the border pixel is omitted when mirroring.

   ╭────┏━━━━━━┓────╮
   │edcb┃abcdef┃edcb│
   ╰────┗━━━━━━┛────╯

:reflect

The border pixels reflect relative to the edge itself.

   ╭────┏━━━━━━┓────╮
   │dcba┃abcdef┃fedc│
   ╰────┗━━━━━━┛────╯

See also: Fill,padarray, Inner and NoPad.

# ImageFiltering.padarray — Function

padarray([T], img, border)

Generate a padded image from an array img and a specification border of the boundary conditions and amount of padding to add.

Return a padded image. The function supports one, two or multi-dimensional images. You can specify the element type T of the output image.

See Pad and Fill for details.

Examples

Padding

The main syntax for Pad is (style, m, n, ...) or (style, (m, n)), where m pixels are added to dimension 1 (top and bottom), n pixels for dimension 2, and so forth.

Add 30 to left and right, 40 to top and bottom:

padarray(A, Pad(:replicate, 30, 40))
padarray(A, Pad(:circular, 30, 40))
padarray(A, Pad(:symmetric, 30, 40))
padarray(A, Pad(:reflect, 30, 40))

Add 30 above, 40 to left, 50 to bottom, 60 to right:

padarray(A, Pad(0, (30, 40), (50, 60)))
padarray(A, Pad(0, (30, 40), (50, 60)))

padarray(A, Pad(:replicate, 1, 1, 1))
padarray(A, Fill(0, (1, 1, 1)))

Filling

The main syntax for Fill is (value, m, n) or (value, (m, n)) where the image is prepended by m pixels and appended by n pixels in each dimension.

Add 20 -1 values above, 30 to left, 40 to bottom, 50 to right:

padarray(A, Fill(-1, (20, 30), (40, 50)))

Find local extrema

# ImageFiltering.findlocalmaxima — Function

findlocalmaxima(img; window=default_window(img), edges=true) -> Vector{CartesianIndex}

Returns the coordinates of elements whose value is larger than all of their immediate neighbors. edges is a Boolean specifying whether to include the first and last elements of each dimension, or a tuple-of-Bool specifying edge behavior for each dimension separately.

The default_window is 3 for each spatial dimension of img, and 1 otherwise, implying that maxima are detected over nearest-neighbors in each spatial "slice" by default.

# ImageFiltering.findlocalminima — Function

findlocalminima(img; window=default_window(img), edges=true) -> Vector{CartesianIndex}

Like findlocalmaxima, but returns the coordinates of the smallest elements.

Algorithms

# ImageFiltering.Algorithm.FIR — Type

Filter using a direct algorithm

# ImageFiltering.Algorithm.FFT — Type

Filter using the Fast Fourier Transform

# ImageFiltering.Algorithm.IIR — Type

Filter with an Infinite Impulse Response filter

# ImageFiltering.Algorithm.Mixed — Type

Filter with a cascade of mixed types (IIR, FIR)

Solvers for predefined models

# ImageFiltering.Models — Module

This submodule provides predefined image-related models and its solvers that can be reused by many image processing tasks.

solve the Rudin Osher Fatemi (ROF) model using the primal-dual method: solve_ROF_PD and solve_ROF_PD!

# ImageFiltering.Models.solve_ROF_PD! — Method

solve_ROF_PD!(out, buffer, img, λ, num_iters)

The in-place version of solve_ROF_PD.

It is not uncommon to use ROF solver in a higher-level loop, in which case it makes sense to preallocate the output and intermediate arrays to make it faster.

!!! note "Buffer" The content and meaning of buffer might change without any notice if the internal implementation is changed. Use preallocate_solve_ROF_PD helper function to avoid potential changes.

Examples

using ImageFiltering.Models: preallocate_solve_ROF_PD

out = similar(img)
buffer = preallocate_solve_ROF_PD(img)
solve_ROF_PD!(out, buffer, img, 0.2, 30)

# ImageFiltering.Models.solve_ROF_PD — Method

solve_ROF_PD([T], img::AbstractArray, λ; kwargs...)

Return a smoothed version of img, using Rudin-Osher-Fatemi (ROF) filtering, more commonly known as Total Variation (TV) denoising or TV regularization. This algorithm is based on the primal-dual method.

This function applies to generic N-dimensional colorant array and is also CUDA-compatible. See also solve_ROF_PD! for the in-place version.

Arguments

T: the output element type. By default it is float32(eltype(img)).
img: the input image, usually a noisy image.
λ: the regularization coefficient. Larger λ results in more smoothing.

Parameters

num_iters::Int: The number of iterations before stopping.

Examples

using ImageFiltering
using ImageFiltering.Models: solve_ROF_PD
using ImageQualityIndexes
using TestImages

img_ori = float.(testimage("cameraman"))
img_noisy = img_ori .+ 0.1 .* randn(size(img_ori))
assess_psnr(img_noisy, img_ori) # ~20 dB

img_smoothed = solve_ROF_PD(img_noisy, 0.015, 50)
assess_psnr(img_smoothed, img_ori) # ~27 dB

# larger λ produces over-smoothed result
img_smoothed = solve_ROF_PD(img_noisy, 5, 50)
assess_psnr(img_smoothed, img_ori) # ~21 dB

Extended help

Mathematically, this function solves the following ROF model using the primal-dual method:

References

[1] Chambolle, A. (2004). "An algorithm for total variation minimization and applications". Journal of Mathematical Imaging and Vision. 20: 89—97
[2] Wikipedia: Total Variation Denoising

Internal machinery

# ImageFiltering.KernelFactors.ReshapedOneD — Type

ReshapedOneD{N,Npre}(data)

Return an object of dimensionality N, where data must have dimensionality 1. The axes are 0:0 for the first Npre dimensions, have the axes of data for dimension Npre+1, and are 0:0 for the remaining dimensions.

data must support eltype and ndims, but does not have to be an AbstractArray.

ReshapedOneDs allow one to specify a "filtering dimension" for a 1-dimensional filter.