Image Processing

In this example, we will demonstrate how to use Engee for image processing, analyze individual functions for image processing, consider the possibilities of accessing images and methods of their processing and representation.

Let's start by selecting the required function libraries.

Pkg.add(["ImageShow"])

using Images # Библиотека обработки изображений
using ImageShow # Библиотека отрисовки изображений

Uploading an image

The load function allows you to save images to workspaces as a matrix.

I = load("/user/start/examples/image_processing/processing_methods/IMG_001.jpg")

Converting RGB to shades of gray

Let's create a custom function for calculating average values for the sum of measurements of a three-dimensional RGB matrix.

RGB (red, green, blue) is an additive color model that describes a way to encode colors for color reproduction using three colors, which are commonly referred to as primary colors.

function brightness(pixel)
    arr = pixel.r + pixel.g + pixel.b;
    return sum(arr) / length(arr)
end

brightness (generic function with 1 method)

After defining the function, we will call it and draw the image.

Ibright = brightness.(I)
simshow(Ibright)

As you can see from the picture above, we have reduced the images to shades of gray.

Detecting angles

The angle detection method attempts to identify points in an image that have a well-defined position and can be reliably identified in multiple images of the same scene. Very often these points lie at the corners or edges of the image objects — hence the name of the method.

Angle detection is useful in a number of computer vision tasks — image registration, motion detection, and panoramic stitching. The method is based on the fact that if the location of the same points is known in two different images, this makes it possible to align these images.

To detect angles, we will use the imcorner function, which has one additional indicator, which is the threshold percentage parameter for detecting points of interest.

corners = imcorner(Ibright, Percentile(98.5));

┌ Error: handle_connection handler error
│   exception = (Base.IOError("write: broken pipe (EPIPE)", -32), Base.StackTraces.StackFrame[uv_write(s::TCPSocket, p::Ptr{UInt8}, n::UInt64) at stream.jl:1064, unsafe_write(s::TCPSocket, p::Ptr{UInt8}, n::UInt64) at stream.jl:1118, unsafe_write(c::HTTP.ConnectionPool.Connection, p::Ptr{UInt8}, n::UInt64) at ConnectionPool.jl:106, write(io::HTTP.ConnectionPool.Connection, s::String) at io.jl:244, write at io.jl:676 [inlined], unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Connection}, p::Ptr{UInt8}, n::UInt64) at Streams.jl:96, write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Connection}, s::String) at io.jl:244, write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Connection}, s::Base.CodeUnits{UInt8, String}) at basic.jl:758, (::HTTP.Handlers.var"#1#2"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.ConnectionPool.Connection}) at Handlers.jl:61, #invokelatest#2 at essentials.jl:729 [inlined], invokelatest at essentials.jl:726 [inlined], handle_connection(f::Function, c::HTTP.ConnectionPool.Connection, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing) at Servers.jl:447, (::HTTP.Servers.var"#16#17"{HTTP.Handlers.var"#1#2"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.ConnectionPool.Connection}, Int64, Nothing, Base.Semaphore, HTTP.ConnectionPool.Connection})() at task.jl:484])
└ @ HTTP.Servers /root/.julia/packages/HTTP/Kxa7P/src/Servers.jl:461

After detecting the points of interest, we mark them in red by converting the grayscale images to RGB and assigning the unit values at the points of interest to the first dimension of the matrix.

img_copy = RGB.(Ibright)
img_copy[corners] .= RGB(1.0, 0.0, 0.0)

┌ Warning: Output swatches are reduced due to the large size (1×29420).
│ Load the ImageShow package for large images.
└ @ Colors /root/.julia/packages/Colors/mIuXl/src/display.jl:159

As we can see, the angles were found on the most contrasting objects in the image. In this case, these are clouds and the sea.

simshow(img_copy)

Detecting the boundaries of depicted objects

Another common task in image processing is to find the boundaries of objects. To do this, use the Kernel.sobel function, which allows you to translate images into the frequency domain. The following is a custom border detection feature.

function find_energy(image)
    energy_y = imfilter(brightness.(image), Kernel.sobel()[1])
    energy_x = imfilter(brightness.(image), Kernel.sobel()[2])    
    return sqrt.(energy_x.^2 + energy_y.^2)
end

find_energy (generic function with 1 method)

In the next step, we will apply the custom feature boundary detection function to the source image and display the result of image processing.

simshow(find_energy(I))

Based on the processing results, we can unambiguously determine the boundaries of each object in the image, and this data can subsequently be applied to further image processing.

Conclusion

As our example of working with a painting has shown, Engee is great for image processing. Given the extensive mathematical functionality of the environment, we can describe any arbitrarily specific processing algorithm using custom functions.