Документация Engee

BRISK Descriptor

Страница в процессе перевода.

The BRISK (Binary Robust Invariant Scalable Keypoints) descriptor has a predefined sampling pattern as compared to BRIEF or ORB. Pixels are sampled over concentric rings. For each sampling point, a small patch is considered around it. Before starting the algorithm, the patch is smoothed using gaussian smoothing.

Two types of pairs are used for sampling, short and long pairs. Short pairs are those where the distance is below a set threshold distmax while the long pairs have distance above distmin. Long pairs are used for orientation and short pairs are used for calculating the descriptor by comparing intensities.

BRISK achieves rotation invariance by trying the measure orientation of the keypoint and rotating the sampling pattern by that orientation. This is done by first calculating the local gradient g(pi,pj) between sampling pair (pi,pj) where I(pj, pj) is the smoothed intensity after applying gaussian smoothing.

g(pi, pj) = (pi - pj) . I(pj, j) -I(pj, j)pj - pi2

All local gradients between long pairs and then summed and the arctangent(gy/gx) between y and x components of the sum is taken as the angle of the keypoint. Now, we only need to rotate the short pairs by that angle to help the descriptor become more invariant to rotation. The descriptor is built using intensity comparisons. For each short pair if the first point has greater intensity than the second, then 1 is written else 0 is written to the corresponding bit of the descriptor.

Example

Let us take a look at a simple example where the BRISK descriptor is used to match two images where one has been translated by (50, 40) pixels and then rotated by an angle of 75 degrees. We will use the lake_color image from the TestImages package for this example.

First, let us create the two images we will match using BRISK.

using ImageFeatures, TestImages, Images, ImageDraw, CoordinateTransformations, Rotations
using MosaicViews

img = testimage("lake_color")
fttljgp

Original Image

img1 = Gray.(img)
rot = recenter(RotMatrix(5pi/6), [size(img1)...] .÷ 2)  # a rotation around the center
tform = rot ∘ Translation(-50, -40)
img2 = warp(img1, tform, axes(img1))
mosaicview(img, img1, img2; nrow=1)
edmxlto

To calculate the descriptors, we first need to get the keypoints. For this tutorial, we will use the FAST corners to generate keypoints (see fastcorners).

features_1 = Features(fastcorners(img1, 12, 0.35))
features_2 = Features(fastcorners(img2, 12, 0.35))
292-element Vector{Feature}:
 Feature(CartesianIndex(208, 1), 0.0, 0.0)
 Feature(CartesianIndex(209, 1), 0.0, 0.0)
 Feature(CartesianIndex(209, 2), 0.0, 0.0)
 Feature(CartesianIndex(210, 2), 0.0, 0.0)
 Feature(CartesianIndex(276, 31), 0.0, 0.0)
 Feature(CartesianIndex(317, 35), 0.0, 0.0)
 Feature(CartesianIndex(318, 35), 0.0, 0.0)
 Feature(CartesianIndex(317, 36), 0.0, 0.0)
 Feature(CartesianIndex(306, 48), 0.0, 0.0)
 Feature(CartesianIndex(313, 53), 0.0, 0.0)
 ⋮
 Feature(CartesianIndex(452, 510), 0.0, 0.0)
 Feature(CartesianIndex(474, 510), 0.0, 0.0)
 Feature(CartesianIndex(475, 510), 0.0, 0.0)
 Feature(CartesianIndex(311, 511), 0.0, 0.0)
 Feature(CartesianIndex(474, 511), 0.0, 0.0)
 Feature(CartesianIndex(475, 511), 0.0, 0.0)
 Feature(CartesianIndex(310, 512), 0.0, 0.0)
 Feature(CartesianIndex(311, 512), 0.0, 0.0)
 Feature(CartesianIndex(474, 512), 0.0, 0.0)

To create the BRISK descriptor, we first need to define the parameters by calling the BRISK constructor.

brisk_params = BRISK()
BRISK(0.25, 4, 1.0, Vector{Tuple{Float16, Float16}}[[(0.0, 0.0), (0.0, 2.465), (1.449, 1.994), (2.344, 0.7617), (2.344, -0.7617), (1.449, -1.994), (0.0, -2.465), (-1.449, -1.994), (-2.344, -0.7617), (-2.344, 0.7617)  …  (0.0, -9.18), (-2.836, -8.734), (-5.395, -7.426), (-7.426, -5.395), (-8.734, -2.836), (-9.18, -0.0), (-8.734, 2.836), (-7.426, 5.395), (-5.395, 7.426), (-2.836, 8.734)], [(0.0, 0.0), (0.01512, 2.465), (1.461, 1.985), (2.35, 0.7476), (2.34, -0.776), (1.437, -2.004), (-0.01512, -2.465), (-1.461, -1.985), (-2.35, -0.7476), (-2.34, 0.776)  …  (-0.05634, -9.18), (-2.89, -8.71), (-5.44, -7.395), (-7.46, -5.35), (-8.75, -2.783), (-9.18, 0.05634), (-8.71, 2.89), (-7.395, 5.44), (-5.35, 7.46), (-2.783, 8.75)], [(0.0, 0.0), (0.03024, 2.465), (1.474, 1.977), (2.354, 0.733), (2.334, -0.7905), (1.424, -2.012), (-0.03024, -2.465), (-1.474, -1.977), (-2.354, -0.733), (-2.334, 0.7905)  …  (-0.1127, -9.18), (-2.943, -8.695), (-5.49, -7.36), (-7.492, -5.305), (-8.766, -2.729), (-9.18, 0.1127), (-8.695, 2.943), (-7.36, 5.49), (-5.305, 7.492), (-2.729, 8.766)], [(0.0, 0.0), (0.04538, 2.465), (1.485, 1.967), (2.357, 0.7183), (2.33, -0.8047), (1.412, -2.021), (-0.04538, -2.465), (-1.485, -1.967), (-2.357, -0.7183), (-2.33, 0.8047)  …  (-0.169, -9.18), (-2.996, -8.68), (-5.53, -7.33), (-7.523, -5.258), (-8.78, -2.676), (-9.18, 0.169), (-8.68, 2.996), (-7.33, 5.53), (-5.258, 7.523), (-2.676, 8.78)], [(0.0, 0.0), (0.0605, 2.465), (1.497, 1.958), (2.363, 0.704), (2.324, -0.819), (1.399, -2.03), (-0.0605, -2.465), (-1.497, -1.958), (-2.363, -0.704), (-2.324, 0.819)  …  (-0.2253, -9.18), (-3.05, -8.66), (-5.58, -7.293), (-7.56, -5.21), (-8.8, -2.621), (-9.18, 0.2253), (-8.66, 3.05), (-7.293, 5.58), (-5.21, 7.56), (-2.621, 8.8)], [(0.0, 0.0), (0.0756, 2.463), (1.51, 1.949), (2.367, 0.6895), (2.32, -0.8335), (1.387, -2.037), (-0.0756, -2.463), (-1.51, -1.949), (-2.367, -0.6895), (-2.32, 0.8335)  …  (-0.2815, -9.17), (-3.104, -8.64), (-5.62, -7.258), (-7.59, -5.164), (-8.81, -2.568), (-9.17, 0.2815), (-8.64, 3.104), (-7.258, 5.62), (-5.164, 7.59), (-2.568, 8.81)], [(0.0, 0.0), (0.09076, 2.463), (1.521, 1.939), (2.371, 0.675), (2.314, -0.8477), (1.374, -2.047), (-0.09076, -2.463), (-1.521, -1.939), (-2.371, -0.675), (-2.314, 0.8477)  …  (-0.338, -9.17), (-3.156, -8.62), (-5.664, -7.223), (-7.62, -5.117), (-8.83, -2.514), (-9.17, 0.338), (-8.62, 3.156), (-7.223, 5.664), (-5.117, 7.62), (-2.514, 8.83)], [(0.0, 0.0), (0.10583, 2.463), (1.533, 1.93), (2.375, 0.66), (2.309, -0.862), (1.362, -2.055), (-0.10583, -2.463), (-1.533, -1.93), (-2.375, -0.66), (-2.309, 0.862)  …  (-0.3943, -9.17), (-3.209, -8.6), (-5.71, -7.188), (-7.652, -5.07), (-8.84, -2.459), (-9.17, 0.3943), (-8.6, 3.209), (-7.188, 5.71), (-5.07, 7.652), (-2.459, 8.84)], [(0.0, 0.0), (0.121, 2.463), (1.545, 1.921), (2.379, 0.646), (2.305, -0.876), (1.35, -2.062), (-0.121, -2.463), (-1.545, -1.921), (-2.379, -0.646), (-2.305, 0.876)  …  (-0.4504, -9.17), (-3.262, -8.58), (-5.754, -7.152), (-7.684, -5.023), (-8.86, -2.404), (-9.17, 0.4504), (-8.58, 3.262), (-7.152, 5.754), (-5.023, 7.684), (-2.404, 8.86)], [(0.0, 0.0), (0.1361, 2.46), (1.557, 1.911), (2.383, 0.6313), (2.299, -0.89), (1.337, -2.07), (-0.1361, -2.46), (-1.557, -1.911), (-2.383, -0.6313), (-2.299, 0.89)  …  (-0.507, -9.164), (-3.314, -8.56), (-5.797, -7.117), (-7.715, -4.977), (-8.875, -2.35), (-9.164, 0.507), (-8.56, 3.314), (-7.117, 5.797), (-4.977, 7.715), (-2.35, 8.875)]  …  [(-0.0, 0.0), (-0.1511, 2.46), (1.324, 2.08), (2.293, 0.904), (2.387, -0.6167), (1.568, -1.901), (0.1511, -2.46), (-1.324, -2.08), (-2.293, -0.904), (-2.387, 0.6167)  …  (0.563, -9.164), (-2.297, -8.89), (-4.93, -7.742), (-7.082, -5.84), (-8.54, -3.367), (-9.164, -0.563), (-8.89, 2.297), (-7.742, 4.93), (-5.84, 7.082), (-3.367, 8.54)], [(-0.0, 0.0), (-0.1361, 2.46), (1.337, 2.07), (2.299, 0.89), (2.383, -0.6313), (1.557, -1.911), (0.1361, -2.46), (-1.337, -2.07), (-2.299, -0.89), (-2.383, 0.6313)  …  (0.507, -9.164), (-2.35, -8.875), (-4.977, -7.715), (-7.117, -5.797), (-8.56, -3.314), (-9.164, -0.507), (-8.875, 2.35), (-7.715, 4.977), (-5.797, 7.117), (-3.314, 8.56)], [(-0.0, 0.0), (-0.121, 2.463), (1.35, 2.062), (2.305, 0.876), (2.379, -0.646), (1.545, -1.921), (0.121, -2.463), (-1.35, -2.062), (-2.305, -0.876), (-2.379, 0.646)  …  (0.4504, -9.17), (-2.404, -8.86), (-5.023, -7.684), (-7.152, -5.754), (-8.58, -3.262), (-9.17, -0.4504), (-8.86, 2.404), (-7.684, 5.023), (-5.754, 7.152), (-3.262, 8.58)], [(-0.0, 0.0), (-0.10583, 2.463), (1.362, 2.055), (2.309, 0.862), (2.375, -0.66), (1.533, -1.93), (0.10583, -2.463), (-1.362, -2.055), (-2.309, -0.862), (-2.375, 0.66)  …  (0.3943, -9.17), (-2.459, -8.84), (-5.07, -7.652), (-7.188, -5.71), (-8.6, -3.209), (-9.17, -0.3943), (-8.84, 2.459), (-7.652, 5.07), (-5.71, 7.188), (-3.209, 8.6)], [(-0.0, 0.0), (-0.09076, 2.463), (1.374, 2.047), (2.314, 0.8477), (2.371, -0.675), (1.521, -1.939), (0.09076, -2.463), (-1.374, -2.047), (-2.314, -0.8477), (-2.371, 0.675)  …  (0.338, -9.17), (-2.514, -8.83), (-5.117, -7.62), (-7.223, -5.664), (-8.62, -3.156), (-9.17, -0.338), (-8.83, 2.514), (-7.62, 5.117), (-5.664, 7.223), (-3.156, 8.62)], [(-0.0, 0.0), (-0.0756, 2.463), (1.387, 2.037), (2.32, 0.8335), (2.367, -0.6895), (1.51, -1.949), (0.0756, -2.463), (-1.387, -2.037), (-2.32, -0.8335), (-2.367, 0.6895)  …  (0.2815, -9.17), (-2.568, -8.81), (-5.164, -7.59), (-7.258, -5.62), (-8.64, -3.104), (-9.17, -0.2815), (-8.81, 2.568), (-7.59, 5.164), (-5.62, 7.258), (-3.104, 8.64)], [(-0.0, 0.0), (-0.0605, 2.465), (1.399, 2.03), (2.324, 0.819), (2.363, -0.704), (1.497, -1.958), (0.0605, -2.465), (-1.399, -2.03), (-2.324, -0.819), (-2.363, 0.704)  …  (0.2253, -9.18), (-2.621, -8.8), (-5.21, -7.56), (-7.293, -5.58), (-8.66, -3.05), (-9.18, -0.2253), (-8.8, 2.621), (-7.56, 5.21), (-5.58, 7.293), (-3.05, 8.66)], [(-0.0, 0.0), (-0.04538, 2.465), (1.412, 2.021), (2.33, 0.8047), (2.357, -0.7183), (1.485, -1.967), (0.04538, -2.465), (-1.412, -2.021), (-2.33, -0.8047), (-2.357, 0.7183)  …  (0.169, -9.18), (-2.676, -8.78), (-5.258, -7.523), (-7.33, -5.53), (-8.68, -2.996), (-9.18, -0.169), (-8.78, 2.676), (-7.523, 5.258), (-5.53, 7.33), (-2.996, 8.68)], [(-0.0, 0.0), (-0.03024, 2.465), (1.424, 2.012), (2.334, 0.7905), (2.354, -0.733), (1.474, -1.977), (0.03024, -2.465), (-1.424, -2.012), (-2.334, -0.7905), (-2.354, 0.733)  …  (0.1127, -9.18), (-2.729, -8.766), (-5.305, -7.492), (-7.36, -5.49), (-8.695, -2.943), (-9.18, -0.1127), (-8.766, 2.729), (-7.492, 5.305), (-5.49, 7.36), (-2.943, 8.695)], [(-0.0, 0.0), (-0.01512, 2.465), (1.437, 2.004), (2.34, 0.776), (2.35, -0.7476), (1.461, -1.985), (0.01512, -2.465), (-1.437, -2.004), (-2.34, -0.776), (-2.35, 0.7476)  …  (0.05634, -9.18), (-2.783, -8.75), (-5.35, -7.46), (-7.395, -5.44), (-8.71, -2.89), (-9.18, -0.05634), (-8.75, 2.783), (-7.46, 5.35), (-5.44, 7.395), (-2.89, 8.71)]], Vector{Float16}[[0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861]  …  [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861], [0.5527, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986, 0.986  …  1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861, 1.861]], Tuple{Float16, Float16}[(0.0, 0.12006), (0.0521, 0.10815), (0.0939, 0.0749), (0.117, 0.02669), (0.117, -0.02669), (0.0939, -0.0749), (0.0521, -0.10815), (0.02051, -0.1171), (0.0, -0.11426), (-0.02051, -0.1171)  …  (0.03366, -0.04633), (0.02505, -0.04913), (0.0168, -0.05176), (0.00863, -0.05447), (0.0, -0.05722), (-0.00957, -0.0604), (-0.02081, -0.0641), (-0.03497, -0.0686), (-0.0544, -0.07495), (-0.0848, -0.0848)], Tuple{Int16, Int16}[(1, 2), (1, 3), (2, 3), (1, 4), (2, 4), (3, 4), (1, 5), (2, 5), (3, 5), (4, 5)  …  (58, 59), (12, 60), (25, 60), (26, 60), (39, 60), (40, 60), (41, 60), (42, 60), (58, 60), (59, 60)], Tuple{Int16, Int16}[(12, 19), (13, 20), (14, 21), (15, 22), (16, 23), (17, 24), (18, 25), (6, 26), (7, 26), (8, 26)  …  (48, 60), (49, 60), (50, 60), (51, 60), (52, 60), (53, 60), (54, 60), (55, 60), (56, 60), (57, 60)])

Now pass the image with the keypoints and the parameters to the create_descriptor function.

desc_1, ret_features_1 = create_descriptor(img1, features_1, brisk_params)
desc_2, ret_features_2 = create_descriptor(img2, features_2, brisk_params)

The obtained descriptors can be used to find the matches between the two images using the match_keypoints function.

matches = match_keypoints(Keypoints(ret_features_1), Keypoints(ret_features_2), desc_1, desc_2, 0.1)

We can use the ImageDraw.jl package to view the results.

grid = hcat(img1, img2)
offset = CartesianIndex(0, size(img1, 2))
map(m -> draw!(grid, LineSegment(m[1], m[2] + offset)), matches)
grid

This page was generated using DemoCards.jl and Literate.jl.