Generating C-code for neural networks

In this example, we will explore the capabilities of libraries. Symbolics and Flux to create independent C code for various neural networks – fully connected, recurrent, and convolutional.

Required libraries

First of all, make sure that you have the necessary libraries installed.

Pkg.add(["Flux", "Symbolics"])
#Pkg.add("ChainPlots") # Временно исключаем эту библиотеку из-за несовместимости с последней версией Flux

   Resolving package versions...
  No Changes to `~/.project/Project.toml`
  No Changes to `~/.project/Manifest.toml`

Library Flux allows you to create neural networks, the code of which is set in the language Julia. The fact that the neural network is written in a high-level language, and not in a third-party library or DSL (Domain Specific Language such as PyTorch), will allow us to use the polymorphism mechanism and transform the neural network into a mathematical expression from which code can be generated.

Before updating the Flux library to the latest version, the following code will produce a harmless error message Error during loading of extension DiffEqBaseZygoteExt... which does not interfere with work. Just right-click on it and select Удалить выбранное.

using Flux, Symbolics, ChainPlots
gr();

Be careful, neural networks can contain a lot of parameters. Translating a neural network containing hundreds or thousands of parameters into a mathematical expression can take a significant amount of time (several tens of seconds), and the expression itself is unlikely to be convenient for perception.

Code generation from a single neuron

Let's define a "neural network" of one neuron using the library Flux and then we get it in the form of a mathematical equation and in the form of a platform-independent code.

We will repeat the names of the libraries once to make it clearer exactly where they are used

using Flux
model = Dense( 1=>1, relu )

Dense(1 => 1, relu)  # 2 parameters

This is a very simple configurable algorithm with two parameters, which can be seen in symbolic form as follows:

using Symbolics
@variables x
model( [x] )

Activation function ReLU, quite predictably, creates a neuron with a piecewise linear description.

All the coefficients of this algorithm are randomly selected, but for the sake of ease of perception, we will assign them some values.

model.weight .= [10]
model.bias .= [20]
model( [x] )

Let's generate the code and see what happens.:

c_model_code = build_function( model( [x] ), [x]; target=Symbolics.CTarget(), fname="neural_net", lhsname=:y, rhsnames=[:x] )
println( """$c_model_code""" )

#include <math.h>
void neural_net(double* y, const double* x) {
  y[0] = ifelse(20.0f0 + 10.0f0 * x[0] < 0, 0, 20.0f0 + 10.0f0 * x[0]);
}

We can see that, at least, there is a non-standard function in the code. ifelse, and the parameters double they have a suffix f0. These instructions are in some C standard that the library is targeting. Symbolics but since we did not configure the code generation for ourselves, but generated the code "out of the box", we will need to transform it a little. For example like this:

c_fixed_model_code = replace( c_model_code, "double" => "float", "f0" => "f" );

Such code can already be placed in a template and compiled.:

c_code_standalone = """
#include <stdio.h>
#include <stdbool.h>

float ifelse(bool cond, float a, float b) { return cond ? a : b; }

$c_fixed_model_code

int main(int argc, char const *argv[]){
    float out[1];
    float in[] = {3.1415};
    
    neural_net( out, in );
    
    printf( "%f\\n", out[0] );
    return 0;
}""";

# Сохраняем в файл доработанный код
open("$(@__DIR__)/neural_net_fc.c", "w") do f
    println( f, "$c_code_standalone" )
end

;gcc -o out_fc neural_net_fc.c -lm

;./out_fc

51.415001

model( [3.1415] )

┌ Warning: Layer with Float32 parameters got Float64 input.
│   The input will be converted, but any earlier layers may be very slow.
│   layer = Dense(1 => 1, relu)  # 2 parameters
│   summary(x) = "1-element Vector{Float64}"
└ @ Flux /usr/local/ijulia-demos/packages/Flux/9PibT/src/layers/stateless.jl:60

1-element Vector{Float32}:
 51.415

At this stage, you might receive a warning that it is better not to use the parameters Float64 In a neural network, this does not benefit the accuracy of calculations, but it harms performance (especially on the GPU). Convert all neural network parameters to Float32 you can use the command model = fmap(f32, model).

As we can see, the value calculated using the C code and using the neural network specified in Julia are almost the same, with rounding accuracy.

Code generation from a multilayer FC neural network

Let's check the code generation using the example of a neural network of two neurons. For now, it will be a regular multilayer fully connected neural network (fc, fully connected, sometimes called a perceptron).

using Flux
mlp_model = Chain( Dense( 1=>2), Dense(2=>1) )

Chain(
  Dense(1 => 2),                        # 4 parameters
  Dense(2 => 1),                        # 3 parameters
)                   # Total: 4 arrays, 7 parameters, 284 bytes.

Here is an illustration of this multi-layered neural network.

# Временно отключена из-за несовместимости версий библиотек ChainPlots и Flux
#using ChainPlots
#p = np = plot( mlp_model, titlefontsize=10, size=(300,300), markersize=8, xticks=((0,1,2),["Входной\nслой", "Скрытый\nслой", "Выходной\nслой"]), markerstrokewidth=2, linewidth=1 )

This model is initialized with random values, but it can already accept vectors, matrices, and more as input. Note that neural networks always require that a vector of parameters be passed to the input, even if they are symbolic variables.

using Symbolics
@variables x
c_model_code = build_function( mlp_model( [x] ), [x]; target=Symbolics.CTarget(), fname="neural_net", lhsname=:y, rhsnames=[:x] )
println( c_model_code )

#include <math.h>
void neural_net(double* y, const double* x) {
  y[0] = 0.1388071f0 * x[0];
}

What's wrong with this code? The expression for a multilayer network has been reduced to an expression for a single neuron. That's how it should be. Library Symbolics simplified the expression before code generation, and ** a multilayer neural network consisting only of "linear layers" is similar to a single neuron**. It was necessary to set the activation function after the first layer of the neural network ** (sigmoid, tanh...), other than linear (identity or -skip-).

After some transformations, this code can be placed in a block. C Function to the Engee canvas, or add a function to it main and compile it into an executable binary file.

Code generation from a recurrent neural network

Recurrent neural networks are well used for time series analysis and sequence generation. They can also be multi-layered, but they have a slightly different neuron topology. Let's see what kind of code we get.

input_channels_nb = 2
hidden_state_size = 3
rnn = Flux.RNNCell(input_channels_nb, hidden_state_size)

# Эти параметры настраиваются в процессе обучения
# - для наглядности, назначим каждой матрице параметров некоторые значения
# - к сожалению, они не могут быть символьными
rnn.Wh .= 0.1 .+ reshape(1:hidden_state_size*hidden_state_size, hidden_state_size, hidden_state_size) ./ 10
rnn.Wi .= 0.01 .+ reshape(1:hidden_state_size*input_channels_nb, hidden_state_size, input_channels_nb) ./ 100
rnn.bias .= 0.001 .+ collect(1:hidden_state_size) ./ 1000

X = Symbolics.variables( :x, 1:input_channels_nb )
H = Symbolics.variables( :h, 1:hidden_state_size )

y,h = rnn( X ) # Этой командой можно выполнять нейросеть и одновременно обновлять состояние
println(y)
println(h)

Num[tanh(0.002 + 0.02x₁ + 0.05x₂), tanh(0.003 + 0.03x₁ + 0.06x₂), tanh(0.004 + 0.04x₁ + 0.07x₂)]
Num[tanh(0.002 + 0.02x₁ + 0.05x₂), tanh(0.003 + 0.03x₁ + 0.06x₂), tanh(0.004 + 0.04x₁ + 0.07x₂)]

Note that since the output of the network in this topology is a hidden vector, the functions for calculating y and for h they are identical.

#p = plot( rnn_model, titlefontsize=10, size=(300,300), markersize=8, xticks=((0,1,2),["Входной\nслой", "Рекуррентный\nслой"]), markerstrokewidth=2, linewidth=1 )

c_model_code = build_function( vcat(y,h), [X; H]; target=Symbolics.CTarget(), fname="neural_net", lhsname=:LHS, rhsnames=[:RHS] )
println( c_model_code )

#include <math.h>
void neural_net(double* LHS, const double* RHS) {
  LHS[0] = tanh(0.002f0 + 0.02f0 * RHS[0] + 0.05f0 * RHS[1]);
  LHS[1] = tanh(0.003f0 + 0.03f0 * RHS[0] + 0.06f0 * RHS[1]);
  LHS[2] = tanh(0.004f0 + 0.04f0 * RHS[0] + 0.07f0 * RHS[1]);
  LHS[3] = tanh(0.002f0 + 0.02f0 * RHS[0] + 0.05f0 * RHS[1]);
  LHS[4] = tanh(0.003f0 + 0.03f0 * RHS[0] + 0.06f0 * RHS[1]);
  LHS[5] = tanh(0.004f0 + 0.04f0 * RHS[0] + 0.07f0 * RHS[1]);
}

This particular architecture returns a state vector as an output variable. h.

Other architectures will require a more complex code generation process. For example, the model can be trained using the usual library tools. Flux but for the code generation stage, a surrogate neural network architecture will be developed in the form of a Julia function (for example, a function that will return to us y and h).

The number of improvements to generate code from a recurrent neural network can be estimated using the following template, which can hardly be called simple:

c_code_standalone = """
#include <stdio.h>

$(replace( c_model_code, "double" => "float", "f0" => "f" ))

int main(int argc, char const *argv[]){
    float out[3];
    float in[] = {10, 10, 10, 10, 10};
    
    for( int i=0; i<5; i++ ){
        neural_net( out, in );
        printf( "%f %f %f\\n", out[0], out[1], out[2] );
        
        /* Подставим выходное значения в входной вектор в качестве вектора состояния */
        in[2] = out[0];
        in[3] = out[1];
        in[4] = out[2];
    }
    
    return 0;
}""";

# Сохраняем в файл доработанный код
open("$(@__DIR__)/neural_net_rnn.c", "w") do f
    println( f, "$c_code_standalone" )
end

We will compile and run the code of this neural network.

;gcc -o out_rnn neural_net_rnn.c -lm

;./out_rnn

0.605636 0.717755 0.801931
0.041686 0.054232 0.066762
0.004701 0.006491 0.008282
0.002363 0.003464 0.004565
0.002217 0.003275 0.004332

We called the recurrent neural network 5 times with the same input data and got different results each time, since the state vector was updated for this neural network.

Code generation from a convolutional neural network

in_channels = 1
out_channels = 1
filter_x_size = 2
filter_y_size = 1
signal_size = 10

cnn_model = Chain(
    Conv((filter_x_size,filter_y_size), in_channels => out_channels, tanh; bias = false),
    Dense(signal_size - filter_x_size + 1 => 1)
    )

Chain(
  Conv((2, 1), 1 => 1, tanh, bias=false),  # 2 parameters
  Dense(9 => 1),                        # 10 parameters
)                   # Total: 3 arrays, 12 parameters, 496 bytes.

To visualize this neural network, we need to provide an example of an input vector. xs with the correct dimension (width, height, channels, patches).

# xs = rand( signal_size );
# p = plot( cnn_model, reshape(xs, :, 1, 1, 1), titlefontsize=10, size=(300,300), markersize=8, xticks=((0,1,2),["Входной\nслой", "Сверточный\nслой", "Выходной\nслой"]), markerstrokewidth=2, linewidth=1 )

For clarity, we will set the last (FC) layer of the neural network to certain values of weights and offsets, which we will see in the generated code.

cnn_model[2].weight .= reshape( collect(1:(signal_size - filter_x_size + 1)), 1, :);
cnn_model[2].bias .= [0.1];

This model has a rather complex input data organization. Let's demonstrate how to do it.:

cnn_model( reshape(xs, :, 1, 1, 1) )[:] # w, h, channels, batch

1-element Vector{Float32}:
 -23.33473

In this scenario, we just have to implement the convolution operation as a Julia function. When we simply try to substitute a four-dimensional matrix of symbolic variables into the model, we get an error UndefRefError: access to undefined reference. Therefore, let's look at this neural network and create a function that uses the parameters of the convolution core, ** and we can take the fully connected layer from the original model**.

function my_convolution_net( c_in )
    c_out = [tanh( sum(c_in[i:i+1] .* reverse(cnn_model[1].weight[:]) )) for i in 1:length(c_in)-1]
    out = cnn_model[2]( c_out ); # Второй слой модели мы используем в нашем коде без изменений
    return out
end;

What will the code look like after generation?

X = Symbolics.variables( :x, 1:signal_size )
c_model_code = build_function( my_convolution_net( X ), X; target=Symbolics.CTarget(), fname="neural_net", lhsname=:LHS, rhsnames=[:RHS] )
println( c_model_code )

#include <math.h>
void neural_net(double* LHS, const double* RHS) {
  LHS[0] = 0.1f0 + 7.0f0 * tanh(-0.72043973f0 * RHS[6] + -1.1665595f0 * RHS[7]) + 3.0f0 * tanh(-0.72043973f0 * RHS[2] + -1.1665595f0 * RHS[3]) + 6.0f0 * tanh(-0.72043973f0 * RHS[5] + -1.1665595f0 * RHS[6]) + 5.0f0 * tanh(-0.72043973f0 * RHS[4] + -1.1665595f0 * RHS[5]) + tanh(-0.72043973f0 * RHS[0] + -1.1665595f0 * RHS[1]) + 2.0f0 * tanh(-0.72043973f0 * RHS[1] + -1.1665595f0 * RHS[2]) + 8.0f0 * tanh(-0.72043973f0 * RHS[7] + -1.1665595f0 * RHS[8]) + 4.0f0 * tanh(-0.72043973f0 * RHS[3] + -1.1665595f0 * RHS[4]) + 9.0f0 * tanh(-1.1665595f0 * RHS[9] + -0.72043973f0 * RHS[8]);
}

On the surface, the code looks quite functional. Let's put it in our established template, execute it, and compare it with the results of previous functions.

c_code_standalone = """
#include <stdio.h>
#include <stdbool.h>

float ifelse(bool cond, float a, float b) { return cond ? a : b; }

$(replace( c_model_code, "double" => "float", "f0" => "f" ))

int main(int argc, char const *argv[]){
    float out[1];
    float in[] = {$(join(xs, ","))};
    
    neural_net( out, in );
    
    printf( "%f\\n", out[0] );
    return 0;
}""";

# Сохраняем в файл доработанный код
open("$(@__DIR__)/neural_net_cnn.c", "w") do f
    println( f, "$c_code_standalone" )
end

;gcc -o out_cnn neural_net_cnn.c -lm

;./out_cnn

-23.334730

cnn_model( reshape(xs, :, 1, 1, 1) )[:] # w, h, channels, batch

1-element Vector{Float32}:
 -23.33473

my_convolution_net( xs )

1-element Vector{Float32}:
 -23.33473

Obviously, the results are similar to rounding accuracy.

Conclusion

We have generated code from three of the most popular types of neural networks: fully connected (FC/MLP), recurrent (RNN) and convolutional (CNN).

Only for fully connected neural networks, code generation is relatively simple and straightforward, which makes it possible to significantly automate this process.

In other cases, when automating the creation of neural network code, it is useful to start from the task (1D or 2D convolution, using a hidden layer of a recurrent neural network). By the number of manual steps in configuring the CNN or RNN code generation process, it is obvious that automating this process requires fixing specific scenarios.