Python neural networks and their integration with Engee models¶
In this demonstration, we will look at an example of training neural networks using the sklearn package.
To work with neural networks using Python in Engee, the PyCall package and Python call commands in Engee are used.
First, let's install the sklearn and PyPlot libraries in Engee.
# Pkg.add("ScikitLearn")
Pkg.add("PyPlot")
using PyPlot
# Импорт необходимых модулей из Python
@pyimport sklearn.neural_network as nn
@pyimport sklearn.tree as tree
@pyimport numpy as np
Generating training data from a model in Engee¶
This model generates a vector of two sinusoidal values that represent a single signal for the classifier. Based on the determination of the amplitude fluctuation of the output signal, the class labels are set. The figure below shows the top level of the model.
In the figure below, you can see the data generation process.
The following shows the formation of the class label over three ranges:
- less than -0.7;
- greater than -0.7 and less than 0.7;
- greater than 0.7.
Let's move on to data generation by connecting the auxiliary function to run the model, run the model and see what data has been logged into simout.
function run_model(name_model)
Path = (@__DIR__) * "/" * name_model * ".engee"
if name_model in [m.name for m in engee.get_all_models()] # Проверка условия загрузки модели в ядро
model = engee.open( name_model ) # Открыть модель
model_output = engee.run( model, verbose=true ); # Запустить модель
else
model = engee.load( Path, force=true ) # Загрузить модель
model_output = engee.run( model, verbose=true ); # Запустить модель
engee.close( name_model, force=true ); # Закрыть модель
end
return model_output
end
run_model("PyDataGen")
sleep(1)
collect(simout)
Next, we will unload the data from simout and convert it to numpy format for further feeding into Python neural networks.
target = simout["PyDataGen/target"];
target = collect(target);
target = np.array(target.value);
Data = simout["PyDataGen/Data"];
Data = collect(Data);
Data = np.array(Data.value);
Multilayer perseptron¶
The first network, a classifier, that we consider in this demonstration is a multilayer perseptron network for predicting data group membership based on oscillation amplitude.
A multilayer perseptron (MLP) is a model of a feed-forward artificial neural network that maps sets of input data to a set of corresponding output data. MLP consists of several layers, each layer is fully connected to the next. The nodes of the layers are neurons with nonlinear activation functions, except for the nodes of the input layer. There may be one or more nonlinear hidden layers between the input layer and the output layer.
The figure below shows an MLP with one hidden layer with scalar output.
Let's move on to initialising and training the neural network in Python. There are several parameters to configure MLP:
- hidden_layer_sizes - the size of the hidden layer;
- max_iter - the maximum allowed number of training iterations;
- alpha - training step;
- solver - solver that defines the algorithm for optimising the weights on the nodes;
- verbose - specifies whether to output additional information;
- random_state - setting to control random values;
- learning_rate_init - learning rate.
clf = nn.MLPClassifier(hidden_layer_sizes=(100,), max_iter=10, alpha=1e-4, solver="sgd", verbose=1, random_state=1, learning_rate_init=.1)
clf[:fit](Data, target)
As we can see from the warning, the maximum number of iterations was reached and the optimisation did not converge. Hence, we need to increase the number of iterations. Let's repeat the initialisation and training of the neural network in Python with new network parameters.
clf = nn.MLPClassifier(hidden_layer_sizes=(100,), max_iter=140, alpha=1e-4, solver="sgd", verbose=1, random_state=1, learning_rate_init=.1)
clf[:fit](Data, target)
Now let us evaluate the quality of the model. As we can see, the neural network has high guessing accuracy.
accuracy = clf[:score](Data, target)
println("Точность: $accuracy")
Decision tree¶
Let's consider an example of code for training a neural network with a decision tree structure. This is a non-parametric learning method with a teacher, used for classification and regression.
This method requires creating a model that predicts the value of a target variable by learning simple decision rules inferred from data features.
The tree can be viewed as a piecewise constant approximation. We start by declaring and training the decision tree.
dt_clf = tree.DecisionTreeClassifier()
dt_clf.fit(Data, target)
Let's construct a graph of the decision tree and increase the pixel size and density to make the graph more visual.
plt.figure(figsize=(13, 4), dpi=200)
tree.plot_tree(dt_clf, filled=true)
Create a new graph to display the result of the neural network.
plt.figure(figsize=(18, 6), dpi=80) # увеличим размер и плотность пикселей
plt.scatter(Data[:, 1], Data[:, 2], c=target, cmap=plt.cm.Paired, edgecolors="k")
plt.title("Neural Network")
As we can see from the resulting graph, all the values we generated were distributed into three separate groups based on the sum of input data.
Conclusion¶
We have shown with concrete examples the options for integrating neural networks into Engee, as well as the possibilities of combining model-oriented design and Python functionality.