Model generation, assembly, loading, and launch

The article describes the software chain of the model in independent mode: the target receives the generated C code, builds the target platform project, downloads and runs the result. Interactive execution and XCP are discussed separately in the article. Executing the model interactively.

Methods for managing the operation of the model

For an independent mode, it is enough to implement four method BaseTarget:

generate_executable_code — prepare a draft of the target platform.
compile_model — assemble the project and get the artifact.
upload_model — transfer the artifact to the device or prepare it for launch.
start_model — confirm the launch or launch the artifact if the platform requires a separate launch.

These methods are called by the infrastructure Target Hardware sequentially, so each method must save the state necessary for the next step: the path to the project, the build directory and firmware, the name of the executable file, loader settings, etc.

Basic APIs

The main modules used in the target code:

targets.base_target — the contract of the target class and methods for managing the operation of the model (start, run, stop).
targets.base_models — data models that target receives and returns.
targets.code_info — low-level description of the generated C-code of the model.

The following APIs are used in the target code for independent execution mode:

BaseTarget sets the required target interface. A custom class is inherited from it. Even if the target does not support interactive mode, the methods start_stream and change_param they should still be implemented, but to target an independent execution mode, they should return an understandable error.
EngeeModel contains the model ID. In practice, it is most often used model.name: it is used to name the project directory, sketch, binary file, or firmware.
ModelSettings contains the settings for launching the model. To generate a project, they are especially important model_settings.cmi and model_settings.is_ext_mode. This chapter discusses the independent execution mode scenario, so is_ext_mode must be False or it should be explicitly rejected.
ModelCode contains the generated model code. For C-targets, it is used model_code.c_code: it has a dictionary of source files and a field c_code_info with metadata.
TargetResponse — the standard response of the targeting methods. For independent execution mode methods, it is enough to return TargetResponse(detail="…"), where detail briefly describes the completed stage.
CMIParser reads the EDM-Target block parameters from model_settings.cmi. Method get_target_block_options transforms the values of the EDM-Target block mask into the Pydantic model of the user platform settings.
create_codegen_info converts a detailed CodeInfo in a more convenient way for templates CodeGenInfo: names of C functions init/step/terminate, base period, list of step functions, model stop time, and other runtime parameters.

render_template_to_file generates a Jinja2 template in a project file: for example, main.c, .ino, CMakeLists.txt or Makefile.
recreate_dir deletes and recreates the project directory. This is convenient for pure generation, but use this API only for the directory created by the target.
store_model_src saves the C-sources of the model in the project. Generated Engee main.c it is skipped because the target creates its own entry point from the template.
store_filed_sources saves the user’s source files embedded in the C Function model blocks. Use it if the target platform needs to support custom C code from the model.
CMakeWrapper — ready wrapper for CMake assembly. It can be used if the target platform is going through CMakeLists.txt. They usually create their own builder class for Arduino, the manufacturer’s SDK, or their own build system.

Generation

Method generate_executable_code binds code generation Engee with a custom platform. It should not compile the project. His task is to create a complete project on yandex. Disk, which he will then be able to assemble compile_model.

The method usually does the following:

Saves model_settings, model_code and other data that the next steps will need.
Reads the EDM-Target block parameters via CMIParser.
Checks that the supported execution mode is selected.
Creates a clean project directory.
Converts model_code.c_code.c_code_info in CodeGenInfo.
Renderit runtime templates: main.c, .ino, CMakeLists.txt, Makefile or the IDE project files.
Saves the C-sources of the model via store_model_src.
Copies drivers, headers, layout files, startup files, SDK bindings, and other platform files.
Saves custom C Function files via store_filed_sources if they are supported.
Returns TargetResponse(detail="generate_executable_code").

Example of the structure generate_executable_code

def generate_executable_code(
    self,
    model: EngeeModel,
    model_settings: ModelSettings,
    model_code: ModelCode,
) -> TargetResponse:
    if model_settings.is_ext_mode:
        raise TargetException("MyTarget supports only independent execution.")

    self.model_settings = model_settings
    self.target_block = self.cmi_parser.get_target_block_options(
        model_settings.cmi,
        MyTargetBlock,
        self.__class__.__name__,
    )

    self.project_path = Path(self.target_block.codegen_folder) / model.name
    recreate_dir(self.project_path)

    project_src = self.project_path / "src"
    project_src.mkdir(parents=True, exist_ok=True)

    codegen_info = create_codegen_info(
        model_code.c_code.c_code_info,
        model_settings.is_ext_mode,
    )

    render_template_to_file(
        self.main_template,
        project_src / "main.c",
        codegen_info.model_dump(),
    )

    store_model_src(model_code.c_code, project_src, codegen_info)
    store_filed_sources(model_settings.cmi, project_src)

    return TargetResponse(detail="generate_executable_code")

Jinja2-templates

Jinja2 is a template engine: it takes a text file with substitutions and control structures, gets a dictionary of values, and generates a final file. In targets, Jinja2 is needed to generate C/C++/CMake code that depends on a specific model.

The template is needed because the names of the model’s functions, the base period, the list of step functions, and the stop time are unknown in advance. They appear only after code generation. Engee. Target receives this data via CodeGenInfo and passes them to the template.

The templates usually use:

{{ name }} — substitution of the value;
{% for item in items %} … {% endfor %} — cycle;
{% if condition %} … {% endif %} — a conditional fragment.

The template should only describe the platform binding. The model code is already in the files it saves. store_model_src.

Example of a template fragment main.c

#include "{{ model_name }}.h"

static unsigned long step_number = 0;

int main(void)
{
    platform_timer_init({{ base_rate_us }}UL);
    {{ init.cname }}();

    while (1) {
        unsigned long start_time = platform_time_us();

{% for step in steps %}
        if ((step_number % {{ step.base_rate_scale }}UL) == 0UL) {
            {{ step.cname }}();
        }
{% endfor %}

        step_number++;

{% if stop_time %}
        if ((step_number * {{ base_rate_us }}UL) > {{ stop_time_us }}UL) {
            {{ terminate.cname }}();
            break;
        }
{% endif %}

        platform_wait_until_next_tick(start_time, {{ base_rate_us }}UL);
    }

    return 0;
}

Example of values that are passed to the template

codegen_info = create_codegen_info(
    model_code.c_code.c_code_info,
    model_settings.is_ext_mode,
)

context = codegen_info.model_dump()
render_template_to_file("templates/main.c", "build/src/main.c", context)

The runtime template

The runtime template of the independent execution mode should contain the following information

where is the entry point of the program or firmware;
when the model initialization function is called;
how are all step functions called?;
how is the base period maintained?;
how are the multiple calculation steps handled?;
what happens when there is an overflow (overrun);
how is the finite simulation time handled?;
when the model completion function is called;
which platform drivers and headers are available to the user’s C code.

The scheduler and execution of the model are discussed in more detail in the article. Model execution in independent mode.

The Python target code creates the project, and the runtime behavior of the model is set by a C/C++ runtime template.

Assembling

Method compile_model It launches a platform-based toolchain and turns the generated project into an artifact that can be downloaded or run.

Possible options:

arduino-cli compile;
CMake and the cross-compiler;
Make or Ninja;
CLI/SDK manufacturer;
building a local executable file;
firmware assembly .hex, .bin, .elf.

compile_model it must verify that the project has already been created, run the build, handle toolchain errors, and save the path to the final artifact in the target object.

An example of an assembly via CMakeWrapper

def compile_model(self, model: EngeeModel) -> TargetResponse:
    if self.project_path is None or not self.project_path.exists():
        raise TargetException(
            "Project was not generated. Call generate_executable_code first."
        )

    self.cmake.build_all(str(self.project_path))
    self.artifact_path = self.project_path / "build" / "my_firmware.elf"

    if not self.artifact_path.exists():
        raise TargetException(f"Build artifact not found: {self.artifact_path}")

    return TargetResponse(detail="compile_model")

Example of calling your own mail importer

def compile_model(self, model: EngeeModel) -> TargetResponse:
    self.builder.build_project(model.name)
    self.artifact_path = self.builder.get_artifact_path(model.name)
    return TargetResponse(detail="compile_model")

Loading

Method upload_model transfers the build result to where it should be executed. For a microcontroller, this is usually firmware via a programmer. For a Linux device, copy over SSH. For a local application, this step may not do anything if the binary file is already on the right machine.

It is important to separate the download from the build: custom toolchain errors should occur in compile_model, and communication errors with the device, programmer, or remote host are in upload_model.

Example of firmware download

def upload_model(self, model: EngeeModel) -> TargetResponse:
    if self.artifact_path is None or not self.artifact_path.exists():
        raise TargetException("Build artifact not found. Compile model first.")

    self.loader.flash(self.artifact_path)
    return TargetResponse(detail="upload_model")

An example of a local application without a separate download

def upload_model(self, model: EngeeModel) -> TargetResponse:
    return TargetResponse(detail="upload_model")

Launch

Method start_model called after loading. In the independent execution mode, there are two possible options:

the firmware has already started after downloading, and the method only returns a successful response.;
the collected artifact is an application, and the method should start the process.

If the process is started from start_model the target should save the process descriptor so that later stop_model I could have stopped him. If the platform does not support remote stopping of the model in independent mode, stop_model It can only return a successful response, but this behavior should be described in the README of the target.

An example of the firmware that runs after booting

async def start_model(self, model: EngeeModel) -> TargetResponse:
    return TargetResponse(detail="start_model")

An example of starting a local process

async def start_model(self, model: EngeeModel) -> TargetResponse:
    if self.artifact_path is None or not self.artifact_path.exists():
        raise TargetException("Executable not found. Compile model first.")

    self.current_process = subprocess.Popen([str(self.artifact_path)])
    return TargetResponse(detail="start_model")

The checklist

To target an independent execution mode, check:

the EDM-Target block is successfully read through CMIParser;
the project is being created in the target directory;
The runtime template uses CodeGenInfo, rather than the hard-coded names of the model functions;
store_model_src saves the source code of the model to the expected location;
drivers, headers, and build files end up in the project.;
compile_model checks for the project and the final artifact.;
upload_model does not mix upload with build;
start_model explicitly describes the behavior of the platform: start confirmation or process start;
The errors of the tulchain, loading, and launching turn into understandable target exceptions.