Engee documentation

Executing the model interactively

See also:
Code Generator Features
Programmatic target management
Custom Engee support packages.Integrations

The interactive mode adds a two-way communication channel to the normal launch of the model. Engee and runtime on the target platform. Through this channel Engee retrieves the values of the selected signals and sends commands to change the configured parameters without reassembling and loading the model.

In the current implementation of user targets, this mode is built around XCP slave, which is embedded in the runtime of the model on the target platform.

Basic concepts

A logical point is the signal or output port of the model that the user has selected to monitor during execution. When generating the C code Engee stores information about such points in metadata and generates Si data from which runtime can read values. In the Python representation of code generation, this data is reflected in CodeInfo and in a simplified way for templates CodeGenInfo.logging_signals.

A configurable parameter is a model parameter whose value can be changed after startup. In the generated C code, such parameters usually fall into a separate structure of model parameters. CodeGenInfo this part is described through tunable_params. For interactive mode, it is important that the corresponding C structure is available in the target application’s memory and is not deleted by the optimizer.

The target platform interface is a physical or programmatic channel through which Engee exchanges data with the model. It can be UART, TCP, UDP, USB CDC, debug channel or other transport. The very fact that there are logable points in the model does not transfer data to Engee: a communication channel is needed at runtime on the target platform.

An ELF file is an assembled artifact with symbols that the target’s Python code uses to find the addresses of signals and parameters in the target application’s memory. For the interactive summary mode .elf It must be available on the client side, and information about the necessary symbols cannot be deleted from it.

The XCP Protocol

XCP (Universal Measurement and Calibration Protocol) is an ASAM MCD-1 XCP standard for accessing internal application data during its operation. The protocol is usually used to measure signals, calibrate parameters, and debug embedded applications without reassembling and flashing after each parameter change.

In this guide Engee it acts as XCP master, and the runtime of the model on the target platform contains XCP slave. The master manages the session: connects to the slave, configures signal transmission, and sends read and write commands. The slave resides inside the target application, executes the master commands, and reads or modifies the model’s memory.

The official description of the standard is available at the link ASAM MCD-1 XCP.

XCP layers: protocol and transport

XCP separates the logical protocol and the transport layer.

The protocol layer describes which commands exist and what they mean: connecting, reading and writing memory, configuring DAQ, starting and stopping transmission. This level does not depend on whether the exchange is over UART, TCP, UDP, CAN, USB or another channel.

The transport layer describes how XCP packets are transmitted over a specific interface: how to open a connection, how to highlight packet boundaries, what is the maximum frame size, whether there are confirmations, how errors and timeouts are handled.

This means that you need to define:

  1. Which XCP slave will be embedded in the runtime of the model.

  2. What kind of transport Engee will exchange with this slave.

As part of the executable file, the protocol part of XCP slave is already supplied with the target infrastructure. For the new platform, you need to select or implement only the transport and platform layer.

Master, slave, and communication session

The XCP session begins with the connection of the master to the slave. Once connected, the master gets the basic features of the slave: supported transport, packet sizes, available resources, and restrictions. The master can then execute commands.

For interactive mode Engee Three groups of teams are important:

  • session service commands: connect, check status, complete exchange;

  • memory access commands: read the signal value or write a new parameter value;

  • DAQ commands: set up lists of measured data and start their regular transmission.

In classic XCP, the slave does not know in advance which signals the user wants to observe. It provides universal access to memory, and the master adjusts which addresses and data sizes need to be read. Engee these addresses are taken from .elf and code generation metadata.

Data addressing

XCP works with target application objects via memory addresses. For Engee such objects are:

  • values of the logged signals;

  • configurable model parameters;

  • the service structures needed for the runtime of the interactive execution mode.

The address that the master sees must be correctly converted to a pointer inside the slave. The platform macro is responsible for this. XCP_ADDRESS_GET. On a microcontroller without virtual memory, the address can often be used directly. On host platforms with PIE/ASLR addresses from .elf there may be offsets, so the platform layer must take into account the base address of the uploaded image.

Because of this, the interactive execution mode is important .elf with symbols: The target’s Python code must find the data addresses, and the XCP slave must be able to access the target application’s memory using these addresses.

DAQ, ODT, and Events

DAQ (data acquisition) is an XCP mechanism for streaming data from a slave to a master. The master pre-configures the composition of the data being sent, and the slave transmits it when an event occurs.

The DAQ setup consists of several levels:

  • DAQ list — a list of data that is activated by an event;

  • ODT (object descriptor table) is a group of elements that fit into a single DTO package.;

  • ODT entry — one measurable object: address, size, and additional attributes;

  • event channel is a runtime event for which the slave must collect and send data.

In the model Engee Such an event is usually the completion of a model step. Runtime causes rtExtModeUpload, and the XCP slave sends the data.

When configuring XCP, it is important to consider the dimensions:

  • how many signals can the user record at the same time;

  • how many bytes can fit in one DTO packet;

  • how many ODT records are allowed in one ODT;

  • how many DAQ lists are available on the platform;

  • whether a timestamp is needed in the DAQ packet and how many bytes it takes.

If the limits are too small, some of the signals will not be able to be included in the stream. If the limits are too large, the XCP slave will take up more RAM, which is critical for small MCUs.

Recording parameters

Changing a parameter interactively is writing a new value to the target application’s memory. The master selects the parameter address and sends the record data. The slave checks the command and writes bytes to memory.

To work correctly, you need to:

  • the parameter was actually in memory and was not deleted by the optimizer.;

  • the size and type of data matched the code generation information.;

  • the write did not occur at the same time as an unsafe read from another thread or interrupt handler (ISR);

  • The model’s runtime was ready to use the new value in the next steps.

If the parameter is part of the model structure, the change usually begins to affect the calculation from the next time the model accesses this structure. If the value is copied to the local state during initialization, a simple XCP write to the source parameter may not have the expected effect.

Target Configuration

To configure XCP, you need to define:

  1. Transport: UART, TCP, UDP or other channel.

  2. DAQ limits: the number of lists, ODTs, and ODT elements.

  3. The method of converting an XCP address to a pointer.

  4. Synchronization model: whether mutexes are needed, where threads or interrupt handlers (thread/ISR) are possible.

General data flow

The interactive execution mode works as a chain of several parts:

  1. Engee generates the model’s C code and metadata about signals, parameters, and entry points.

  2. The target generates a runtime template for the interactive mode.

  3. XCP slave and platform transport are being added to the project.

  4. The toolchain collects the artifact and .elf with the symbols of the model.

  5. Target’s Python code parses .elf and gets the addresses of logged points and parameters.

  6. Engee connects to the XCP slave via the selected interface.

  7. The scheduler of the model performs step functions and calls DAQ events.

  8. XCP slave reads values from memory and passes them to Engee.

  9. When changing the parameter Engee sends a write command, and the XCP slave writes the new value to the model’s memory.

Python-the target code

The interactive mode target is usually inherited from BaseTarget and XCPTarget. BaseTarget sets methods for managing the operation of the model, and XCPTarget provides Python exchange code: configuring endpoints, creating streaming queues, starting data reading, and changing parameters.

An example of basic inheritance

from targets.base_target import BaseTarget
from targets.xcp_target.xcp_target import XCPTarget


class MyTarget(BaseTarget, XCPTarget):
    def __init__(self) -> None:
        XCPTarget.__init__(self)
        ...

In generate_executable_code We need to prepare a draft interactive mode.:

  • check model_settings.is_ext_mode;

  • choose a runtime template with challenges rtExtMode*;

  • save the source code of the model;

  • add shared XCP slave sources via get_xcp_slave_src_list;

  • add platform transport files;

  • add necessary paths to header files and compilation definitions;

  • provide assembly .elf with symbols.

In start_model We need to prepare a connection Engee with an already built and running model:

  1. Set up an endpoint via XCPTarget.set_endpoint.

  2. Find the final result .elf.

  3. Get data about signals and parameters via get_engee_elf_info.

  4. Call up XCPTarget.setup_streaming.

  5. Return TargetResponse with data_stream_q, controll_stream_q and discovered_signals.

start_stream usually causes XCPTarget._start_stream, change_param causes set_param, and stop_model stops the flow through XCPTarget.stop_stream.

An example of a scheme of methods for the Python code of the target

from targets.xcp_target.engee_elf_info import get_engee_elf_info
from targets.xcp_target.xcp_target import get_xcp_slave_src_list


async def start_model(self, model: EngeeModel) -> TargetResponse:
    XCPTarget.set_endpoint(self, interface="uart", url=self.target_block.com_port)

    elf_data = get_engee_elf_info(
        elf_path=self.elf_path,
        code_info=self.code_info,
    )

    data_q, control_q, discovered = await XCPTarget.setup_streaming(
        self,
        elf_data,
        self.model_settings,
    )

    return TargetResponse(
        detail="start_model",
        data_stream_q=data_q,
        controll_stream_q=control_q,
        discovered_signals=discovered,
    )


async def start_stream(self, simulation_uuid: str) -> Result:
    await XCPTarget._start_stream(self, simulation_uuid)
    return Result.success()


async def change_param(self, block_key, param, data) -> TargetResponse:
    await self.set_param(block_key, param, data)
    return TargetResponse(detail="Param is changed")


async def stop_model(self) -> TargetResponse:
    await XCPTarget.stop_stream(self)
    return TargetResponse(detail="stop_model")

Runtime-an interactive execution mode template

The interactive execution mode runtime is built on top of the independent mode scheduler, but adds API calls to the interactive mode adapter (external-mode adapter) from ext_work.h.

Minimum order:

  1. Initialize the model.

  2. Call up rtExtModeCheckInit.

  3. Wait for the start command via rtExtModeWaitForStartPkt.

  4. Perform the model’s step functions in the loop.

  5. After the call step rtExtModeUpload to publish a DAQ event.

  6. At each step, call rtExtModeOneStep to process the master commands.

  7. When requesting a stop, call terminate.

  8. Upon normal completion, call rtExtModeShutdown if the platform supports the correct shutdown (clean shutdown).

It is important that rtExtModeOneStep He was called up regularly. If the scheduler is blocked in the driver for a long time, while waiting for the peripheral or the sleep function, Engee it will receive data and commands to change parameters with a delay.

An example of a runtime cycle scheme

#include "ext_work.h"

RTWExtModeInfo ext_mode_info;
boolean_T stop_requested = false;

model_initialize();

rtExtModeCheckInit(1);
rtExtModeWaitForStartPkt(&ext;_mode_info, 1, &stop;_requested);
rtERTExtModeStartMsg();

while (!stop_requested) {
    run_due_model_steps();
    rtExtModeUpload(0, getCurrentTimestamp());
    rtExtModeOneStep(&ext;_mode_info, 1, &stop;_requested);
    wait_until_next_tick();
}

model_terminate();
rtExtModeShutdown(1);

XCP slave in Engee.Integrations

The shared XCP slave is included in the supplied executable file. engee-device-manager. The Python code of the target adds it to the project via get_xcp_slave_src_list, and adds the platform layer separately.

The XCP slave structure:

  • include/ext_work.h — The API that calls the model’s runtime template;

  • src/ext_work.c — an adapter between rtExtMode* and XCP slave;

  • src/xcp_daq.c, src/xcp_daq.h — processing of XCP commands, DAQ lists, reading signals and writing parameters;

  • src/xcp_frame/xcp_frame_uart.c — framing for UART;

  • src/xcp_frame/xcp_frame_tcp.c — framing for TCP;

  • src/xcp_frame/xcp_frame_udp.c — framing for UDP;

  • include/xcp_config.h — configuration of XCP slave at the compilation stage;

  • include/rtiostream.h — transport I/O interface.

Main functions rtExtMode*:

  • rtExtModeCheckInit — initializes the XCP slave and opens the transport;

  • rtExtModeWaitForStartPkt — waiting for the start command from the master;

  • rtExtModeUpload — publishes a DAQ event;

  • rtExtModeOneStep — performs one non-blocking command processing pass;

  • rtExtModePauseIfNeeded — serves commands in a pause state;

  • rtExtModeShutdown — resets the slave state and closes the runtime;

  • rtExtModeParseArgs and rtERTExtModeStartMsg left for compatibility with the generated interactive mode API (external-mode).

Compilation definitions for XCP

The transport must be selected in the project.:

-DXCP_UART_TRANSPORT

or

-DXCP_TCP_TRANSPORT

or

-DXCP_UDP_TRANSPORT

For an embedded platform, they usually add:

-DXCP_CUSTOM_PLATFORM

You can use a ready-made platform layer for the x86 host platform.:

-DXCP_PLATFORM_X86

xcp_config.h It also contains customizable sizes.:

  • XCP_MAX_ODT_ENTRIES_COUNT — maximum of elements in one ODT;

  • XCP_MAX_DAQ_LISTS — maximum of DAQ lists;

  • XCP_DAQ_TIMESTAMP_SIZE — the size of the timestamp in the DAQ package;

  • XCP_DAQ_DATA_SIZE — maximum useful DAQ data in one DTO;

  • XCP_MAX_SHORT_DOWNLOAD_DATA_LEN — maximum data for recording a parameter with a single command;

  • XCP_MAX_CTO_LEN — the size of the CTO command/response package;

  • XCP_MAX_DTO_LEN — the size of the DTO data transmission packet.

For small MCUs, these values can be reduced, but after that you need to check that the selected log points fit within the DAQ limits.

User Platform

If the XCP slave build is not running on a ready-made x86 layer, you need to enable -DXCP_CUSTOM_PLATFORM and provide xcp_platform_custom.h.

The minimum set of platform requirements includes the implementation of the following macros and directives:

  • XCP_PRINTF — diagnostic output or an empty macro;

  • XCP_MUTEX_DEFINE;

  • XCP_MUTEX_INIT;

  • XCP_MUTEX_LOCK;

  • XCP_MUTEX_UNLOCK;

  • XCP_ADDRESS_GET;

  • XCP_SLEEP;

  • packing/alignment macros: XCP_PRAGMA_PACK_BEGIN, XCP_PRAGMA_PACK_END, XCP_ATTRIBUTE_ALIGNED, XCP_ATTRIBUTE_PACKED.

Basic example xcp_platform_custom.h

#ifndef XCP_PLATFORM_CUSTOM_H
#define XCP_PLATFORM_CUSTOM_H

#include <stdint.h>
#include "rtwtypes.h"

#define XCP_PRINTF(...)

#define XCP_MUTEX_DEFINE(lock)
#define XCP_MUTEX_INIT(lock)
#define XCP_MUTEX_LOCK(lock)
#define XCP_MUTEX_UNLOCK(lock)

#define XCP_ADDRESS_GET(addressExtension, address) \
    ((uint8_T *)((uintptr_t)(address)))

#define XCP_SLEEP(seconds, microseconds)  platform_sleep(seconds, microseconds)

#define PRAGMA(n) _Pragma(#n)
#define XCP_PRAGMA_PACK_BEGIN(n) PRAGMA(pack(push, n))
#define XCP_PRAGMA_PACK_END()    PRAGMA(pack(pop))
#define XCP_ATTRIBUTE_ALIGNED(n)
#define XCP_ATTRIBUTE_PACKED

#endif

XCP_ADDRESS_GET — a mission-critical macro. It translates the address from the XCP command into a real pointer in the address space of the target application.

For microcontrollers without virtual memory, direct address-to-pointer conversion is usually sufficient. For PIE/ASLR host applications, the address from the ELF file can be set as an offset, and the base address of the downloaded image must be added to it. This has already been implemented in ritm-xcpslave/platform_x86.

Mutexes depend on the execution model:

  • if the entire XCP slave is called from only one main loop, critical sections may be empty.;

  • if I/O, DAQ, or scheduler are running from different tasks, threads, or interrupt handlers, a real lock is needed.;

  • blocking should not permanently suspend the execution of the basic step of the model.

Custom transport

The transport must implement an API rtIOStream:

int rtIOStreamOpen(int argc, void *argv[]);
int rtIOStreamSend(int streamID, const void *src, size_t size, size_t *sizeSent);
int rtIOStreamRecv(int streamID, void *dst, size_t size, size_t *sizeRecvd);
int rtIOStreamClose(int streamID);

Rules for rtIOStreamSend:

  • Return it RTIOSTREAM_NO_ERROR if the transport is working properly;

  • if you can’t send the data now, install *sizeSent = 0;

  • do not confirm partial frame sending if the selected crop layer does not support partial sending.;

  • if there is a real transport error, return RTIOSTREAM_ERROR.

Rules for rtIOStreamRecv:

  • The function must be non-blocking or have a short time limit.;

  • if there is no data, install *sizeRecvd = 0 and return it RTIOSTREAM_NO_ERROR;

  • you can return fewer bytes than requested: the frame parser will finish reading the packet with the following calls;

  • in case of a transport error, return RTIOSTREAM_ERROR.

For UART, the current frame format uses a start byte. 0xFF, the length of the payload, the data itself, and the XOR CRC. A UART transport must transmit bytes without echo, text processing, or end-of-line character conversion.

What to copy to the project

For an interactive execution project, you need:

  • ritm-xcpslave/include/*.h;

  • ritm-xcpslave/src/ext_work.c;

  • ritm-xcpslave/src/xcp_daq.c;

  • ritm-xcpslave/src/xcp_daq.h;

  • ritm-xcpslave/src/xcp_frame/xcp_frame.h;

  • one frame handler file: xcp_frame_uart.c, xcp_frame_tcp.c or xcp_frame_udp.c;

  • platform - based xcp_platform_custom.h or ready-made platform_x86;

  • realization rtIOStreamOpen, rtIOStreamSend, rtIOStreamRecv, rtIOStreamClose;

  • if necessary, your own rtwtypes.h and ext_mode_types.h.

ELF and symbols

For interactive execution Engee It must match the logged points and model parameters with the addresses in the target application’s memory. To do this, the Python code of the target uses .elf and get_engee_elf_info.

In practice, it is necessary to ensure the following:

  • summary .elf must be available on the client side;

  • the symbols of signals and parameters should not be deleted by strip;

  • The optimizer should not delete the structures that are needed for logging and changing parameters.;

  • the structure layer must correspond to the information from CodeInfo;

  • for a non-standard architecture, you need to check ELF parsing, pointer size, and addressing rules.

The checklist

  1. The model has a logical point and at least one verification signal.

  2. XCP slave has been added to the project.

  3. One transport is selected: UART, TCP or UDP.

  4. The compilation of the transport and platform is defined.

  5. Implemented or selected xcp_platform_custom.h.

  6. Implemented rtIOStream for the selected interface.

  7. The runtime template causes rtExtModeCheckInit, rtExtModeWaitForStartPkt, rtExtModeUpload, rtExtModeOneStep and rtExtModeShutdown.

  8. .elf it contains the necessary characters and is available to the Python target.

  9. XCPTarget.setup_streaming returns queues and a list of found signals.

  10. The start, signal reading, parameter change, and stop were tested on the demo model.