Calling code in C and Fortran

Julia functions can be passed to unmanaged C functions that take function pointers as arguments. For example, this way you can ensure compliance with the prototype C of the following type.

The macro @cfunction creates a C-compatible function pointer to call the Julia function. Method @cfunction has the following arguments:

A classic example is the `qsort' function from the C standard library, declared as follows.

The base argument is a pointer to an array with a length of nitems', each element of which has a size of `size bytes. 'compare` is a callback function that takes pointers to two elements (a and b) and returns an integer greater than or less than zero if element a should be before or after element b (or zero if any order is allowed).

Now let’s assume that Julia has a one-dimensional array of values A that needs to be sorted using the qsort function (rather than the built-in Julia sort function). Before calling qsort and passing arguments, you need to write a comparison function.:

'qsort` expects a comparison function that returns a value of type C `int', so we annotate the returned type as `Cint'.

To pass this function to C, we get its address using the macro @cfunction:

The macro '@cfunction` accepts three arguments: the Julia function (mycompare'), the return type (`Cint), and a literal tuple of input argument types, in this case for sorting an array of elements of type Cdouble' (`Float64).

In the final form, the `qsort' call looks like this.

As shown in the example, the original Julia A array is now sorted.: [-2.7, 1.3, 3.1, 4.4]. Please note that Julia performs array conversion to Ptr{Cdouble}, calculating the size of the element type in bytes, etc.

For interest, try inserting the string println("mycompare($a, $b)") into mycompare'. This will allow you to see what comparisons the `qsort function makes (and check if it actually calls the passed Julia function).

It is extremely important that the type declarations in C and Julia match exactly. Due to inconsistencies, code that works correctly on one system may fail or produce uncertain results on another.

Please note that the C header files are not used in the process of calling functions in C: you are responsible for matching the types and signatures of Julia calls to the C header file[using https://github.com/ihnorton/Clang.jl [of the Clang package], you can automatically create Julia code based on the C header file].

Let’s start with a simple example of a C shell that returns the type Ptr:

In https://www.gnu.org/software/gsl /[GNU scientific library] (here it is assumed that it is available by the name :libgsl) the opaque pointer gsl_permutation * is defined as the return type of the C function gsl_permutation_alloc. Since the user code does not need to access the gsl_permutation structure, to implement the corresponding Julia shell, it is enough to declare a new type gsl_permutation, which has no internal fields and whose only purpose is to specify the type Ptr in the parameter. For the function ccall the return type Ptr' has been declared{gsl_permutation}, because the memory area that is allocated using the output_ptr pointer and to which it points is controlled by the C environment.

The input argument n is passed by value, so the signature of the input data of the function has a very simple form: ::Csize_t'. There is no need for `Ref or Ptr'. (If the shell had called the Fortran function instead, the corresponding input data signature would have been `+::Ref{Csize_t}+, since variables in Fortran are passed by pointers.) Moreover, n can have any type that can be converted to the integer value Csize_t'; function `ccall implicitly calls Base.cconvert(Csize_t, n).

Here is another example, this time of the shell of the corresponding destructor:

Here is a third example in which Julia arrays are passed:

The wrapped function C returns an integer error code, and the results of calculating the Bessel function J are entered into the Julia result_array array. This variable is declared as Ref{Cdouble}, since a block of memory is allocated for it and managed by the Julia code. Implicit challenge Base.cconvert(Ref{Cdouble}, result_array) decompresses Julia’s pointer to the Julia array data structure into a form understandable to C.

In the following example, using ccall', a function from the Fortran standard library (libBLAS) is called to calculate the scalar product. Please note: the arguments in this case are compared slightly differently than above, since the comparison is made from Julia to Fortran. For each type of argument, `Ref or Ptr' is specified. This conversion convention may depend on the specific Fortran compiler and operating system and is most likely not reflected in the documentation. However, many Fortran compiler implementations require enclosing each argument type in a `Ref (or Ptr, where applicable):

When transferring data to @ccall, it is better not to use the function pointer. Instead, define a method Base.cconvertand pass the variables to @ccall directly. @ccall automatically protects all its arguments from garbage collection until the call returns control. If the C API will store a reference to the memory allocated by the Julia code, after returning control to the @ccall call, it is necessary to ensure that the object is accessible to the garbage collector. The recommended way is to store these values in a global variable of type Array'.{Ref,1}, until the C library notifies that work with them is completed.

Whenever you create a pointer to Julia data, that data must exist as long as the pointer is in use. Many methods in Julia, for example unsafe load and String, create copies of the data instead of taking control of the buffer. This allows you to safely delete (or modify) the source data from memory so that it does not affect the execution of Julia code. An important exception is the method unsafe_wrap, which, for performance reasons, uses the base buffer in a shared mode (in other words, takes control of it).

The garbage collector does not guarantee any particular order of disposal of objects. In other words, if object a contains a reference to object b and both are subject to garbage collection, there is no guarantee that object b will be eliminated after a. If object b must exist to eliminate object a, a different approach is required.

In some cases, the name or path of the required library is unknown in advance and must be determined at runtime. In such cases, the library component specification can be a function call, for example, find_blas().dgemm'. The call expression will be executed when the `ccall operation is performed. However, it is assumed that once the library location is determined, it does not change, so the result of the call can be cached and reused. Thus, the expression can be executed any number of times, and returning different values can give an indeterminate result.

If even more flexibility is required, calculated values can be used as function names using the function eval as follows.

This expression generates a name using a string, and then substitutes it into a new expression @ccall, which is then evaluated. Note that the eval function is a high-level function, so local variables will not be available in this expression (unless their values are substituted with $). For this reason, the eval function is usually used to create only top-level definitions, for example, when encapsulating libraries containing many similar functions. A similar example is possible for a macro @cfunction.

However, such code will run very slowly and with memory leaks, so try not to use this option. The next section explains how to achieve the same result using indirect calls.

The first argument in the @ccall call can also be an expression evaluated at runtime. The result of such an expression should be a pointer Ptr, which will be used as the address of the called native function. This behavior occurs when the first argument @ccall contains references to non-constant values, such as local variables, function arguments, or non-constant global variables.

For example, you can define a function using dlsym and then cache it in a shared link within a session. For example:

The first argument @cfunction can be marked with the symbol $. In this case, the return value will be a struct CFunction object with an argument closure. This returned object must exist until its use is completed. The contents and code of the cfunction pointer will be destroyed by the function finalizer after deleting this link and exiting. This is usually not required, since there is no such functionality in C, but it can be useful when working with poorly designed APIs that do not provide a separate parameter for the closure environment.

Sometimes it is useful to close (unload) a library so that it can be downloaded again. For example, when writing C code to be used with Julia, you may need to compile, call the C code from Julia, then close the library, make changes, compile again, and upload new changes. To do this, you can either restart Julia, or use the Libdl functions to manage the library explicitly, like this.

Note that when using @ccall with input data (for example, @ccall'./my_lib.so ".my_fcn(...)::Cvoid) the library opens implicitly and may not be explicitly closed.

To call C functions with a variable number of arguments, you can separate the required arguments from the arguments of a variable number in the argument list with a semicolon (semicolon). Below is an example of calling the printf function.

This is another interface that can serve as an alternative to `@ccall'. It is a little less convenient, but it allows you to set challenge agreement.

Method ccall has the following arguments:

The conversion table between the macro and function interfaces is shown below.

The second argument of the ccall call (immediately before the return type) can be a call agreement specifier (currently, the @ccall macro does not allow you to specify a call agreement). If no specifier is specified, the C calling convention adopted by default for the platform is used. In addition, the following conventions are supported: stdcall', `cdecl', `fastcall and thiscall' (not valid on 64-bit Windows systems). For example, this is what a call to the `gethostname`ccall function looks like, similar to the one above, but with the correct signature for Windows (code taken from `base/libc.jl'):

For more information, see https://llvm.org/docs/LangRef.html#calling-conventions [LLVM language reference].

There is another special call agreement llvmcall, which allows you to directly insert calls into internal LLVM instructions. This can be especially useful when writing code for unusual platforms such as GPGPU. For example, for https://llvm.org/docs/NVPTXUsage.html [CUDA] you may need to get a stream index.:

As with any ccall call, it is important to get the correct signature of the arguments. Also, note that there is no compatibility wrapper that ensures that the internal instruction is correct and valid for the current purpose, unlike the Julia functions provided by Core.Intrinsics`.

Global variables provided by native libraries can be accessed by name using the function cglobal. Function arguments 'cglobal` are a symbolic specification (same as for the call ccall) and the type of value stored in the variable:

The result is a pointer to the address of the value. Using this pointer, you can perform operations with the value using functions unsafe load and unsafe_store!.

The methods presented below are unsafe, as an incorrect pointer or type declaration can cause Julia to suddenly stop working.

If the pointer is Ptr{T}, the contents of type T, as a rule, can be copied from the memory area to which it refers to the Julia object by calling unsafe load(ptr, [index]). The index argument is optional (by default it is 1) and is indexed from 1 according to the convention adopted in Julia. The behavior of this function is intentionally made to resemble the behavior of functions getindex and 'setindex! (for example, the syntax matches `[]).

A new object is returned containing a copy of the contents of the memory area referenced by the pointer. After that, this memory area can be safely released.

If the type of T is Any, it is assumed that the memory area contains a reference to the Julia object ('jl_value_t*). The result will be a reference to this object, and the object itself is not copied. In this case, it is necessary to ensure that the object is accessible to the garbage collector (pointers are not taken into account, unlike a new reference) so that the memory is not cleared ahead of time. Please note: if the object was not originally placed in memory by the Julia code, the new object will not be eliminated by the Julia garbage collector. If the object `Ptr is itself jl_value_t*, it can be converted back to a reference to the Julia object using the function unsafe_pointer_to_objref(ptr). (Julia v values can be converted to jl_value_t' pointers*`how `+Ptr{Cvoid}+ by calling pointer_from_objref(v).)

Reverse operation (writing data to Ptr{T}) can be performed using the function unsafe_store!(ptr, value, [index]). Currently, this feature is supported only for primitive types or other immutable types of structures without pointers (isbits).

If the operation returns an error, it may not have been implemented yet. This should be reported so that we can fix the problem.

If the pointer represents an array of ordinary data (a primitive type or an immutable structure), the function unsafe wrap(Array, ptr,dims, own = false) may be more useful. The last parameter must be set to true if the Julia environment is to monitor the base buffer and call free(ptr) after eliminating the returned Array object. If the 'own` parameter is omitted or set to false, the caller must ensure the buffer exists while it needs to be accessed.

Arithmetic operations with the type Ptr in Julia (for example, +) are performed differently than with pointers in C. When adding an integer to Ptr in Julia, the pointer is shifted by a certain number of _bytes, not elements. Due to this, the address values obtained as a result of arithmetic operations with pointers do not depend on the types of pointer elements.

Some C libraries make callbacks from another thread, and since Julia is not a thread-safe language, additional precautions need to be taken. In particular, you should set up a two-tier system: the C callback should only schedule the execution of a "real" callback (via the Julia event loop). To do this, create an object AsyncCondition and apply the function to it wait:

The callback passed to C should only make the call 'ccall` applied to :uv_async_send with passing cond.handle as an argument. There should be no memory allocation or other interactions with the Julia runtime.

Note that events can be combined, so that multiple calls to uv_async_send can result in a single notification to activate a condition.

For more information about passing callbacks to C libraries, see https://julialang.org/blog/2013/05/callback [this blog entry].

Tools for creating C bindings++ can be found in the package https://github.com/JuliaInterop/CxxWrap.jl [CxxWrap].