Calling C and Fortran Code

It is possible to pass Julia functions to native C functions that accept function pointer arguments. For example, to match C prototypes of the form:

The macro @cfunction generates the C-compatible function pointer for a call to a Julia function. The arguments to @cfunction are:

A classic example is the standard C library qsort function, declared as:

The base argument is a pointer to an array of length nitems, with elements of size bytes each. compare is a callback function which takes pointers to two elements a and b and returns an integer less/greater than zero if a should appear before/after b (or zero if any order is permitted).

Now, suppose that we have a 1-d array A of values in Julia that we want to sort using the qsort function (rather than Julia’s built-in sort function). Before we consider calling qsort and passing arguments, we need to write a comparison function:

qsort expects a comparison function that return a C int, so we annotate the return type to be Cint.

In order to pass this function to C, we obtain its address using the macro @cfunction:

@cfunction requires three arguments: the Julia function (mycompare), the return type (Cint), and a literal tuple of the input argument types, in this case to sort an array of Cdouble (Float64) elements.

The final call to qsort looks like this:

As the example shows, the original Julia array A has now been sorted: [-2.7, 1.3, 3.1, 4.4]. Note that Julia takes care of converting the array to a Ptr{Cdouble}), computing the size of the element type in bytes, and so on.

For fun, try inserting a println("mycompare($a, $b)") line into mycompare, which will allow you to see the comparisons that qsort is performing (and to verify that it is really calling the Julia function that you passed to it).

It is critical to exactly match the declared C type with its declaration in Julia. Inconsistencies can cause code that works correctly on one system to fail or produce indeterminate results on a different system.

Note that no C header files are used anywhere in the process of calling C functions: you are responsible for making sure that your Julia types and call signatures accurately reflect those in the C header file.^[2]

Let’s start with a simple example of a C wrapper that returns a Ptr type:

The GNU Scientific Library (here assumed to be accessible through :libgsl) defines an opaque pointer, gsl_permutation *, as the return type of the C function gsl_permutation_alloc. As user code never has to look inside the gsl_permutation struct, the corresponding Julia wrapper simply needs a new type declaration, gsl_permutation, that has no internal fields and whose sole purpose is to be placed in the type parameter of a Ptr type. The return type of the ccall is declared as Ptr{gsl_permutation}, since the memory allocated and pointed to by output_ptr is controlled by C.

The input n is passed by value, and so the function’s input signature is simply declared as ::Csize_t without any Ref or Ptr necessary. (If the wrapper was calling a Fortran function instead, the corresponding function input signature would instead be ::Ref{Csize_t}, since Fortran variables are passed by pointers.) Furthermore, n can be any type that is convertible to a Csize_t integer; the ccall implicitly calls Base.cconvert(Csize_t, n).

Here is a second example wrapping the corresponding destructor:

Here is a third example passing Julia arrays:

The C function wrapped returns an integer error code; the results of the actual evaluation of the Bessel J function populate the Julia array result_array. This variable is declared as a Ref{Cdouble}, since its memory is allocated and managed by Julia. The implicit call to Base.cconvert(Ref{Cdouble}, result_array) unpacks the Julia pointer to a Julia array data structure into a form understandable by C.

The following example utilizes ccall to call a function in a common Fortran library (libBLAS) to compute a dot product. Notice that the argument mapping is a bit different here than above, as we need to map from Julia to Fortran. On every argument type, we specify Ref or Ptr. This mangling convention may be specific to your Fortran compiler and operating system and is likely undocumented. However, wrapping each in a Ref (or Ptr, where equivalent) is a frequent requirement of Fortran compiler implementations:

When passing data to a @ccall, it is best to avoid using the pointer function. Instead define a convert method and pass the variables directly to the @ccall. @ccall automatically arranges that all of its arguments will be preserved from garbage collection until the call returns. If a C API will store a reference to memory allocated by Julia, after the @ccall returns, you must ensure that the object remains visible to the garbage collector. The suggested way to do this is to make a global variable of type Array{Ref,1} to hold these values until the C library notifies you that it is finished with them.

Whenever you have created a pointer to Julia data, you must ensure the original data exists until you have finished using the pointer. Many methods in Julia such as unsafe_load and String make copies of data instead of taking ownership of the buffer, so that it is safe to free (or alter) the original data without affecting Julia. A notable exception is unsafe_wrap which, for performance reasons, shares (or can be told to take ownership of) the underlying buffer.

The garbage collector does not guarantee any order of finalization. That is, if a contained a reference to b and both a and b are due for garbage collection, there is no guarantee that b would be finalized after a. If proper finalization of a depends on b being valid, it must be handled in other ways.

In some cases, the exact name or path of the needed library is not known in advance and must be computed at run time. To handle such cases, the library component specification can be a function call, e.g. find_blas().dgemm. The call expression will be executed when the ccall itself is executed. However, it is assumed that the library location does not change once it is determined, so the result of the call can be cached and reused. Therefore, the number of times the expression executes is unspecified, and returning different values for multiple calls results in unspecified behavior.

If even more flexibility is needed, it is possible to use computed values as function names by staging through eval as follows:

This expression constructs a name using string, then substitutes this name into a new @ccall expression, which is then evaluated. Keep in mind that eval only operates at the top level, so within this expression local variables will not be available (unless their values are substituted with $). For this reason, eval is typically only used to form top-level definitions, for example when wrapping libraries that contain many similar functions. A similar example can be constructed for @cfunction.

However, doing this will also be very slow and leak memory, so you should usually avoid this and instead keep reading. The next section discusses how to use indirect calls to efficiently achieve a similar effect.

The first argument to @ccall can also be an expression evaluated at run time. In this case, the expression must evaluate to a Ptr, which will be used as the address of the native function to call. This behavior occurs when the first @ccall argument contains references to non-constants, such as local variables, function arguments, or non-constant globals.

For example, you might look up the function via dlsym, then cache it in a shared reference for that session. For example:

The first argument to @cfunction can be marked with a $, in which case the return value will instead be a struct CFunction which closes over the argument. You must ensure that this return object is kept alive until all uses of it are done. The contents and code at the cfunction pointer will be erased via a finalizer when this reference is dropped and atexit. This is not usually needed, since this functionality is not present in C, but can be useful for dealing with ill-designed APIs which don’t provide a separate closure environment parameter.

It is sometimes useful to close (unload) a library so that it can be reloaded. For instance, when developing C code for use with Julia, one may need to compile, call the C code from Julia, then close the library, make an edit, recompile, and load in the new changes. One can either restart Julia or use the Libdl functions to manage the library explicitly, such as:

Note that when using @ccall with the input (e.g., @ccall "./my_lib.so".my_fcn(...)::Cvoid), the library is opened implicitly and it may not be explicitly closed.

To call variadic C functions a semicolon can be used in the argument list to separate required arguments from variadic arguments. An example with the printf function is given below:

There is another alternative interface to @ccall. This interface is slightly less convenient but it does allow one to specify a calling convention.

The arguments to ccall are:

A table of translations between the macro and function interfaces is given below.

The second argument to ccall (immediatel preceding return type) can optionally be a calling convention specifier (the @ccall macro currently does not support giving a calling convention). Without any specifier, the platform-default C calling convention is used. Other supported conventions are: stdcall, cdecl, fastcall, and thiscall (no-op on 64-bit Windows). For example (from base/libc.jl) we see the same gethostname``ccall as above, but with the correct signature for Windows:

For more information, please see the LLVM Language Reference.

There is one additional special calling convention llvmcall, which allows inserting calls to LLVM intrinsics directly. This can be especially useful when targeting unusual platforms such as GPGPUs. For example, for CUDA, we need to be able to read the thread index:

As with any ccall, it is essential to get the argument signature exactly correct. Also, note that there is no compatibility layer that ensures the intrinsic makes sense and works on the current target, unlike the equivalent Julia functions exposed by Core.Intrinsics.

Global variables exported by native libraries can be accessed by name using the cglobal function. The arguments to cglobal are a symbol specification identical to that used by ccall, and a type describing the value stored in the variable:

The result is a pointer giving the address of the value. The value can be manipulated through this pointer using unsafe_load and unsafe_store!.

The following methods are described as "unsafe" because a bad pointer or type declaration can cause Julia to terminate abruptly.

Given a Ptr{T}, the contents of type T can generally be copied from the referenced memory into a Julia object using unsafe_load(ptr, [index]). The index argument is optional (default is 1), and follows the Julia-convention of 1-based indexing. This function is intentionally similar to the behavior of getindex and setindex! (e.g. [] access syntax).

The return value will be a new object initialized to contain a copy of the contents of the referenced memory. The referenced memory can safely be freed or released.

If T is Any, then the memory is assumed to contain a reference to a Julia object (a jl_value_t*), the result will be a reference to this object, and the object will not be copied. You must be careful in this case to ensure that the object was always visible to the garbage collector (pointers do not count, but the new reference does) to ensure the memory is not prematurely freed. Note that if the object was not originally allocated by Julia, the new object will never be finalized by Julia’s garbage collector. If the Ptr itself is actually a jl_value_t*, it can be converted back to a Julia object reference by unsafe_pointer_to_objref(ptr). (Julia values v can be converted to jl_value_t* pointers, as Ptr{Cvoid}, by calling pointer_from_objref(v).)

The reverse operation (writing data to a Ptr{T}), can be performed using unsafe_store!(ptr, value, [index]). Currently, this is only supported for primitive types or other pointer-free (isbits) immutable struct types.

Any operation that throws an error is probably currently unimplemented and should be posted as a bug so that it can be resolved.

If the pointer of interest is a plain-data array (primitive type or immutable struct), the function unsafe_wrap(Array, ptr,dims, own = false) may be more useful. The final parameter should be true if Julia should "take ownership" of the underlying buffer and call free(ptr) when the returned Array object is finalized. If the own parameter is omitted or false, the caller must ensure the buffer remains in existence until all access is complete.

Arithmetic on the Ptr type in Julia (e.g. using +) does not behave the same as C’s pointer arithmetic. Adding an integer to a Ptr in Julia always moves the pointer by some number of bytes, not elements. This way, the address values obtained from pointer arithmetic do not depend on the element types of pointers.

Some C libraries execute their callbacks from a different thread, and since Julia isn’t thread-safe you’ll need to take some extra precautions. In particular, you’ll need to set up a two-layered system: the C callback should only schedule (via Julia’s event loop) the execution of your "real" callback. To do this, create an AsyncCondition object and wait on it:

The callback you pass to C should only execute a ccall to :uv_async_send, passing cond.handle as the argument, taking care to avoid any allocations or other interactions with the Julia runtime.

Note that events may be coalesced, so multiple calls to uv_async_send may result in a single wakeup notification to the condition.

For more details on how to pass callbacks to C libraries, see this blog post.

For tools to create C++ bindings, see the CxxWrap package.