Engee documentation

GDB Debugging Tips

Displaying Julia variables

In 'gdb`, any object (obj) jl_value_t* can be displayed as follows.

(gdb) call jl_(obj)

The object will be displayed in the julia session, not in the gdb session. This is a good way to determine the types and values of objects that are manipulated by Julia’s C code.

In addition, if you are debugging some internal Julia components (for example, compiler.jl), you can output an object (obj) using the function

ccall(:jl_, Cvoid, (Any,), obj)

This is the best option to work around problems that arise due to the initialization order of julia output streams.

The flisp interpreter in Julia uses value_t objects. They can be displayed using `call fl_print(fl_ctx, ios_stdout, obj)'.

Julia’s useful variables for checking

Although outputting the addresses of many variables, such as single instances, can be useful for many failures, there are a number of additional variables (for a full list, see the description of julia.h) that are even more relevant.

  • (when in jl_apply_generic') `mfunc and jl_uncompress_ast(mfunc->def, mfunc->code): to determine some information about the call stack

  • 'jl_lineno` and `jl_filename': to determine the line in the test from which debugging should begin (or to determine the extent to which the file has been analyzed)

  • '$1`: not exactly a variable, but still a useful shorthand for referring to the result of the last gdb command (for example, print)

  • 'jl_options': sometimes useful because it lists all command line parameters that have been successfully analyzed

  • 'jl_uv_stderr': used as an alternative to interacting with stdio

Julia’s useful functions for checking these variables

  • 'jl_print_task_backtraces(0)': similar to thread apply all bt in gdb or thread backtrace all in lldb.

  • 'jl_gdblookup($pc)`: to search for the current function and string.

  • 'jl_gdblookupinfo($pc)`: to search for the current method instance object.

  • 'jl_gdbdumpcode(mi): to reset the entire `code_typed/code_llvm/code_asm when the REPL is not working properly.

  • 'jlbacktrace()': to save the current Julia backtrace stack to stderr. It is used only after calling `record_backtrace()'.

  • 'jl_dump_llvm_value(Value*)': to call the function Value->dump() in gdb, where it does not work initially. For example, f->linfo->functionObject, f->linfo->specFunctionObject and to_function(f->linfo).

  • jl_dump_llvm_module(Module*): to call the function `Module->dump()' in gdb, where it does not work initially.

  • Type->dump(): works only in lldb. Note. Add something like ;1 to prevent lldb from displaying its window during output.

  • 'jl_eval_string("expr")`: to invoke side effects in order to change the current state or to search for characters.

  • 'jl_typeof(jl_value_t*)': to extract the Julia value type label (in gdb, first call macro define jl_typeof jl_typeof or select something short like ty as the first argument to define the abbreviation).

Inserting breakpoints for validation from gdb

In the gdb session, set the breakpoint in the jl_breakpoint function as follows.

(gdb) break jl_breakpoint

Then, in Julia’s code, insert a call to the jl_breakpoint function by adding:

ccall(:jl_breakpoint, Cvoid, (Any,), obj)

obj can be any variable or tuple that should be available at the breakpoint.

It is especially useful to return to the jl_apply frame, from which you can display arguments for a function, for example as follows.

(gdb) call jl_(args[0])

Another useful frame is to_function(jl_method_instance_t li, bool cstyle)'. The `jl_method_instance_t argument is a structure with a reference to the final AST tree sent to the compiler. However, the AST is usually compressed at this stage. To view the AST, call jl_uncompress_ast' and then pass the result to `jl_.

#2  0x00007ffff7928bf7 in to_function (li=0x2812060, cstyle=false) at codegen.cpp:584
584          abort();
(gdb) p jl_(jl_uncompress_ast(li, li->ast))

Inserting breakpoints under certain conditions

Uploading a specific file

Let’s say the file sysimg.jl is used.

(gdb) break jl_load if strcmp(fname, "sysimg.jl")==0

Calling a specific method

(gdb) break jl_apply_generic if strcmp((char*)(jl_symbol_name)(jl_gf_mtable(F)->name), "method_to_break")==0

Since this function is used for every call, things will slow down significantly if you do this.

Working with signals

Several signals are required for Julia to function correctly. The profiler uses SIGUSR2 for sampling, and the garbage collector uses SIGSEGV for thread synchronization. If you are debugging code that is used by a profiler or several threads, you can allow the debugger to ignore these signals, since they can be activated very often during normal operation. To do this, GDB uses the following command (replace SIGSEGV with SIGUSR2 or other signals that need to be ignored).

(gdb) handle SIGSEGV noprint nostop pass

The corresponding LLDB command (executed after the process is started) is the following.

(lldb) pro hand -p true -s false -n false SIGSEGV

If you are debugging an emergency with streaming code, you can set a breakpoint in jl_critical_error (sigdie_handler should also work on Linux and BSD) to intercept only the actual emergency, and not the synchronization points of the garbage collector.

Debugging during the Julia build (bootstrap)

Errors that occur during the build (make) require special handling. Julia is built in two stages with the construction of sys0 and `sys.ji'. To see which commands were running at the time of the crash, use `make VERBOSE=1'.

At the time of writing this document, you can debug build errors at the sys0 stage from the base directory as follows.

julia/base$ gdb --args ../usr/bin/julia-debug -C native --build ../usr/lib/julia/sys0 sysimg.jl

You may need to delete all the files in usr/lib/julia/ to make this work.

You can debug 'sys.ji` as follows.

julia/base$ gdb --args ../usr/bin/julia-debug -C native --build ../usr/lib/julia/sys -J ../usr/lib/julia/sys0.ji sysimg.jl

By default, any errors cause Julia to shut down, even in gdb. To catch an error in the process, set a breakpoint to jl_error (there are several more useful breakpoints for specific types of failures, including jl_too_few_args, jl_too_many_args and `jl_throw').

After catching the error, it is recommended to go up the stack and examine the function by checking the associated function call jl_apply. Let’s give a real example.

Breakpoint 1, jl_throw (e=0x7ffdf42de400) at task.c:802
802 {
(gdb) p jl_(e)
ErrorException("auto_unbox: unable to determine argument type")
$2 = void
(gdb) bt 10
#0  jl_throw (e=0x7ffdf42de400) at task.c:802
#1  0x00007ffff65412fe in jl_error (str=0x7ffde56be000 <_j_str267> "auto_unbox:
   unable to determine argument type")
   at builtins.c:39
#2  0x00007ffde56bd01a in julia_convert_16886 ()
#3  0x00007ffff6541154 in jl_apply (f=0x7ffdf367f630, args=0x7fffffffc2b0, nargs=2) at julia.h:1281
...

The last function jl_apply' is in frame 3, so we can go back there and look at the AST for the function `julia_convert_16886'. This is the unique name of some conversion method (`convert'). The `f in this frame is a function of jl_function_t*, so you can view the type signature, if any, from the `specTypes' field.

(gdb) f 3
#3  0x00007ffff6541154 in jl_apply (f=0x7ffdf367f630, args=0x7fffffffc2b0, nargs=2) at julia.h:1281
1281            return f->fptr((jl_value_t*)f, args, nargs);
(gdb) p f->linfo->specTypes
$4 = (jl_tupletype_t *) 0x7ffdf39b1030
(gdb) p jl_( f->linfo->specTypes )
Tuple{Type{Float32}, Float64}           # <-- сигнатура типа для julia_convert_16886

Then we can look at the AST for this function.

(gdb) p jl_( jl_uncompress_ast(f->linfo, f->linfo->ast) )
Expr(:lambda, Array{Any, 1}[:#s29, :x], Array{Any, 1}[Array{Any, 1}[], Array{Any, 1}[Array{Any, 1}[:#s29, :Any, 0], Array{Any, 1}[:x, :Any, 0]], Array{Any, 1}[], 0], Expr(:body,
Expr(:line, 90, :float.jl)::Any,
Expr(:return, Expr(:call, :box, :Float32, Expr(:call, :fptrunc, :Float32, :x)::Any)::Any)::Any)::Any)::Any

Finally, you can perform a forced recompilation of the function to go through the code generation process. To do this, clear the cached object functionObject from jl_lamdbda_info_t*.

(gdb) p f->linfo->functionObject
$8 = (void *) 0x1289d070
(gdb) set f->linfo->functionObject = NULL

Then set the breakpoint in the right place (for example, emit_function, emit_expr, emit_call, etc.) and start code generation.

(gdb) p jl_compile(f)
... # Здесь находится ваша точка останова

Debugging pre-compilation errors

Precompiling a module generates a separate Julia process for precompiling each module. To set a breakpoint or track failures in the pre-compilation work role, you need to connect a debugger. The simplest approach is to set up a debugger to monitor the launch of new processes matching a given name. For example:

(gdb) attach -w -n julia-debug

or:

(lldb) process attach -w -n julia-debug

Then run the script or command to run the pre-compilation. As described earlier, use conditional breakpoints in the parent process to intercept certain file upload events and narrow the debugging window. (Some operating systems may require alternative approaches, such as following each branch (fork) from the parent process.)

Mozilla Recording and Playback Platform (rr)

Julia is currently working directly with https://rr-project.org /[rr], Mozilla’s simplified recording and deterministic debugging platform. This allows you to deterministically reproduce the execution trace. The address spaces, register contents, system call data, etc. of the reproduced execution are exactly the same in each run.

The latest version of rr is required (3.1.0 or higher).

Reproducing concurrency errors using rr

By default, rr simulates a single-threaded computer. To debug parallel code, you can use rr record --chaos, which will make rr work in simulated situations where the number of cores randomly ranges from one to eight. Therefore, you can set JULIA_NUM_THREADS=8 and re-execute the code in rr until the error is caught.