Distribution of binary files

This information is intended for those who want to compile Julia binaries for distribution on various platforms. We welcome the distribution of Julia by users, as it allows you to experience the environment in a wide variety of operating systems and on all possible hardware configurations. Since each platform has its own nuances that must be taken into account to create a portable Julia distribution, we have shared most of the information on the OS.

Please note: although the Julia code https://github.com/JuliaLang/julia/blob/master/LICENSE.md [with a few exceptions, it is distributed under the MIT license], the distribution created by the methods described here will be distributed under the GPL license, since this license applies to a number of dependent libraries, such as `SuiteSparse'. We hope that in the future it will be possible to get the Julia distribution without the GPL license.

Version Control and Git

The makefile uses the 'VERSION` file, as well as hashes and commit tags from the Git repository to create a base/version_git.jl file with information that is displayed on the home screen and in the output of the versioninfo() function. If for any reason the Git repository is not available during the build, the file base/version_git.jl must be created in advance using the following command:

make -C base version_git.jl.phony

Many Julia build dependencies are used in patch versions that are not yet included in popular package managers. These dependencies are usually loaded automatically during build, but if you want to build Julia on a computer without Internet access, you should create a full-source-dist archive with a special purpose to make:

make full-source-dist

As a result, an archive will be created. julia-version-commit.tar.gz with all the necessary dependencies.

When compiling a tagged release in the Git repository, information about the branch hash and commit is not displayed on the home screen. You can use this line to display a release description up to 45 characters long. To set this line, create a Make.user file containing the following instructions:

override TAGGED_RELEASE_BANNER = "my-package-repository build"

Target architectures

By default, Julia optimizes the system image for the machine architecture of the build computer. This is usually not what is required when building packages, as as a result Julia will not be able to run on computers with incompatible CPUs (especially on older ones with limited instruction sets).

Therefore, when calling make, we recommend passing the MARCH variable, specifying as its value the target underlying architecture to be supported. It will determine the target CPUs for both the Julia executable and libraries, as well as for the system image (which can also be specified using JULIA_CPU_TARGET'). For x86 CPUs, typical values are `x86-64 and core2 (for 64-bit builds) and pentium4 (for 32-bit builds). Unfortunately, CPUs older than the Pentium 4 are currently not supported (see https://github.com/JuliaLang/julia/issues/7185 [this problem]).

The full list of target CPUs supported by LLVM can be obtained with the command `llc -mattr=help'.

Linux

On Linux, the make binary-dist command creates a Tarball archive containing a fully functional Julia installation. Additional efforts will be required to create a distribution package, for example, in the .deb or .rpm format. For an example of the metadata required to create .deb packages for Debian and Ubuntu-based systems, see the repository https://github.com/staticfloat/julia-debian [julia-debian]. For RPM-based distributions, see https://src.fedoraproject.org/rpms/julia [the Fedora package]. To create Julia packages for various Linux distributions, you can use the program https://wiki.debian.org/Alien [Alien], although we haven’t tried it out yet.

Julia supports redefining standard installation directories using prefix and other environment variables that can be passed when calling make and make install'. For a list, see Make.inc. You can also use `DESTDIR to force installation into a temporary directory.

By default, Julia downloads $prefix/etc/julia/startup.jl as the initialization file for the entire installation. Distribution managers can use it to set custom paths or initialization code. If the $prefix variable is set to /usr for the Linux distribution, the /usr/etc directory is missing. Therefore, it is necessary to change the path to Julia’s private directory etc. This can be done using the make sysconfdir variable during assembly. Just pass sysconfdir=/etc to make' during build, and Julia will first check the file `/etc/julia/startup.jl and only then — `$prefix/etc/julia/startup.jl'.

OS X

To create a binary distribution in OS X, first build Julia, and then go to the contrib/mac/app directory and run make with the same variables that were used with the make command when building Julia. As a result, a .dmg file with a completely standalone Julia.app application will be created in the contrib/mac/app directory.

In addition, Julia can be built as a framework by calling make to build darwinframework and the specified variable `DARWIN_FRAMEWORK=1'. For example, `make DARWIN_FRAMEWORK=1 darwinframework'.

Windows

Instructions for creating the Julia distribution in Windows are given in https://github.com/JuliaLang/julia/blob/master/doc/src/devdocs/build/windows.md [build documentation for Windows].

Notes on BLAS and LAPACK

By default, Julia builds OpenBLAS, including the BLAS and LAPACK libraries. On 32-bit architectures, Julia builds OpenBLAS using 32-bit integers, and on 64-bit architectures, it builds using 64-bit integers (ILP64). It is important that all Julia functions that call the BLAS and LAPACK API routines use integers of the correct width.

Most of the BLAS and LAPACK distributions included in Linux distributions, and even commercial implementations, provide libraries using 32-bit APIs. Often, the 64-bit API is provided as a separate library.

When using libraries provided by the vendor or the OS, the make parameter, called USE_BLAS64, is available as part of the Julia build. When executing the make USE_BLAS64=0 command, Julia calls BLAS and LAPACK, assuming that a 32-bit API is used, where all integers are 32 bits wide, even on 64-bit architectures.

Other libraries used by Julia, such as SuiteSparse, also use BLAS and LAPACK. The APIs must be consistent for all libraries that depend on BLAS and LAPACK. When building Julia, all these libraries are assembled correctly, but when redefining default values and using system libraries, this consistency must be ensured.

Keep in mind that Linux distributions sometimes include multiple versions of OpenBLAS, some of which support multithreading, while others only work sequentially. For example, the Fedora version libopenblasp.so ` is multithreaded, and `libopenblas.so ` — no. For optimal performance, we recommend using the first option. To select an OpenBLAS library other than the default library (`libopenblas.so `), pass the variables `LIBBLAS=-l$(YOURBLAS) and LIBBLASNAME=lib$(YOURBLAS) to make, replacing $(YOURBLAS)`in the name of your library. You can also add `.so.0 to the library name so that the package does not require a symbolic link .so without a version.

Finally, OpenBLAS includes its own optimized version of LAPACK. If you set the variables USE_SYSTEM_BLAS=1 and USE_SYSTEM_LAPACK=1, you must also set LIBLAPACK=-l$(YOURBLAS) and LIBLAPACKNAME=lib$(YOURBLAS). Otherwise, the reference version of LAPACK will be used, and performance will usually be much lower.

Starting with Julia 1.7, a different BLAS library is used at runtime to select it. https://github.com/JuliaLinearAlgebra/libblastrampoline [libblastrampoline].

The basics of releasing modified versions

The creation of a revised or corrective issue is carried out in several separate steps.

Fixing backports

Some pull requests are marked "backport pending x.y" (x.y backport is expected), for example "backport pending 0.6". This means that the next release from the release-x branch.y, must include commits in the pull request. After merging the pull request with the master branch, the changes from each commit should https://git-scm.com/docs/git-cherry-pick [copied] to a dedicated branch that will eventually be merged with release-x.y.

Creating a branch of backports

First, create a branch based on release-x.y. According to the Julia convention, you should add your initials before the branch name if it is to be personal. For example, let’s assume that the author of the branch is user Jane Smith.

git fetch origin
git checkout release-x.y
git rebase origin/release-x.y
git checkout -b js/backport-x.y

This ensures that before creating a new branch based on your local copy of release-x.y this copy is current and corresponds to the original branch.

Copying changes to commits

Now let’s do the backporting itself. In the GitHub web interface, find all pull requests for which a merge has been performed and which are marked as "backport pending x.y". For each of them, scroll down to the label "someperson merged commit 123abc into master XX minutes ago" (someone merged commit 123abc with master XX minutes ago). Note that the commit name is a link: if you click on it, the contents of the commit will be shown. If the page shows that 123abc is a merge commit, return to the pull request page.: we don’t need merge commits, but actual commits. However, if it is not shown that this is a merge commit, then the pull request merge was performed with compression. In this case, use the commit git SHA value specified next to the commit on this page.

After receiving the SHA value of the commit, copy its changes to the backport branch.:

git cherry-pick -x -e <sha>

There may be conflicts that need to be resolved manually. After resolving the conflicts (if any), add a link to the pull request in GitHub, in which the commit was added, in the text of the commit message.

After adding all the necessary commits to the backport branch, send this branch to GitHub.

Checking for performance degradation

Improved releases should not lead to lower performance. Fortunately, Julia’s performance testing bot called Nanosoldier allows you to run tests for any branch, not just master. In this case, we need to check the js/backport-x test results.y compared to release-x.y. To do this, activate the Nanosoldier bot by commenting on the pull request for backporting:

@nanosoldier `runbenchmarks(ALL, vs=":release-x.y")`

As a result, for release-x.y and js/backport-x.y All registered performance tests will be performed, with a summary of the results showing all improvements and impairments.

If Nanosoldier detects performance degradation, try checking locally, and then restart Nanosoldier if necessary. If it looks like there really are impairments, you will need to find the commit they are associated with in the master branch and backport it. If there is no such commit, you should determine what caused the deterioration and send the fix to master (or ask someone who understands the code to do this), and then backport the commit after merging. (If applicable, you can send the fix to the backport branch directly.)

Assembling test binaries

After merging the pull request for backporting with the release-x' branch.y update the local Julia clone, and then get the SHA of the branch using the following command:

git rev-parse origin/release-x.y

Keep this value handy, as it will need to be entered in the Revision field in the user interface of the build bot.

In the meantime, all you need are the binaries for Linux x86-64, as they are used to run PackageEvaluator. Go to the website https://buildog.julialang.org , submit the task for nuke_linux64', and then queue the task for `package_linux64', specifying SHA as the revision. When the packaging task is completed, the binary file will be sent to the `julialang2 container in AWS. Get the URL, as it will be needed for PackageEvaluator.

Checking for package failures

Improved releases should not disrupt the operation of packages. The only possible exception may be packages that perform extremely questionable actions using the internal mechanisms of the Base module, which are not intended for users. (In this case, perhaps you should discuss the situation with the author of the package.)

You can use the tool to check whether changes in the upcoming version will cause the packages to malfunction. https://github.com/JuliaCI/PackageEvaluator.jl [PackageEvaluator], which is often abbreviated as PkgEval. It is PkgEval that assigns status logos in the GitHub repositories and on the website. pkg.julialang.org . This tool usually runs on one of the Nanosoldier nodes that are not designed for performance testing, and uses Vagrant to perform its tasks on separate VirtualBox virtual machines running in parallel.

Configuring PackageEvaluator

Clone PackageEvaluator, create the backport-x.y.z branch and extract it. Keep in mind that the required changes are quite complex and confusing. Hopefully, the situation will change in a future version of PackageEvaluator. The changes will be reproduced according to the sample. https://github.com/JuliaCI/PackageEvaluator.jl/commit/5ba6a3b000e7a3793391d16f695c8704b91d6016 [this fixation].

The configuration script takes the Julia version as the first argument, and the package name range as the second (AK for packages with names from A to K, LZ for packages with names from L to Z). Our task is to configure the script so that only two versions of Julia are running: the current release of x.y and the backport version, each with three package ranges.

In comparison, the link indicates that if the second argument has the value LZ, binary files compiled from the backport branch should be used. Otherwise, (AK) binary files of the release branch should be used. Then we use the first argument to run part of the package list: A-F for the input value of 0.4, G-N for 0.5, and O-Z for 0.6.

Launching PackageEvaluator

To run PkgEval, select a sufficiently powerful machine (for example, a Nanosoldier 1 node), and then run the following commands:

git clone https://github.com/JuliaCI/PackageEvaluator.jl.git
cd PackageEvaluator.jl/scripts
git checkout backport-x.y.z
./runvagrant.sh

Several folders will be created in the scripts/ directory. The names of these folders and their contents are listed below.

Folder Name	Julia’s version	Package Range
0.4AK	Release	A-F
0.4LZ	Backport	A-F
0.5AK	Release	G-N
0.5LZ	Backport	G-N
0.6AK	Release	O-Z
0.6LZ	Backport	O-Z

Folder Name

Julia’s version

Package Range

0.4AK

Release

A-F

0.4LZ

Backport

A-F

0.5AK

Release

G-N

0.5LZ

Backport

G-N

0.6AK

Release

O-Z

0.6LZ

Backport

O-Z

Examining the results

After that, using `./summary.sh ` you can get a summary report on the results from the same catalog. We will do this for each of the folders to get cumulative results by version.

./summary.sh 0.4AK/*.json > summary_release.txt
./summary.sh 0.5AK/*.json >> summary_release.txt
./summary.sh 0.6AK/*.json >> summary_release.txt
./summary.sh 0.4LZ/*.json > summary_backport.txt
./summary.sh 0.5LZ/*.json >> summary_backport.txt
./summary.sh 0.6LZ/*.json >> summary_backport.txt

Now we have two files, `summary_release.txt `and `summary_backport.txt `, with the PackageEvaluator test results (passed or failed) for each package in two versions.

To make them easier to transfer to Julia, we will convert them to CSV files, and then use the DataFrames package to process the results. To convert to CSV format, copy each TXT file to the corresponding CSV file, and then log in to Vim and run ggVGI"<esc> followed by :%s/\.json /",/g. (You don’t have to use Vim, it’s just one of the ways.) Now process the results using Julia code like the following.

using DataFrames

release = readtable("summary_release.csv", header=false, names=[:package, :release])
backport = readtable("summary_backport.csv", header=false, names=[:package, :backport])

results = join(release, backport, on=:package, kind=:outer)

for result in eachrow(results)
    a = result[:release]
    b = result[:backport]
    if (isna(a) && !isna(b)) || (isna(b) && !isna(a))
        color = :yellow
    elseif a != b && occursin("pass", b)
        color = :green
    elseif a != b
        color = :red
    else
        continue
    end
    printstyled(result[:package], ": Release ", a, " -> Backport ", b, "\n", color=color)
end

As a result, color-coded strings will be displayed in stdout. All red lines should be examined, as they indicate potential violations caused by the backport version. You should also pay attention to the yellow lines.: they mean that a package is executed in one version, but for some reason is not executed in another. If you find that there are violations due to the backported branch, identify the problematic commits using the git bisect command, run the git revert command for these commits, and repeat the procedure.

Merging backports with a release branch

If all of the following conditions are met:

backported commits pass all Julia unit tests;
due to backported commits, performance does not decrease compared to the release branch.;
backported commits do not disrupt the operation of registered packages.;

the backport branch is ready to merge with release-x.y. After merging, go through all pull requests with backported commits and delete the label "backport pending x.y". Do not remove this label from pull requests that have not been backported.

The release-x.y branch should now contain all new commits. The last thing to do with the branch is to adjust the version number. To do this, send a pull request to release-x.y, which modifies the VERSION file so that the -pre element is removed from the version number. After merging it, you can proceed to adding the tag.

Adding a tag to a release

The time has come! Extract the release-x branch.y and make sure that its local copy is synchronized with the remote one. At the command prompt, run the following commands:

git tag v$(cat VERSION)
git push --tags

As a result, the tag will be created locally and sent to GitHub.

After adding the tag to the release, send it to release-X.y another pull request to update the fix number and add the -pre element at the end again. This means that the branch status corresponds to the preliminary version of the next revised release in the x.y series.

Follow the further instructions in the Makefile.

Signing binary files

Some of the actions described below will require secure passwords. To get them, contact Elliot Saba (staticfloat) or Alex Arslan (ararslan). Please note that code signing for each platform must be performed on that platform (for example, signing for Windows must be performed on Windows, etc.).

Linux

On Linux, the code must be signed manually, but it is done quite easily. First, get the julia.key file from the CodeSigning folder in the AWS juliasecure container. Add it to your GnuPG key set using the following command:

gpg --import julia.key

You will need to enter a password, which should be obtained from Elliot or Alex. Next, set the maximum trust level for the key. First` log into the 'gpg` session:

gpg --edit-key julia

At the command prompt, type trust, and then, when the trust level is requested, specify the maximum possible value (most likely 5). Log out of GnuPG.

Now, for each Tarball Linux archive created by the build bots, enter the following command:

gpg -u julia --armor --detach-sig julia-x.y.z-linux-<arch>.tar.gz

As a result, a corresponding ASC file will be created for each Tarball archive. That’s all!

macOS

macOS build bots should sign the code automatically. However, you need to check if everything was successful. Download the DMG file created by the build bots on your macOS system or virtual machine. For example, let’s assume that the DMG file is named `julia-x.y.z-osx.dmg'. Run the following commands:

mkdir ./jlmnt
hdiutil mount -readonly -mountpoint ./jlmnt julia-x.y.z-osx.dmg
codesign -v jlmnt/Julia-x.y.app

At the connection stage, remember the name of the connected disk! Let’s say it’s a disk3' disk. If the code signing verification was completed successfully, there will be no output data at the `codesign step. If so, the DMG file can be disabled.:

hdiutil eject /dev/disk3
rm -rf ./jlmnt

If you receive a message like the following:

Julia-x.y.app: code object is not signed at all

the code must be signed manually.

To sign the code manually, first get the OS X certificates from the CodeSigning folder in the AWS juliasecure container. Add the P12 file to your keychain using Keychain.app. For the password to the key, contact Elliot Saba (staticfloat) or Alex Arslan (ararslan). Now run the following commands.

hdiutil convert julia-x.y.z-osx.dmg -format UDRW -o julia-x.y.z-osx_writable.dmg
mkdir ./jlmnt
hdiutil mount -mountpoint julia-x.y.z-osx_writable.dmg
codesign -s "AFB379C0B4CBD9DB9A762797FC2AB5460A2B0DBE" --deep jlmnt/Julia-x.y.app

The following error message may appear:

Julia-x.y.app: resource fork, Finder information, or similar detritus not allowed

In this case, you need to remove the extra attributes.:

xattr -cr jlmnt/Julia-x.y.app

After that, try to sign the code again. If there are no errors, repeat the check. If everything is OK now, disable the writable DMG file and make it read-only again.:

hdiutil eject /dev/disk3
rm -rf ./jlmnt
hdiutil convert julia-x.y.z-osx_writable.dmg -format UDZO -o julia-x.y.z-osx_fixed.dmg

Check if the resulting DMG file has been fixed by double-clicking on it. If there are no problems, extract it and remove the _fixed suffix from the name. That’s all!

Windows

In Windows, the code is signed manually. First, get the SDK for Windows 10 with the necessary signing tools from the Microsoft website. We will need the SignTool program, which should be installed along a path like 'C:\Program Files (x86)\Windows Kits\10\App Certification Kit`. Get the Windows certificate files from CodeSigning in 'juliasecure` and place them in the executable directory. Open the CMD window in Windows, navigate (cd) to the directory with all the files and run the following commands:

set PATH=%PATH%;C:\Program Files (x86)\Windows Kits\10\App Certification Kit;
signtool sign /f julia-windows-code-sign_2017.p12 /p "PASSWORD" ^
   /t http://timestamp.verisign.com/scripts/timstamp.dll ^
   /v julia-x.y.z-win32.exe

Note that ^ is a line continuation character in Windows CMD, and PASSWORD is a placeholder for the certificate password. As usual, contact Elliot or Alex for passwords. If there are no errors, then everything is fine!

Sending binary files

Now that the code is signed, you need to send the binary files to AWS. You can use a program like Cyberduck or the aws command line program. The binary files must be placed in the appropriate folders in the container `julialang2'. For example, for x86-64 Linux, this is `julialang2/bin/linux/x.y'. Don’t forget to delete the current `julia-x.y-latest-linux-<arch>' file.tar.gz ` and replace it with a copy of `julia-x.y.z-linux-<arch>.tar.gz `.

In addition, you need to send checksums for all assembled components, including Tarball source code archives and release binaries. It’s not difficult to do this.

shasum -a 256 julia-x.y.z* | grep -v -e sha256 -e md5 -e asc > julia-x.y.z.sha256
md5sum julia-x.y.z* | grep -v -e sha256 -e md5 -e asc > julia-x.y.z.md5

Keep in mind that if these commands are executed on macOS, the output will be slightly different. You can reformat them based on an existing file. Additionally, Mac users will need to use 'md5 -r` instead of md5sum. Send the MD5 and SHA256 files to julialang2/bin/checksums in AWS.

For all files sent to AWS, the "All: READ" permission must be set.

For each file sent, it is necessary to clear the Fastly cache so that the links on the website point to the files sent. Example:

curl -X PURGE https://julialang-s3.julialang.org/bin/checksums/julia-x.y.z.sha256

Sometimes you can do without it, but it won’t be superfluous.