Values with limited scope

The page is in the process of being translated.

The values with limited scope represent the implementation of dynamic scopes in Julia.

!!! note "Lexical scoping vs dynamic scoping" Lexical scoping is the default behavior in Julia. In the lexical definition of areas, the area of a variable is determined by the lexical (textual) the structure of the program. When dynamically defining scopes, the variable is bound to the last assigned value during program execution.

The state of a value with a limited scope depends on the execution path of the program. This means that there can be several different values for a single value at the same time.

Compatibility: Julia 1.11

Limited scope values appeared in Julia 1.11. In Julia version 1.8 and later, a compatible implementation is available in the ScopedValues.jl package.

In its simplest form 'ScopedValue` can be created with a default value, and then used with or @with to add a new dynamic area. The new scope inherits all values from the parent scope (and recursively from all external scopes), with the provided limited-scope value taking precedence over the previous definitions.

First, let’s look at an example of a lexical domain. The 'let` operator starts a new lexical domain in which the external definition of x is obscured by its internal definition.

x = 1
let x = 5
    @show x # 5
end
@show x # 1

In the following example, since Julia uses a lexical scope, the variable x in the body of the function f refers to the variable x defined in the global scope, and adding the scope let does not change the value observed by the function f.

x = 1
f() = @show x
let x = 5
    f() # 1
end
f() # 1

Now, using `ScopedValue', you can use a dynamic scope.

using Base.ScopedValues

x = ScopedValue(1)
f() = @show x[]
with(x=>5) do
    f() # 5
end
f() # 1

Note that the observed value of ScopedValue depends on the program execution path.

It often makes sense to use the const variable to indicate a value with a limited scope, and you can set the value of several scopedvalues by calling with in one go.

using Base.ScopedValues

f() = @show a[]
g() = @show b[]

const a = ScopedValue(1)
const b = ScopedValue(2)

f() # a[] = 1
g() # b[] = 2

# Добавляем новую динамическую область и задаем значение.
with(a => 3) do
    f() # a[] = 3
    g() # b[] = 2
    with(a => 4, b => 5) do
        f() # a[] = 4
        g() # b[] = 5
    end
    f() # a[] = 3
    g() # b[] = 2
end

f() # a[] = 1
g() # b[] = 2

'ScopedValues` provides a macro version for with'. The expression `+@with var⇒val expr+ calculates expr in a new dynamic area with the variable var, which is set to val. @with var=>val expr is equivalent to with(var=>val) do expr end. However, with requires a closure or a function with a null argument, which results in an additional call frame. For example, consider the following function f:

using Base.ScopedValues
const a = ScopedValue(1)
f(x) = a[] + x

To run the function f in the dynamic range with a with the value 2, use with:

with(() -> f(10), a=>2)

However, this requires enclosing the function f in a function with a null argument. To avoid the appearance of an additional call frame, you can use the macro @with:

@with a=>2 f(10)

Dynamic areas are inherited by tasks Task at the time of the task creation. Dynamic regions are not distributed through Distributed.jl operations.

In the example below, we open a new dynamic area before starting a task. The parent task and two child tasks simultaneously observe independent values of the same value with a limited scope.

using Base.ScopedValues
import Base.Threads: @spawn

const scoped_val = ScopedValue(1)
@sync begin
    with(scoped_val => 2)
        @spawn @show scoped_val[] # 2
    end
    with(scoped_val => 3)
        @spawn @show scoped_val[] # 3
    end
    @show scoped_val[] # 1
end

Values with a limited area are constant throughout the entire area, but a mutable state can be stored in such a value. Remember that in the context of parallel programming, there are the usual caveats regarding global variables.

Care must also be taken when storing references to mutable state in values with limited scope. Perhaps, when adding a new dynamic area, you will want to explicitly cancel shared access to the changeable state.

using Base.ScopedValues
import Base.Threads: @spawn

const sval_dict = ScopedValue(Dict())

# Пример неправильного использования изменяемого значения
@sync begin
    # `Dict`не является потокобезопасным, приведенное ниже использование является недопустимым
    @spawn (sval_dict[][:a] = 3)
    @spawn (sval_dict[][:b] = 3)
end

@sync begin
    # Если вместо этого мы передадим каждой задаче уникальный словарь,
    # то сможем обращаться к словарям без гонок.
    with(sval_dict => Dict()) do
        @spawn (sval_dict[][:a] = 3)
    end
    with(sval_dict => Dict()) do
        @spawn (sval_dict[][:b] = 3)
    end
end

Example

In the example below, a limited scope value is used to implement access permission verification in a web application. After defining the query permissions, a new dynamic scope is added and a value with a limited scope of `LEVEL' is set. Other parts of the application can request a value with a limited scope and receive the corresponding value. Other alternatives, such as task-local storage and global variables, are not well suited for this kind of distribution. Our only alternative would be to run the value through the entire call chain.

using Base.ScopedValues

const LEVEL = ScopedValue(:GUEST)

function serve(request, response)
    level = isAdmin(request) ? :ADMIN : :GUEST
    with(LEVEL => level) do
        Threads.@spawn handle(request, response)
    end
end

function open(connection::Database)
    level = LEVEL[]
    if level !== :ADMIN
        error("Access disallowed")
    end
    # ... открыть подключение
end

function handle(request, response)
    # ...
    open(Database(#=...=#))
    # ...
end

Idioms

Revoking shared access to a changeable state

using Base.ScopedValues
import Base.Threads: @spawn

const sval_dict = ScopedValue(Dict())

# Если вы хотите добавить новые значения в словарь, а не заменять
# их, явным образом отмените общий доступ к значениям. В этом примере мы используем `merge`
# для отмены общего доступа к состоянию словаря в родительской области.
@sync begin
    with(sval_dict => merge(sval_dict[], Dict(:a => 10))) do
        @spawn @show sval_dict[][:a]
    end
    @spawn sval_dict[][:a] = 3 # Гонка отсутствует, поскольку общий доступ отменен.
end

Values with limited scope as global variables

To access the meaning of a limited-scope value, the limited-scope value itself must be in the (lexical) scope. This means that most often you will use values with limited scope as permanent global variables.

using Base.ScopedValues
const sval = ScopedValue(1)

In fact, values with limited scope can be considered as hidden arguments to functions.

This does not exclude their use as non-global variables.

using Base.ScopedValues
import Base.Threads: @spawn

function main()
    role = ScopedValue(:client)

    function launch()
        #...
        role[]
    end

    @with role => :server @spawn launch()
    launch()
end

But perhaps in such cases it would be easier to just directly pass the argument to the function.

Too many ScopedValue values

If you create a lot ScopedValue for a single module, it might be better to use a special structure to store them.

using Base.ScopedValues

Base.@kwdef struct Configuration
    color::Bool = false
    verbose::Bool = false
end

const CONFIG = ScopedValue(Configuration(color=true))

@with CONFIG => Configuration(color=CONFIG[].color, verbose=true) begin
    @show CONFIG[].color # true
    @show CONFIG[].verbose # true
end

API Documentation

# Base.ScopedValues.ScopedValue — Type

ScopedValue(x)

Create a container that distributes values across dynamic areas. Use with to create a dynamic area and enter it.

The values can only be set when entering a new dynamic area, and they will be constant throughout the lifetime of this area.

Dynamic areas apply to all tasks.

Examples

julia> using Base.ScopedValues;

julia> const sval = ScopedValue(1);

julia> sval[]
1

julia> with(sval => 2) do
           sval[]
       end
2

julia> sval[]
1

Compatibility: Julia 1.11

Limited scope values appeared in Julia 1.11. In Julia version 1.8 and later, a compatible implementation is available in the ScopedValues.jl package.

# Base.ScopedValues.with — Function

with(f, (var::ScopedValue{T} => val)...)

Executes f in a new dynamic area with the variable var, which has been assigned the value val'. `val is converted to type `T'.

See also the description ScopedValues.@with, ScopedValues.ScopedValue, ScopedValues.get.

Examples

julia> using Base.ScopedValues

julia> a = ScopedValue(1);

julia> f(x) = a[] + x;

julia> f(10)
11

julia> with(a=>2) do
           f(10)
       end
12

julia> f(10)
11

julia> b = ScopedValue(2);

julia> g(x) = a[] + b[] + x;

julia> with(a=>10, b=>20) do
           g(30)
       end
60

julia> with(() -> a[] * b[], a=>3, b=>4)
12

# Base.ScopedValues.@with — Macro

@with (var::ScopedValue{T} => val)... expr

The with version as a macro. The expression @with var=>val expr calculates expr in a new dynamic area with the variable var, which is set to val. val is converted to type T'. `+@with var⇒val expr+ is equivalent to with(var=>val) do expr end, but @with avoids creating a closure.

See also the description ScopedValues.with, ScopedValues.ScopedValue, ScopedValues.get.

Examples

julia> using Base.ScopedValues

julia> const a = ScopedValue(1);

julia> f(x) = a[] + x;

julia> @with a=>2 f(10)
12

julia> @with a=>3 begin
           x = 100
           f(x)
       end
103

# Base.isassigned — Method

isassigned(val::ScopedValue)

Checks whether `ScopedValue' has been assigned a value.

See also the description ScopedValues.with, ScopedValues.@with, ScopedValues.get.

Examples

julia> using Base.ScopedValues

julia> a = ScopedValue(1); b = ScopedValue{Int}();

julia> isassigned(a)
true

julia> isassigned(b)
false

# Base.ScopedValues.get — Function

get(val::ScopedValue{T})::Union{Nothing, Some{T}}

If a value with a limited area is not specified and has no default value, returns nothing'. Otherwise, it returns `+Some{T}+ with the current value.

See also the description ScopedValues.with, ScopedValues.@with, ScopedValues.ScopedValue.

Examples

julia> using Base.ScopedValues

julia> a = ScopedValue(42); b = ScopedValue{Int}();

julia> ScopedValues.get(a)
Some(42)

julia> isnothing(ScopedValues.get(b))
true

Implementation notes and performance

Scope uses a permanent dictionary. The search and insertion are performed using O(log(32, n)). When entering a dynamic area, a small amount of data is copied, and the unchanged data is distributed among other areas.

The object itself Scope is not intended for users and may be changed in a future version of Julia.

Ideas for development

This development was heavily influenced by the document https://openjdk.org/jeps/429 [JEPS-429], which in turn relied on free variables in the dynamic domain in many Lisp dialects. In particular, in Interlisp-D and its deep linking strategy.

The design of context variables was discussed earlier https://peps.python.org/pep-0567 /[PEPS-567], implemented in Julia as https://github.com/tkf/ContextVariablesX .jl[ContextVariablesX].