Interfaces

Sequential iteration is implemented by the iterate function. Instead of mutating objects as they are iterated over, Julia iterators may keep track of the iteration state externally from the object. The return value from iterate is always either a tuple of a value and a state, or nothing if no elements remain. The state object will be passed back to the iterate function on the next iteration and is generally considered an implementation detail private to the iterable object.

Any object that defines this function is iterable and can be used in the many functions that rely upon iteration. It can also be used directly in a for loop since the syntax:

is translated into:

A simple example is an iterable sequence of square numbers with a defined length:

With only iterate definition, the Squares type is already pretty powerful. We can iterate over all the elements:

We can use many of the builtin methods that work with iterables, like in or sum:

There are a few more methods we can extend to give Julia more information about this iterable collection. We know that the elements in a Squares sequence will always be Int. By extending the eltype method, we can give that information to Julia and help it make more specialized code in the more complicated methods. We also know the number of elements in our sequence, so we can extend length, too:

Now, when we ask Julia to collect all the elements into an array it can preallocate a Vector{Int} of the right size instead of naively push!ing each element into a Vector{Any}:

While we can rely upon generic implementations, we can also extend specific methods where we know there is a simpler algorithm. For example, there’s a formula to compute the sum of squares, so we can override the generic iterative version with a more performant solution:

This is a very common pattern throughout Julia Base: a small set of required methods define an informal interface that enable many fancier behaviors. In some cases, types will want to additionally specialize those extra behaviors when they know a more efficient algorithm can be used in their specific case.

It is also often useful to allow iteration over a collection in reverse order by iterating over Iterators.reverse(iterator). To actually support reverse-order iteration, however, an iterator type T needs to implement iterate for Iterators.Reverse{T}. (Given r::Iterators.Reverse{T}, the underling iterator of type T is r.itr.) In our Squares example, we would implement Iterators.Reverse{Squares} methods:

For the Squares iterable above, we can easily compute the ith element of the sequence by squaring it. We can expose this as an indexing expression S[i]. To opt into this behavior, Squares simply needs to define getindex:

Additionally, to support the syntax S[begin] and S[end], we must define firstindex and lastindex to specify the first and last valid indices, respectively:

For multi-dimensional begin/end indexing as in a[3, begin, 7], for example, you should define firstindex(a, dim) and lastindex(a, dim) (which default to calling first and last on axes(a, dim), respectively).

Note, though, that the above only defines getindex with one integer index. Indexing with anything other than an Int will throw a MethodError saying that there was no matching method. In order to support indexing with ranges or vectors of Ints, separate methods must be written:

While this is starting to support more of the indexing operations supported by some of the builtin types, there’s still quite a number of behaviors missing. This Squares sequence is starting to look more and more like a vector as we’ve added behaviors to it. Instead of defining all these behaviors ourselves, we can officially define it as a subtype of an AbstractArray.

If a type is defined as a subtype of AbstractArray, it inherits a very large set of rich behaviors including iteration and multidimensional indexing built on top of single-element access. See the arrays manual page and the Julia Base section for more supported methods.

A key part in defining an AbstractArray subtype is IndexStyle. Since indexing is such an important part of an array and often occurs in hot loops, it’s important to make both indexing and indexed assignment as efficient as possible. Array data structures are typically defined in one of two ways: either it most efficiently accesses its elements using just one index (linear indexing) or it intrinsically accesses the elements with indices specified for every dimension. These two modalities are identified by Julia as IndexLinear() and IndexCartesian(). Converting a linear index to multiple indexing subscripts is typically very expensive, so this provides a traits-based mechanism to enable efficient generic code for all array types.

This distinction determines which scalar indexing methods the type must define. IndexLinear() arrays are simple: just define getindex(A::ArrayType, i::Int). When the array is subsequently indexed with a multidimensional set of indices, the fallback getindex(A::AbstractArray, I...) efficiently converts the indices into one linear index and then calls the above method. IndexCartesian() arrays, on the other hand, require methods to be defined for each supported dimensionality with ndims(A) Int indices. For example, SparseMatrixCSC from the SparseArrays standard library module, only supports two dimensions, so it just defines getindex(A::SparseMatrixCSC, i::Int, j::Int). The same holds for setindex!.

Returning to the sequence of squares from above, we could instead define it as a subtype of an AbstractArray{Int, 1}:

Note that it’s very important to specify the two parameters of the AbstractArray; the first defines the eltype, and the second defines the ndims. That supertype and those three methods are all it takes for SquaresVector to be an iterable, indexable, and completely functional array:

As a more complicated example, let’s define our own toy N-dimensional sparse-like array type built on top of Dict:

Notice that this is an IndexCartesian array, so we must manually define getindex and setindex! at the dimensionality of the array. Unlike the SquaresVector, we are able to define setindex!, and so we can mutate the array:

The result of indexing an AbstractArray can itself be an array (for instance when indexing by an AbstractRange). The AbstractArray fallback methods use similar to allocate an Array of the appropriate size and element type, which is filled in using the basic indexing method described above. However, when implementing an array wrapper you often want the result to be wrapped as well:

In this example it is accomplished by defining Base.similar(A::SparseArray, ::Type{T}, dims::Dims) where T to create the appropriate wrapped array. (Note that while similar supports 1- and 2-argument forms, in most case you only need to specialize the 3-argument form.) For this to work it’s important that SparseArray is mutable (supports setindex!). Defining similar, getindex and setindex! for SparseArray also makes it possible to copy the array:

In addition to all the iterable and indexable methods from above, these types can also interact with each other and use most of the methods defined in Julia Base for AbstractArrays:

If you are defining an array type that allows non-traditional indexing (indices that start at something other than 1), you should specialize axes. You should also specialize similar so that the dims argument (ordinarily a Dims size-tuple) can accept AbstractUnitRange objects, perhaps range-types Ind of your own design. For more information, see Arrays with custom indices.

A strided array is a subtype of AbstractArray whose entries are stored in memory with fixed strides. Provided the element type of the array is compatible with BLAS, a strided array can utilize BLAS and LAPACK routines for more efficient linear algebra routines. A typical example of a user-defined strided array is one that wraps a standard Array with additional structure.

Here are some examples to demonstrate which type of arrays are strided and which are not:

Broadcasting is triggered by an explicit call to broadcast or broadcast!, or implicitly by "dot" operations like A .+ b or f.(x, y). Any object that has axes and supports indexing can participate as an argument in broadcasting, and by default the result is stored in an Array. This basic framework is extensible in three major ways:

Not all types support axes and indexing, but many are convenient to allow in broadcast. The Base.broadcastable function is called on each argument to broadcast, allowing it to return something different that supports axes and indexing. By default, this is the identity function for all AbstractArrays and Numbers — they already support axes and indexing.

If a type is intended to act like a "0-dimensional scalar" (a single object) rather than as a container for broadcasting, then the following method should be defined:

that returns the argument wrapped in a 0-dimensional Ref container. For example, such a wrapper method is defined for types themselves, functions, special singletons like missing and nothing, and dates.

Custom array-like types can specialize Base.broadcastable to define their shape, but they should follow the convention that collect(Base.broadcastable(x)) == collect(x). A notable exception is AbstractString; strings are special-cased to behave as scalars for the purposes of broadcast even though they are iterable collections of their characters (see Strings for more).

The next two steps (selecting the output array and implementation) are dependent upon determining a single answer for a given set of arguments. Broadcast must take all the varied types of its arguments and collapse them down to just one output array and one implementation. Broadcast calls this single answer a "style". Every broadcastable object each has its own preferred style, and a promotion-like system is used to combine these styles into a single answer — the "destination style".

Sometimes, it is desirable to change how the end-user interacts with the fields of an object. Instead of granting direct access to type fields, an extra layer of abstraction between the user and the code can be provided by overloading object.field. Properties are what the user sees of the object, fields what the object actually is.

By default, properties and fields are the same. However, this behavior can be changed. For example, take this representation of a point in a plane in polar coordinates:

As described in the table above dot access p.r is the same as getproperty(p, :r) which is by default the same as getfield(p, :r):

However, we may want users to be unaware that Point stores the coordinates as r and ϕ (fields), and instead interact with x and y (properties). The methods in the first column can be defined to add new functionality:

It is important that getfield and setfield are used inside getproperty and setproperty! instead of the dot syntax, since the dot syntax would make the functions recursive which can lead to type inference issues. We can now try out the new functionality:

Finally, it is worth noting that adding instance properties like this is quite rarely done in Julia and should in general only be done if there is a good reason for doing so.