Engee documentation

I/O functions

Input/Output functions in Julia provide the ability to work with files, data streams and other data sources/receivers (see I/O and Network). For files imported into the file browser, functions can be applied:

Discovery

open - opening a file or data stream.

Details

The open function provides convenient and safe work with files, because after usage of a file in the context of open call, the file is automatically closed, which helps to avoid resource leaks or other problems associated with implicitly leaving open files.

The function applies the f function to the result open(args...; kwargs...) and closes the resulting file descriptor upon completion.

open(f::Function, args...; kwargs...)

A file descriptor is an abstract concept used to represent an open file in a programme. It is a data structure that contains information about a file, such as its path, access mode (read, write, etc.), current position in the file, and other properties. Examples:

write("my_file.txt", "Hello world!"); # a file is being created my_file.txt containing the text Hello world!

open(io->read(io, String), "my_file.txt")
"Hello world!" # opens my_file.txt for reading and immediately reads its contents into a string

rm("my_file.txt") # deletes my_file.txt after its contents have been read and output

The io->read(io, String) function that takes an I/O object (io) and reads its contents into a string. The contents of the file "my_file.txt" are then output as the result, and the final result is the string "Hello world!".

Opens the file in the mode specified by the five logical named arguments:

open(filename::AbstractString; lock = true, keywords...) -> IOStream

Keyword

Description

By default

read
write
create
truncate
`append

open for reading
write
create if none exists
trim to zero
search to the end

!write
truncate | append
!read & write | truncate | append
!read & write
`false

By default, if no keywords are passed, files will be opened read-only. A thread is returned to access the open file. The lock named argument specifies whether operations will be locked for secure multithreaded access.

An alternative open syntax that uses a mode descriptor based on string operations instead of five logical arguments. The mode values correspond to those from fopen(3) or Perl open and are are equivalent to setting the following logical groups:

Mode

Description

Keywords

r
w
r+
w+
a+

reading
reading, creating, trimming
reading, creating, joining
reading, writing
reading, writing, creating, trimming
reading, writing, creating, joining

not write = true
append = true
read = true, write = true
`truncate = true, read = true
append = true, read = tru

* Examples:

io = open("my_file.txt", "w"); # opens my_file.txt for writing (write)

write(io, "Hello world!"); # writes the string Hello world! to the file that was opened for writing in the previous line

close(io); # closes my_file.txt , freeing up resources and completing the write operation

io = open("my_file.txt", "r"); # opens my_file.txt for reading (read), returns an input/output object (io)

read(io, String) # reads the contents of the my_file file.txt as a string
"Hello world!" # the result of the operation

write(io, "This file is read only") # since the file was opened read-only (on the previous line), this write operation is not allowed and will generate an ArgumentError error.
ERROR: ArgumentError: write failed, IOStream is not writeable
[...]

close(io) # closes the file my_file.txt after a write attempt to free up resources

io = open("myfile.txt", "a"); # opens my_file.txt to add (append), which will allow writing data to the end of the file without deleting or overwriting its contents

write(io, "This stream is not read only") # writes the line "This stream is not read only" to the end of the file my_file.txt because the file was opened in append mode ("a"). The write function returns the number of bytes written (28)
28 # the result of the operation

close(io)# closes my_file.txt after performing all write operations to free up resources

rm("my_file.txt") # deletes the file my_file.txt

Read more about all the features of the open function in I/O and Network.

isopen - determines whether the object (thread or timer) is already closed.

Details

Once an object is closed, it can no longer create new events. However, given that a closed thread may still have data to read in the buffer, you should use eof to check if the data can be read. Use the FileWatching package to be notified when writing to or reading from a stream is supported. The function outputs true if the object is open and false if the object is closed.

io = open("my_file.txt", "w+"); # opens the file my_file.txt in "w+" mode (write and read)

isopen(io) # checks whether the file associated with the io I/O object is open (my_file.txt )
true # the result of the operation

close(io) # closes the file associated with the io I/O object. After closing, the file is no longer available for writing or reading.

isopen(io) # after the file has been closed (the previous operation), the command returns false because the file is no longer open.
false # the result of the operation

For the function io = open("my_file.txt", "w+"):

  • If the file does not exist, it will be created.

  • If the file already exists, its contents will be deleted.

The open function also returns an I/O object (io) that is associated with the open file.

Closing

close - closes the I/O stream.

Details

The flush (forced I/O reset) is executed first.

close(io)

closeall - closes all open windows of the graphical user interface.

Details

Useful if you want to make sure that all files are closed before terminating the programme.

closeall()

closeread - closes only the reading thread from the file associated with the I/O object.

Details

If the file has been opened for reading and writing, the write stream remains open. Used to close only the read stream, with the option to continue writing to the file.

closeread(io)

closewrite - stops writing half of a full-duplex I/O stream.

Details

The flush operation is executed first. Notifies the other party that data is no longer being written to the base file. This operation may not be supported for some I/O types.

* Examples:

io = Base.BufferStream(); # a new buffered input/output stream is created. In this case, the lock is never executed, so read and write operations are available for the same job.

write(io, "request"); # the request string is written to the buffered io stream

# when calling `read(io)` here, a permanent lock is applied.

closewrite(io); # The io write stream is closed and no write operations will be accepted, while the read stream remains open.

read(io, String) # a read operation is being performed from io

Recording

write - writes a canonical binary file representation of a value to the specified I/O stream or file.

Details

Returns the number of bytes written to the stream. The order of bytes in the written value depends on the order of bytes in the host system.

It is possible to write multiple values with a single write call, i.e. the following values are equivalent:

write(io, x, y...)
write(io, x) + write(io, y...)

Examples:

# Consistent serialization

fname = tempname(); # arbitrary temporary file name

open(fname,"w") do f
             # Make sure that a 64-bit integer value is being written in the direct byte order format.
             write(f,htol(Int64(42)))
         end
8

julia> open(fname,"r") do f
             # Perform the reverse conversion to the basic byte order and the basic integer type
             Int(ltoh(read(f,Int64)))
         end
42

# Combining calls of write operations:

io = IOBuffer();

write(io, "JuliaLang is a GitHub organization.", " It has many members.")
56

String(take!(io))
"JuliaLang is a GitHub organization. It has many members."

write(io, "Sometimes those members") + write(io, " write documentation.")
44

String(take!(io))
"Sometimes those members write documentation."

# User-defined core data types without write methods can be written, and, note, encapsulated in Ref

struct MyStruct; x::Float64; end

io = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)

write(io, Ref(MyStruct(42.0)))
8

seekstart(io); read!(io, Ref(MyStruct(NaN)))
Base.RefValue{MyStruct}(MyStruct(42.0))

iswritable - checks if writing to the specified object is supported.

Details

Returns false if writing to the specified I/O object is not supported.

*Examples:

open("my_file.txt", "w") do io
    print(io, "Hello world!");
    iswritable(io)
end
true

open("my_file.txt", "r") do io
    iswritable(io)
end
false

rm("my_file.txt")

Reading

read - performs reading of the file contents.

Details

By default read reads the file in binary mode:

read("sample1.json")

# conclusion
65-element Vector{UInt8}:
 0x7b
 0x0a
 0x20
 0x20
 0x20
    ⋮

To avoid representation as an array of bytes, it is recommended to use a representation argument. For example, the read function can take several arguments, including the data type into which the file content should be converted (we use String):

read("sample1.json", String) # reads the contents of the sample1 file.json and returns it as a string

# conclusion
"{\n    \"fruit\": \"Apple\",\n    \"size\": \"Large\",\n    \"color\": \"Red\"\n}"

You can read the first character from a file and return it in Char format (character type) using the Char argument:

read("sample1.json", Char)

# conclusion
'{': ASCII/Unicode U+007B (category Ps: Punctuation, open)

This function read(file_name::IOStream, number_of_bytes::Integer) performs a read of at most number_of_bytes bytes from the stream file_name and returns an array Vector{ UInt8 } containing the bytes read. To get a stream from a file, you must open it for reading. For example:

open("sample1.json", "r") # opening the file for reading

# output of the reading result
IOStream(<file sample1.json="">)

read("sample1.json", 10) # reading the first 10 bytes from the file

# conclusion
10-element Vector{UInt8}:
 0x7b
 0x0a
 0x20
 0x20
 0x20
 0x20
 0x22
 0x66
 0x72
 0x75
*A stream is a data abstraction that represents a sequence of bytes. A sequence can be read or written from or to sources. For example, text files, network connections, memory, etc. Streams can also be used to read data from a source (input) or write data to it (output). Use:

  • flush - will commit all records currently in the buffer to this stream.

  • close(stream) - closes the I/O stream. Executes flush in priority.

  • closewrite(steam) - disables half-write of a full-duplex stream. Executes flush in priority. No more data will be written to the base file. This is not supported by all I/O types.

You can execute cmd commands using the read function. For example, let’s create the date command (outputs the current date and time):

# creating a command that outputs the current date and time
cmd = `date`

read(cmd) # outputs the result as an array of bytes

read(cmd, String) # outputs the result as a string
# conclusion
"Thu Mar 28 07:44:58 UTC 2024\n"

read! - performs reading of binary data from an I/O stream or file, filling the data matrix.

Details

The read! function is used to read data from an I/O stream or file into a preallocated buffer or array. It is commonly used to efficiently fill an array with data from an external source without creating additional temporary arrays. To get a stream from a file, you must open it for reading. For example:

open("sample1.json", "r") # opening the file for reading

my_array = zeros(UInt8, 5) # an array of 5 UInt8 type elements filled with zeros is created
# conclusion
5-element Vector{UInt8}:
 0x00
 0x00
 0x00
 0x00
 0x00

read!("sample1.json", my_array) # reads data from sample1.json and fills the my_array array with them
# conclusion
5-element Vector{UInt8}:
 0x7b
 0x0a
 0x20
 0x20
 0x20

readavailable - performs reading of available buffered data from the stream.

Details

If there is data in the stream that has not yet been read, the function will return that data. Actual I/O is only performed if the data has not yet been buffered. If there is no data in the buffer, the function will return an empty byte array.

The amount of data returned will be implementation dependent, for example, it may depend on the internal choice of buffer size. Other functions such as read should generally be used instead. 
IOS=open("sample1.json", "r") # opens the file in read mode and writes it to the contents of the IOS I/O buffer

readavailable(IOS) # reads available data from the I/O buffer and returns it

readbytes! - performs a read of at most nb bytes from stream to b, returning the number of bytes read.

Details

The readbytes! function reads at most n bytes from stream and writes them to the data array. If n is not specified, it reads the number of bytes equal to the length of the array. It then returns the number of bytes read. If fewer bytes are available in the stream than requested, only the available number of bytes will be read.

IOS=open("sample1.json", "r") # opens the file in read mode and writes it to the contents of the IOS I/O buffer

buffer = Vector{UInt8}(undef, 5) # an empty buffer array of type UInt8 with a length of 5 elements is created

readbytes!(IOS, buffer) # reads 5 bytes from the IOS stream and saves them to the buffer array

# conclusion
5 # as a result, a number is output that indicates the number of bytes successfully read. Since the array consisted of 5 elements, it was fully successfully read.

readchomp - performs reading of the complete file as a line and removes one line break character at the end (if any).

Details

The readchomp function is equivalent to chomp(read(x, String)).

write("my_file.txt", "Welcome to Engee.\n");

readchomp("my_file.txt")

# conclusion
"Welcome to Engee."

readdir - returns names in a directory (dir) or the current working directory if it is not specified.

Details

The readdir function is used to read the contents of a directory. It returns an array of strings, each of which represents the name of a file or subdirectory in the specified directory.

By default, readdir sorts the list of names returned. To skip sorting the names and get them in the order in which they are listed on the file system, you can use readdir(dir, sort=false).

cd("/user") # going to the user start directory

readdir() # returns the names of all files in the current directory (currently "/user")
7-element Vector{String}:
 ".git"
 ".packages"
 ".project"
 "11.ngscript"
 "engee_commands.csv"
 "my_file.txt"
 "sample1.json"

readdir(join=true) # combining the returned name with the current directory
7-element Vector{String}:
 "/user/.git"
 "/user/.packages"
 "/user/.project"
 "/user/11.ngscript"
 "/user/engee_commands.csv"
 "/user/my_file.txt"
 "/user/sample1.json"

readdir("my_dir") # returns the names of all files in the my_dir directory
7-element Vector{String}:
 "RTFM-HISTORY.txt"
 "delay.html"
 "difference.pdf"
 "difference.xml"
 "littleendian.svg"
 "markdown_test.md"
 "mat.mat"

readdir("my_dir", join=true) # combining the returned name with the selected directory (my_dir)
7-element Vector{String}:
 "my_dir/RTFM-HISTORY.txt"
 "my_dir/delay.html"
 "my_dir/difference.pdf"
 "my_dir/difference.xml"
 "my_dir/littleendian.svg"
 "my_dir/markdown_test.md"
 "my_dir/mat.mat"

readeach - returns an iterated object that gives read(io, T) as a result.

Details

The readeach function is used to iterate over the contents of a file or stream, performing the specified operation for each line or piece of data. It takes as input a stream to read and an argument to be applied to each line or piece of data.

io = IOBuffer("Welcome to Engee.\n Text to be deleted by the function.\n") # an input/output buffer is created

for c in readeach(io, Char) # iterating over characters from the stream until it reaches \n, then completes the iteration.
             c == '\n' && break
             print(c)
         end

# conclusion
Welcome to Engee.

readline - performs reading of a single line of text from a given I/O stream or file.

Details

When reading from a file, it is assumed that UTF-8 encoding is used for the text. Lines at the input end with '\n' or "\r\n" or at the end of the input stream. The keep argument may also be used. If keep is set to false (by default), these end characters of the newline are removed from the string before it is returned. If keep has the value true, they are returned as part of the string.

write("my_file.txt", "Welcome to Engee.\Ntext to be deleted by the function.\n");

readline("my_file.txt")
"Welcome to Engee."

readline("my_file.txt", keep=true)
"Welcome to Engee.\n"

rm("my_file.txt") # deletes the file my_file.txt

print("Enter your name: ")

your_name = readline() # opens the user name input window.
# enter a name, for example Engee_User
"Engee_User"

readlines - performs reading of all lines in an I/O stream or file as vector or string values.

Details

The behaviour is similar to repeatedly saving the result of reading readline with the same arguments and saving the resulting lines as vector or string values.

write("my_file.txt", "Welcome to Engee.\Ntext that will NOT be deleted by the function.\n");

readlines("my_file.txt")
# conclusion
2-element Vector{String}:
 "Welcome to Engee."
 "Text that will NOT be deleted by the function."

readlines("my_file.txt", keep=true)
2-element Vector{String}:
 "Welcome to Engee.\n"
 "Text that will NOT be deleted by the function.\n"

rm("my_file.txt") # deletes the file my_file.txt

readuntil - performs reading of a string from an I/O stream or a file (up to the specified delimiter character).

Details

The delimiter character can be a UInt8, AbstractChar, string or vector. The named argument keep determines whether the delimiter character is included in the result. It is assumed that UTF-8 encoding is used for text.

write("my_file.txt", "Welcome to Engee.\Ntext to be deleted by the function.\n")

readuntil("my_file.txt", 'L')

readuntil("my_file.txt", '.', keep = true)
  "Welcome to Engee."

rm("my_file.txt")

Read more about other file management features in Engee at Software control of modelling.

  1. Data import

  2. Working with Julia packages

  3. File formats