Engee documentation
Notebook

Universal Media Player

In modern data analytics and processing, it is often necessary to visualize and play media files directly in the development environment. The presented code implements a universal media player capable of working with audio, video, and images. This solution is especially useful for researchers, data analysts, and engineers working at Engee, providing them with a convenient tool for interactively viewing and listening to media files without the need for external applications.

The main technologies used in the example

  1. Base64 encoding is used to embed media files directly into HTML via a Data URL, which allows them to be played without the need for a server or file system.
  2. HTML5 and JavaScript provide an interactive player interface with control buttons and the ability to switch between files.
  3. Julia Packages:
    • Base64for encoding binary data;
    • FileIO for working with the file system;
    • WAV for processing WAV files (reading and changing the sampling rate).
In [ ]:
# Pkg.add("Base64")
# Pkg.add("FileIO")
# Pkg.add("WAV")

Realization

This function determines the type of files, encodes them in Base64, and generates HTML with JavaScript to control playback.

Input parameters and supported formats

1. The main parameters of the function:
  • path::AbstractString – the path to the file or folder with media files.

    • If a folder is specified, the player will download all supported files of the selected type (mode).
  • If a file is specified, only it will be processed.

  • mode::String (optional, default "audio") – media type:

    • "audio" – audio player (WAV, MP3)
    • "video" – video player (MP4)
    • "image" – view images (JPG, PNG, GIF, WEBP)
  • fs::Union{Nothing, Int} (optional, only for mode="audio") – sampling rate (Hz).

  • If nothing – the original file frequency is used.

    • If a number is specified (for example, 24000) – WAV files are resampled, MP3 files remain unchanged.

2. Supported file formats:
Mode (mode) Extensions Processing
"audio" .wav, .mp3 WAV: oversampling (if specified fs), normalization. MP3: encoded as is.
"video" .mp4 Without processing, direct Base64 encoding.
"image" .jpg, .jpeg, .png, .gif, .webp Is encoded unchanged. The main image formats are supported.
In [ ]:
using Base64
using FileIO
using WAV

"""
Parameters:
- path: the path to a file or folder with media files
- mode: тип контента ("audio", "video" или "image")
- FS: Sampling rate (audio only, optional)

Returns:
- HTML/JS widget for display in Engee
"""
function media_player(path::AbstractString; mode::String="audio", fs::Union{Nothing, Int}=nothing)
    # Dictionary of supported formats and their corresponding encoding functions
    ext_map = Dict(
        # Audio: WAV (with processing) and MP3 (as is)
        "audio" => ([".wav", ".mp3"], (f -> base64encode_audio(f, fs))),
        # Video: MP4 only
        "video" => ([".mp4"], base64encode_video),
        # Images: basic formats
        "image" => ([".jpg", ".jpeg", ".png", ".gif", ".webp"], base64encode_image)
    )
    
    # Checking the correctness of the operating mode
    haskey(ext_map, mode) || error("Unsupported mode: $mode")
    # We get a list of extensions and an encoder function for the selected mode.
    supported_extensions, encoder = ext_map[mode]
    # We get a list of files (one file or all files from a directory)
    files = isdir(path) ? readdir(path, join=true) : [path]
    # Filtering files by supported extensions
    selected_files = filter(f -> any(endswith(lowercase(f), ext) for ext in supported_extensions), files)
    isempty(selected_files) && error("Files in the $mode format were not found in the specified path.")
    # Preparing metadata for display
    filenames = [basename(f) for f in selected_files]
    # We encode all files in base64 and get their MIME types.
    encoded_data = [encoder(f) for f in selected_files]
    base64_encoded = [data[1] for data in encoded_data]
    mime_types = [data[2] for data in encoded_data]
    # Generating a unique ID for HTML elements (to support multiple players)
    unique_id = string(rand(UInt))
    # Customize styles depending on the type of content
    bg_color = "#f5f5f5"  # uniform background color for all media types
    text_color = "#555"   # uniform text color for all media types
    # Defining the HTML tag depends on the type of media.
    media_tag_name = mode == "audio" ? "audio" : mode == "video" ? "video" : "img"
    # Adding controls for audio/video
    controls_attr = mode in ["audio", "video"] ? "controls autoplay" : ""
    # Generating HTML for the media element
    media_html = if mode == "image"
        # The img tag for images
        """<img id="$(media_tag_name)_$(unique_id)" src="data:$(mime_types[1]);base64,$(base64_encoded[1])" 
           style="max-width: 100%; border-radius: 10px;" />"""
    else
        # Audio/video tags with source inside
        """
        <$media_tag_name id="$(media_tag_name)_$(unique_id)" $controls_attr style="width: 100%; border-radius: 10px;">
            <source id="src_$(unique_id)" src="data:$(mime_types[1]);base64,$(base64_encoded[1])" type="$(mime_types[1])" />
        </$media_tag_name>
        """
    end
    # Generating the full HTML code of the player with controls
    html_interface = """
    <div style="background: $bg_color; border-radius: 15px; padding: 15px; max-width: 640px; margin: 0 auto;">
        <!-- Панель управления с кнопками и именем файла -->
        <div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px; font-size: 14px; color: $text_color;">
            <!-- Кнопка "back" -->
            <button id="prevBtn_$(unique_id)" style="background: none; border: none; cursor: pointer; font-size: 22px;">⏮️</button>
            <!-- Имя текущего файла -->
            <span id="name_$(unique_id)" style="font-weight: bold;">$(filenames[1]) (1 of $(length(filenames)))</span>
            <!-- Кнопка "forward" -->
            <button id="nextBtn_$(unique_id)" style="background: none; border: none; cursor: pointer; font-size: 22px;">⏭️</button>
            <!-- Дополнительная кнопка цикла для аудио -->
            $(mode == "audio" ? """<button id="loopToggle_$(unique_id)" style="background: none; border: none; cursor: pointer; font-size: 20px;" title="Toggle Loop">🔂</button>""" : "")
        </div>
        <!-- Основной медиа-элемент -->
        $media_html
    </div>
    
    <!-- JavaScript для управления плеером -->
    <script>
        // Current file index
        let currentIndex_$(unique_id) = 0;
        // File data in base64
        const mediaFiles_$(unique_id) = [$(join(["\"$(e)\"" for e in base64_encoded], ","))];
        // MIME file types
        const mimeTypes_$(unique_id) = [$(join(["\"$(m)\"" for m in mime_types], ","))];
        // File Names
        const fileNames_$(unique_id) = [$(join(["\"$(n)\"" for n in filenames], ","))];
        // The cyclic playback flag
        let loopEnabled_$(unique_id) = false;
        // Getting DOM elements
        const mediaElement_$(unique_id) = document.getElementById("$(media_tag_name)_$(unique_id)");
        const sourceElement_$(unique_id) = document.getElementById("src_$(unique_id)");
        const nameElement_$(unique_id) = document.getElementById("name_$(unique_id)");
        /* *
         * The function of updating the current media file
         * @param newIndex - index of the new file
         */
        function updateMedia_$(unique_id)(newIndex) {
            // Adjusting the index when going out of bounds
            if (newIndex < 0) newIndex = mediaFiles_$(unique_id).length - 1;
            if (newIndex >= mediaFiles_$(unique_id).length) newIndex = 0;
            currentIndex_$(unique_id) = newIndex;
            const currentMime = mimeTypes_$(unique_id)[newIndex];
            const mediaData = "data:" + currentMime + ";base64," + mediaFiles_$(unique_id)[newIndex];
            // Updated depending on the type of media
            if ("$mode" === "image") {
                // For images, just change the src
                mediaElement_$(unique_id).src = mediaData;
            } else {
                // For audio/video, update the source and start playback
                sourceElement_$(unique_id).src = mediaData;
                sourceElement_$(unique_id).type = currentMime;
                mediaElement_$(unique_id).load();
                mediaElement_$(unique_id).play();
            }
            // Updating the displayed file name
            nameElement_$(unique_id).innerText = fileNames_$(unique_id)[newIndex] + 
                " (" + (newIndex + 1) + " of " + mediaFiles_$(unique_id).length + ")";
        }
        // Assigning button handlers
        document.getElementById("prevBtn_$(unique_id)").onclick = function() {
            updateMedia_$(unique_id)(currentIndex_$(unique_id) - 1);
        };
        document.getElementById("nextBtn_$(unique_id)").onclick = function() {
            updateMedia_$(unique_id)(currentIndex_$(unique_id) + 1);
        };
        // Additional functions for audio
        $(mode == "audio" ? """
        const loopButton_$(unique_id) = document.getElementById("loopToggle_$(unique_id)");
        // Loop button handler
        loopButton_$(unique_id).onclick = function() {
            loopEnabled_$(unique_id) = !loopEnabled_$(unique_id);
            loopButton_$(unique_id).innerHTML = loopEnabled_$(unique_id) ? "🔁" : "🔂";
            loopButton_$(unique_id).style.color = loopEnabled_$(unique_id) ? "dodgerblue" : "";
        };
        // Handler for the end of playback
        mediaElement_$(unique_id).addEventListener("ended", function() {
            if (loopEnabled_$(unique_id)) {
                updateMedia_$(unique_id)(currentIndex_$(unique_id) + 1);
            }
        });
        """ : "")
    </script>
    """
    # Displaying the generated HTML
    display("text/html", html_interface)
end

# Encodes an audio file in base64 with possible oversampling (for WAV).
function base64encode_audio(filepath, fs)
    ext = lowercase(splitext(filepath)[2])
    if ext == ".wav"
        # Reading a WAV file
        audio_data, original_fs = wavread(filepath)
        # Data normalization (mono/stereo)
        audio_data = ndims(audio_data) == 1 ? reshape(audio_data, :, 1) : audio_data
        audio_data = clamp.(audio_data, -1, 1)  # Limiting the amplitude
        # Conversion to 16-bit integer format
        audio_int = round.(Int16, audio_data .* typemax(Int16))
        # Write to the buffer with a new sampling rate (if specified)
        buffer = IOBuffer()
        wavwrite(audio_int, buffer; Fs=fs === nothing ? original_fs : fs, nbits=16)
        seekstart(buffer)
        # Encoding in base64
        return base64encode(take!(buffer)), "audio/wav"
    elseif ext == ".mp3"
        # MP3 is encoded as it is
        return base64encode(read(filepath)), "audio/mpeg"
    else
        error("Unsupported audio format: $ext")
    end
end

function base64encode_video(filepath) # Encodes a video file (MP4) in base64.
    lowercase(splitext(filepath)[2]) == ".mp4" || error("Only MP4 is supported")
    return base64encode(read(filepath)), "video/mp4"
end

function base64encode_image(filepath) # Encodes the image in base64.
    ext = lowercase(splitext(filepath)[2])
    # Defining the MIME type by extension
    mime_type = if ext == ".jpg" || ext == ".jpeg"
        "image/jpeg"
    elseif ext == ".png"
        "image/png"
    elseif ext == ".gif"
        "image/gif"
    elseif ext == ".webp"
        "image/webp"
    else
        error("Unsupported image format: $ext")
    end
    return base64encode(read(filepath)), mime_type
end
Out[0]:
base64encode_image (generic function with 1 method)

Let's move on to testing the developed algorithm and start with the processing features.

  • For WAV:
  • automatic amplitude normalization (limited to [-1, 1]),
  • Supports mono and stereo,
    • if fs set, the sampling rate is changing (WAV only, MP3 ignores this parameter).
  • For MP3/MP4/images:
  • data is encoded in Base64 without modification.
  • The interface contains:
  • toggle buttons (⏮️, ⏭️),
  • for audio – cycle button (🔂/🔁),
  • auto-start when switching (for audio/video).
In [ ]:
media_player("$(@__DIR__)/test_files/sample_3s.wav", mode="audio")
sample_3s.wav (1 of 1)
In [ ]:
media_player("test_files", mode="audio", fs = 24000)
sample_3s.wav (1 of 2)
In [ ]:
media_player("test_files", mode="image")
Intro Engee.gif (1 of 2)
No description has been provided for this image
In [ ]:
media_player("test_files", mode="video")
Engee neon.mp4 (1 of 1)

Conclusion

The presented example demonstrates the capabilities of Engee in creating interactive tools for working with multimedia.

This media player will be a convenient solution for:

  1. Quick verification and analysis of media files during the development process,
  2. Demonstration of audio and video processing results,
  3. Creating interactive reports with media elements.

The ability to programmatically process audio (for example, changing the sampling rate) before playback is especially valuable. This makes the tool useful for digital signal processing and machine learning tasks.

This media player is a great example of how you can go beyond traditional computing tasks without leaving the engineering environment.