LibAudio

Audio codec support and playback system for SerenityOS

LibAudio provides comprehensive audio file loading, decoding, and encoding capabilities for SerenityOS. It supports multiple audio formats and offers a unified interface for audio stream processing.

Overview

LibAudio handles audio data through a plugin-based architecture that supports various codecs and formats:

• Loader System: Format-agnostic audio file loading
• Codec Support: FLAC, MP3, WAV, and QOA formats
• Sample Processing: Multi-channel audio with various sample formats
• Metadata Extraction: ID3 tags and embedded pictures
• Server Connection: Integration with AudioServer for playback

Core Components

Loader

The main interface for loading audio files.

Loader · class

High-level audio file loader supporting multiple formats.

class Loader : public RefCounted<Loader> {
    static ErrorOr<NonnullRefPtr<Loader>, LoaderError> create(StringView path);
    static ErrorOr<NonnullRefPtr<Loader>, LoaderError> create(ReadonlyBytes buffer);
    
    LoaderSamples get_more_samples(size_t samples_to_read_from_input = 128 * KiB);
    MaybeLoaderError reset();
    MaybeLoaderError seek(int const position);
};

Factory Methods:

• create(StringView path): Load from file path
• create(ReadonlyBytes buffer): Load from memory buffer

Audio Properties

sample_rate · method

Get the sample rate in Hz.

u32 sample_rate() const;

Common values: 44100, 48000, 96000 Hz

num_channels · method

Get the number of audio channels.

u16 num_channels() const;

Typical values: 1 (mono), 2 (stereo), 6 (5.1 surround)

total_samples · method

Get total sample count (per channel).

int total_samples() const;

For a 3-second stereo file at 44.1kHz: returns 132,300

loaded_samples · method

Get number of samples loaded so far.

int loaded_samples() const;

Sample Formats

Supported PCM Formats

LibAudio supports various PCM sample formats:

enum class PcmSampleFormat {
    Uint8,
    Int16,
    Int24,
    Int32,
    Float32,
    Float64
};

u16 bits_per_sample() const;
PcmSampleFormat pcm_format() const;

The Sample type represents a normalized floating-point sample value.

Supported Formats

FLAC (Free Lossless Audio Codec)

FlacLoader · class

Lossless audio compression format loader.

Features:

• Lossless compression
• Seek table support
• Embedded metadata and pictures
• Streaming decoding

Location: LibAudio/FlacLoader.h

MP3 (MPEG Audio Layer III)

MP3Loader · class

Lossy audio compression format loader.

Features:

• Frame-based decoding
• Variable and constant bitrate support
• ID3v2 tag parsing
• Huffman table decoding

Location: LibAudio/MP3Loader.h

WAV (Waveform Audio)

Uncompressed PCM audio format with RIFF container.

QOA (Quite OK Audio)

Lightweight lossy audio compression format.

Metadata Support

Metadata · struct

Audio file metadata including tags and embedded data.

Metadata const& metadata() const;

Contains:

• Artist, album, title information
• Track numbers and dates
• Genre and comment fields
• Custom metadata tags

Picture Data

Embedded Album Art

LibAudio can extract embedded pictures from audio files:

struct PictureData {
    ID3PictureType type;        // FrontCover, BackCover, etc.
    String mime_string;          // "image/jpeg", "image/png"
    String description_string;
    
    u32 width;
    u32 height;
    u32 color_depth;
    u32 colors;
    
    Vector<u8> data;             // Image data
};

Vector<PictureData> const& pictures() const;

Picture Types:

• FrontCover: Album front cover
• BackCover: Album back cover
• LeadArtist: Lead artist photo
• Media: Media (e.g., CD) image
• And 17 other types defined in ID3v2 spec

Seeking and Navigation

Seek Points

SeekTable · class

Efficient seeking within audio streams.

struct SeekPoint {
    u64 sample_index;
    u64 byte_offset;
};

class SeekTable {
    Optional<SeekPoint const&> seek_point_before(u64 sample_index) const;
    ErrorOr<void> insert_seek_point(SeekPoint);
    size_t size() const;
};

Seek tables enable fast random access within compressed audio files.

Seeking Precision:

• Maximum seek point distance: 1000ms
• Seek tolerance: ±5000ms
• Loaders may provide frame-accurate seeking

Audio Server Integration

Connection Management

Server Connections

LibAudio provides connections to AudioServer and AudioManagerServer:

class ConnectionToServer {
    // Main audio playback connection
};

class ConnectionToManagerServer {
    // Audio device and configuration management
};

Location:

• ConnectionToServer.h: Audio playback
• ConnectionToManagerServer.h: Device management

Plugin Architecture

LoaderPlugin · class

Base class for format-specific audio decoders.

class LoaderPlugin {
    virtual ErrorOr<Vector<FixedArray<Sample>>, LoaderError> 
        load_chunks(size_t samples_to_read_from_input) = 0;
    
    virtual MaybeLoaderError reset() = 0;
    virtual MaybeLoaderError seek(int const sample_index) = 0;
    
    virtual int loaded_samples() = 0;
    virtual int total_samples() = 0;
    virtual u32 sample_rate() = 0;
    virtual u16 num_channels() = 0;
    
    virtual ByteString format_name() = 0;
    virtual PcmSampleFormat pcm_format() = 0;
};

Chunk-Based Loading

Audio is loaded in chunks for efficient streaming:

• Chunks are format-specific (FLAC frames, MP3 frames, etc.)
• Default buffer size: 8 KiB
• Loaders return multiple chunks to satisfy sample requests
• May overshoot requested sample count by small amounts

Error Handling

LoaderError · enum

Audio loading and decoding errors.

using LoaderSamples = ErrorOr<FixedArray<Sample>, LoaderError>;
using MaybeLoaderError = ErrorOr<void, LoaderError>;

Location: LibAudio/LoaderError.h

Generic Types

Sample Representation

LibAudio uses normalized floating-point samples:

// Defined in GenericTypes.h
struct Sample {
    float left;
    float right;
    // ... additional channels
};

Samples are normalized to the range [-1.0, 1.0] regardless of source format.

Encoding Support

FlacWriter · class

FLAC encoding capabilities.

class FlacWriter {
    // Encode audio to FLAC format
};

Location: LibAudio/FlacWriter.h

Usage Example

// Load audio file
auto loader = TRY(Audio::Loader::create("/path/to/audio.flac"));

// Get audio properties
dbgln("Sample rate: {} Hz", loader->sample_rate());
dbgln("Channels: {}", loader->num_channels());
dbgln("Duration: {} samples", loader->total_samples());
dbgln("Format: {}", loader->format_name());

// Read samples
while (loader->loaded_samples() < loader->total_samples()) {
    auto samples = TRY(loader->get_more_samples());
    // Process samples...
}

// Access metadata
auto const& metadata = loader->metadata();
dbgln("Title: {}", metadata.title);

// Extract album art
for (auto const& picture : loader->pictures()) {
    if (picture.type == Audio::ID3PictureType::FrontCover) {
        // Use picture.data
    }
}

Source Location

Directory: Userland/Libraries/LibAudio/

Key Files:

• Loader.h: Main loader interface
• GenericTypes.h: Sample and common types
• FlacLoader.h: FLAC decoder
• MP3Loader.h: MP3 decoder
• LoaderError.h: Error definitions