not-fl3 / quad-snd

Cross-platform audio for Rust

quad-snd

High-level, light-weight, and opinionated audio library.

• Web: WebAudio
• Android: OpenSLES
• Linux: Alsa
• macOS: CoreAudio
• Windows: Wasapi
• iOS: CoreAudio

Being high-level enough allows quad-snd to use very different approaches to each backend. For example, for WebAudio all the playback and mixing is based on Audio nodes, while in OpenSLES quad-snd itself is responsible for mixing.

quad-snd lacks lots of features and the best way to use the library - either fork a repo and fine-tune it for your needs or just copy-paste some code from certain audio backends.

Biggest difference from any other sound library in rust:
quad-snd is small. Each backend implementation is ~300LoC code and is self sufficient - you can copy-paste the whole thing and run it, (almost)no common code, dependencies or anything like that would be required.

The only dependency is audrey. audrey helps backends that do not have file parsing functionality (all the platforms but web) to get bytes out of encoded .wav/.ogg. When web is not required - getting rid of audrey and use anything else(or nothing at all) for audio decoding is a super easy fix.

Attribution

While building quad-snd I looked into the implementation of the following libraries:

https://github.com/floooh/sokol/blob/master/sokol_audio.h
https://github.com/norse-rs/audir
https://github.com/unrust/uni-snd

quad-snd's Issues

Support simple stereo panning

Here is a link supposed to serve as a reference:
http://www.cs.cmu.edu/~music/icm-online/readings/panlaws/

It would be much easier to implement such a thing than some complex binaural sound model based on HRTF.
For simple 2D games this should do the work nicely

Output device does not switch to headphones when user jack is plugged in

32bit support

Trying to use this in a Raspberry PI 3 with a stock raspbian 32bit kernel/os.

consts::PCM_BUFFER_SIZE is set as a u64

This is causing issues with calls to quad_alsa_sys::snd_pcm_hw_params_set_buffer_size because it's also platform dependent, and will be built to expect an u32 or u64.

error[E0308]: mismatched types
  --> src/alsa_snd.rs:43:70
   |
43 |     if sys::snd_pcm_hw_params_set_buffer_size(pcm_handle, hw_params, consts::PCM_BUFFER_SIZE) < 0 {
   |                                                                      ^^^^^^^^^^^^^^^^^^^^^^^ expected `u32`, found `u64`
   |
help: you can convert a `u64` to a `u32` and panic if the converted value doesn't fit
   |
43 |     if sys::snd_pcm_hw_params_set_buffer_size(pcm_handle, hw_params, consts::PCM_BUFFER_SIZE.try_into().unwrap()) < 0 {
   |                                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Also since quad_alsa_sys::snd_pcm_writei will return the frames written as a u32, this line fails:

error[E0308]: mismatched types
   --> src/alsa_snd.rs:128:29
    |
128 |         if frames_writen == -libc::EPIPE as i64 {
    |                             ^^^^^^^^^^^^^^^^^^^ expected `i32`, found `i64`
    |
help: you can convert `frames_writen` from `i32` to `i64`, matching the type of `-libc::EPIPE as i64`
    |
128 |         if i64::from(frames_writen) == -libc::EPIPE as i64 {
    |            ^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

This later I can certainly do the i64 up convert as suggested, but for the first issue, with the snd_pcm_hw_params_set_buffer_size what is the best way to handle this?

I am still pretty new to rust. I make it a usize, but quad_alsa_sys is making this a u32 or u64 at compile time probably from the underlying C library. Is this something to be fixed in that library (quad_alsa_sys) to change the function to use usize?

If I force the const to u32 of course everything works fine, but that's now hard coding it to only work on 32bit systems.

'Native frameworks are only available on macOS targets' when targeting wasm32 on macOS

Probably modify build.rs
if cfg!(target_os = "macos") && cfg!(target_arch = "wasm32")

Decouple sound and buffer management

Hi, this library is really neat! =D

Little request: currently it seems that the sound file is decoded every time a source is add (at least on WASM).
It would be nice to split these into two stages: creation of buffer and sound that uses it, so one could do decoding at load time, and not during the game.

"rust-lld: error: function signature mismatch: audio_init" wasm32-unknown-unknown error

I'm on MacOS Catalina 10.15.7, quad_snd version 0.1.0-alpha.1 (I'm using an old version because it's what @TanTanDev was using with his game using this, and the latest version doesn't work for me). Here's the full error:
note: rust-lld: error: function signature mismatch: audio_init
>>> defined as (i32) -> i32 in /Users/{GAME_PATH}/target/wasm32-unknown-unknown/debug/deps/libquad_snd-9a787bf96d4e8e9b.rlib(quad_snd-9a787bf96d4e8e9b.quad_snd.f517ac66-cgu.14.rcgu.o)
>>> defined as () -> void in /Users/{GAME_PATH}/target/wasm32-unknown-unknown/debug/deps/libquad_snd-b3ce2925c0a480b8.rlib(quad_snd-b3ce2925c0a480b8.quad_snd.371d2a4f-cgu.10.rcgu.o)

Support WAV sample rates different than 44100

When I try to play a file with sample rate 22050 library panics with an assert.
It's not exactly what I've been expecting

Pitch/Playback speed support

It would be great to be able to change playback speed/pitch of a sound dynamically so one can introduce variation into repeating sound without spending much memory.

Add CI?

travis/appveyour or Github Actions

Panic: Can't set rate

Just tried running some of the macroquad examples, but the audio thread paniced. I get the same error just trying the examples here.

$ cargo run --example simple
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/examples/simple`
thread '<unnamed>' panicked at 'Can't set rate.', src/alsa_snd.rs:57:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Audio thread died
Audio thread died

backtrace:

stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/6dba4ed215e7a60f0a2a19c04f3f73691f89c509/library/std/src/panicking.rs:616:12
   1: quad_snd::snd::setup_pcm_device
             at ./src/alsa_snd.rs:57:9
   2: quad_snd::snd::audio_thread
             at ./src/alsa_snd.rs:104:22
   3: quad_snd::snd::AudioContext::new::{{closure}}
             at ./src/alsa_snd.rs:160:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Loading music is slow, especially in debug builds

The music for our game is 9 minutes long.
In debug mode, it takes 30 seconds to load the .ogg file, and 10 seconds to load the .wav.

In release mode (and with the hack below) it loads in 3 seconds.
Not terrible, but perhaps a "streaming" solution would be better for music?

Adding this to Cargo.toml eliminates the lag in debug builds:

[profile.dev.package.quad-snd]
opt-level = 2

It seems that stereo sounds could be played in reversed channels order

Since we do not control if anything about channel order here:

value += sound.data.samples[sound.progress / divisor] * volume;

We could reach a situation when the sound with intended order of channels would split the order of channels (e.g. what you should here from a left ear is heard from a right ear)

To ensure such situations aren't possible, we should store an ear state for a sound and a mixer and keep them sync

this could look something like this:

if self.ear == sound.ear {
    let volume = sound.volume.0 * self.volume.0;
    let volume = volume * volume;
    value += sound.data.samples[sound.progress / divisor] * volume;
    sound.ear.switch();
    sound.progress += 1;
}

// and somewhere near the end of a next_value() method:
self.ear.switch();

setting volume doesn't work on Wasm

I just updated audio support in good-web-game and while testing it I noticed that though setting audio volume works fine on desktop it doesn't work on Wasm.

For a live example check gwg's sounds example: https://psteinhaus.github.io/sounds_example/

As the example shows neither starting the sound with a certain volume set in the params, nor changing it while playing changes anything on the actual volume.

Is this the same audio library that's built into Macroquad?

I'm making a game, and I'm trying to figure out sound effects. When I used Macroquad's sound library, there's a delay before playing, and some weird (subtle) distortion on my sound effects. I'm using .wav formats.

Multiple instances of the same `Sound` playing

This is something I want to implement, but I'm curious if the API for this sounds good enough to push upstream:

• Sound::play returns a new type Playback (happy to see suggestions for better name), which is a newtype wrapper around auto-incrementing u32. Calling it multiple times doesn't stop any previous playbacks of the same sound.
• Playback::stop and Playback::set_volume analogs to those on Sound that only affect that specific play instance (mapped to SoundState in the mixer).
• Sound::stop and Sound::set_volume are still there and affect all instances of the same sound id.

What this means is that SoundState would have one id for the sound it's sourcing from, and another id unique to its specific instance. The latter can be generated in the MixerController when calling play, same as it's done for adding new sounds, which means this can be a really cheap feature to implement (no extra synchronization, and no effect on Mixer runtime performance).

I believe I can also make the Wasm/JS side of things work with the same API surface.

Yay or nay?

Delay on MacOS

When I call audio::play_sound_once() there is a little bit of a delay. The wasm32-unknown-unknown build doesn't, but only the native build.

wasapi: Crash on alt-tab

thread '' panicked at 'assertion failed: wasapi_buffer.is_null() == false',

Bug from discord.

Can't reproduce it myself, so any help welcome!

Way to reproduce (doesn't really reproduce it on my machine :/)

main.rs:

use macroquad::{audio, prelude::*, ui};

#[macroquad::main("Audio")]
async fn main() {
    let sound1 = audio::load_sound("test_13000.wav ").await.unwrap();
    let sound2 = audio::load_sound("test.ogg ").await.unwrap();

    loop {
        clear_background(LIGHTGRAY);

        if ui::root_ui().button(None, "Play sound 1") {
            warn!("play 1!");
            audio::play_sound_once(sound1);
        }
        if ui::root_ui().button(None, "Play sound 2") {
            warn!("play 2!");
            audio::play_sound_once(sound2);
        }
        next_frame().await
    }
}

Cargo.toml:

[package]
name = "test"
version = "0.1.0"
edition = "2018"

[dependencies]
macroquad = "0.3" 

#[patch.crates-io]
#quad-snd = { git = "https://github.com/not-fl3/quad-snd.git"}
#quad-snd = { path = "../quad-snd"}

Sound files may be found here https://github.com/not-fl3/quad-snd/tree/master/examples

and alt-tab while sound is playing

44.1khz audio plays faster & higher pitched when user device is at 48khz

panicked at 'Can't set rate'

I'm not using pipewire, just alsa & pulseaudio

thread '<unnamed>' panicked at 'Can't set rate.', /home/sheep/.cargo/registry/s
rc/github.com-1ecc6299db9ec823/quad-snd-0.2.5/src/alsa_snd.rs:57:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Audio thread died

with RUST_BACKTRACE=1

thread '<unnamed>' panicked at 'Can't set rate.', /home/sheep/.cargo/registry/s
rc/github.com-1ecc6299db9ec823/quad-snd-0.2.5/src/alsa_snd.rs:57:9
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src
/panicking.rs:616:12
   1: quad_snd::snd::setup_pcm_device
             at /home/sheep/.cargo/registry/src/github.com-1ecc6299db9ec823/qua
d-snd-0.2.5/src/alsa_snd.rs:57:9
   2: quad_snd::snd::audio_thread
             at /home/sheep/.cargo/registry/src/github.com-1ecc6299db9ec823/qua
d-snd-0.2.5/src/alsa_snd.rs:104:22
   3: quad_snd::snd::AudioContext::new::{{closure}}
             at /home/sheep/.cargo/registry/src/github.com-1ecc6299db9ec823/qua
d-snd-0.2.5/src/alsa_snd.rs:160:13

Version bump

Chatted about this in discord, but I think we need to do a version bump to get sounds to play on (some versions of?) linux in macroquad. If I change the quad-snd dependency from version = "0.2" to git = "https://github.com/not-fl3/quad-snd" then the examples in macroquad work.

The last release was 11 months ago, and I see there's been a few changes.

Allow audio generation

Not every game neccessarilly wants to play prerecorded audio files, rather they might want to generate the audio in real time.

To do this the Mixer would have to be exposed publically or rather the AudioContext would have to have an option to specify if the sample should be loaded from a file or the buffer should be filled using a callback or a second thread.

I'm thinking of a sgstem similar to how rust-sdl2 handles it: example can be found in their docs

struct SquareWave {
    phase_inc: f32,
    phase: f32,
    volume: f32
}

impl AudioCallback for SquareWave {
    type Channel = f32;

    fn callback(&mut self, out: &mut [f32]) {
        // Generate a square wave
        for x in out.iter_mut() {
            *x = if self.phase <= 0.5 {
                self.volume
            } else {
                -self.volume
            };
            self.phase = (self.phase + self.phase_inc) % 1.0;
        }
    }
}

I'm not completely sure if and how this could be implemented for all the supported platforms, but looking at the alsa module for linux, the mixer already had a function to fill a given buffer

Panicked at 'Can't set rate.' when using jack/alsa with a sample rate other than 44100

I'm running on Linux using Jack for audio and using the default ALSA -> JACK plugin for the bridge (no Pulseaudio, no Pipewire) at a sample rate of 48000. This causes a thread '<unnamed>' panicked at 'Can't set rate.', src/alsa_snd.rs:57:9 error when I run the example code.

A workaround is changing my jack configuration to use a sample rate of 44100.

The full stacktrace is below:

$ cargo run --example simple
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/examples/simple`
thread '<unnamed>' panicked at 'Can't set rate.', src/alsa_snd.rs:57:9
stack backtrace:
   0:     0x55d265bf2140 - std::backtrace_rs::backtrace::libunwind::trace::h1d00f3fcf4cb5ac4
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55d265bf2140 - std::backtrace_rs::backtrace::trace_unsynchronized::h920a6ff332484ee2
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55d265bf2140 - std::sys_common::backtrace::_print_fmt::hd7323920c925af6d
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x55d265bf2140 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h3155a8c966b4beb5
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55d265c0cd2e - core::fmt::write::h062c617411b691df
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/fmt/mod.rs:1209:17
   5:     0x55d265bf0165 - std::io::Write::write_fmt::hb61fdf1275c61e1c
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/io/mod.rs:1682:15
   6:     0x55d265bf1f05 - std::sys_common::backtrace::_print::hd1b4d9664ab500e0
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:47:5
   7:     0x55d265bf1f05 - std::sys_common::backtrace::print::hca896ae22beb06cb
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:34:9
   8:     0x55d265bf378f - std::panicking::default_hook::{{closure}}::h0b5eeed5cf36ab5f
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:267:22
   9:     0x55d265bf34ca - std::panicking::default_hook::h8932b573145a321b
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:286:9
  10:     0x55d265bf3e88 - std::panicking::rust_panic_with_hook::h4b1447a24e3e94f8
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:688:13
  11:     0x55d265bd9408 - std::panicking::begin_panic::{{closure}}::h589a3bf2d2644459
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:608:9
  12:     0x55d265bd9726 - std::sys_common::backtrace::__rust_end_short_backtrace::he559fabaf885b24d
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:137:18
  13:     0x55d265bd9340 - std::panicking::begin_panic::h008fb3d754feeedd
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:607:12
  14:     0x55d265b1fb4b - quad_snd::snd::setup_pcm_device::h7f73e85bfcf806df
                               at /home/n/Projects/rust/quad-snd/src/alsa_snd.rs:57:9
  15:     0x55d265b1fd72 - quad_snd::snd::audio_thread::h86b2012b14fa8cf9
                               at /home/n/Projects/rust/quad-snd/src/alsa_snd.rs:104:22
  16:     0x55d265b51294 - quad_snd::snd::AudioContext::new::{{closure}}::h68fd989d4cc78137
                               at /home/n/Projects/rust/quad-snd/src/alsa_snd.rs:160:13
  17:     0x55d265b52ea3 - std::sys_common::backtrace::__rust_begin_short_backtrace::hcf333de2932cddba
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:121:18
  18:     0x55d265b30fd4 - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h05535e2e84fe1fb2
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/mod.rs:551:17
  19:     0x55d265b60f24 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h210f370aefd641b1
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panic/unwind_safe.rs:271:9
  20:     0x55d265b52ce6 - std::panicking::try::do_call::h056daec0624d5620
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
  21:     0x55d265b52e6b - __rust_try
  22:     0x55d265b52b7b - std::panicking::try::h77960d6c0f4026df
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
  23:     0x55d265b4e294 - std::panic::catch_unwind::h864133c4dbdc14e5
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
  24:     0x55d265b30de1 - std::thread::Builder::spawn_unchecked_::{{closure}}::hb7376751fe54742b
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/mod.rs:550:30
  25:     0x55d265b27c9f - core::ops::function::FnOnce::call_once{{vtable.shim}}::h9543f2293c6ebf61
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:251:5
  26:     0x55d265bf62c3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6202d10b0224e7b0
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/alloc/src/boxed.rs:1987:9
  27:     0x55d265bf62c3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h4dbea73c9cec160b
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/alloc/src/boxed.rs:1987:9
  28:     0x55d265bf62c3 - std::sys::unix::thread::Thread::new::thread_start::h0a2f4c32ba2f2278
                               at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys/unix/thread.rs:108
:17
  29:     0x7feedc99c8fd - <unknown>
  30:     0x7feedca1ed20 - <unknown>
  31:                0x0 - <unknown>
Audio thread died
Audio thread died
Audio thread died
Audio thread died

Playback state callbacks

Now library lacks support of any kind of feedback when particular sound has complete its playback.
For the in-game music this feature is must have

Sound volume variability

It would be nice to have an ability to spawn a sound with a volume correction