haberdashpi / weber.jl

A julia package for creating simple psychophysical experiments

Weber

About

Weber is a Julia package that can be used to generate simple psychology experiments that present visual and auditory stimuli at precise times.

Plans

While this is working on older versions of Julia (0.6), I have not had time to commit to updating this project as I am not actively developing any psychoacoustics experiments at the moment. I would not recommend using this package as is. It might be a useful place to start if you want to write your own package.

Installation

julia> Pkg.add("Weber")

For more details, see the documentation.

Documentation

• STABLE - Most recent version of documentation
• LATEST - In developement documentation

weber.jl's Issues

Port to Linux

Weber doesn't work on Linux right now. The two barriers are:

• We can't automatically install binaries due to BinDeps 199.
• My tests in VirtualBox lead to LVM errors. I need to test out Weber on an actual Linux install (maybe the LVM errors are VirtualBox specific).

I have no need for a Linux version right now, so this issue is low priority.

Create more examples

Examples should cover all relevant features of Weber, and be referenced in the documentation guide.

To start I can translate old psychotask experiments into Weber.

Create simple adaptive AFC primitives

Create some primitives for running adaptive tracks, to make this easier to implement for novice programmers.

Tests for @addtrials

There are some preliminary tests, but I can probably make more demanding tests of this macro.

Refactor tests

As we start getting more tests, it would be useful to organize them into separate files.

Fix title of getting started guide

Some a are repetitive or inconsistent with subsequent content.

Fix writing in extension guide

Use method and function consistently. There are some repeated uses of the same word in one sentence.

Improve code coverage and memory allocation

Once I've finished #6 I should be able to make some more thorough tests and then evaluate and improve the code coverage and memory allocation of Weber.

Add oddball_paradigm

This function, implemented for navy_aba is probably best moved to Weber.

Tests for ResponseMoment

Create machinery to generate mock events (using an extension) and then trigger those events, so we can test the ResponseMoment methods

Audible clicks during adaptive track usage.

The new sound library is introducing audible clicks... I think this is related to adaptive track usage

Report sound latency

Provide an easy way to report how long in advance a sound must be played on the present hardware. (Or throw some sort of latency warning).

Improve latency of timeout

Timeout sound events could occur precisely, if they could be aborted when they aren't needed. This would require a change to the interface of timeout and a change to weber_sound.c

Allow multiple sounds to play at once.

Current low-latency sound implementation supports playing only one sound at once. We could replace the single sound with an array of sounds (whose pointer is copied during callback) and update the entire array anytime a sound is added. There are some further sticking points to get this working right that I have to think through.

Eliminate long form moments

Remove long form moment implementation, and document new usage format. This includes changing timeout signature and other non-standard timed moments.

Proofread documentation

The new API updates have introduced a lot of typos into the documentation. Fix them.

Incorporate Unitful.jl

We could use the promotion rules here to easily handle Hz, milliseconds, and so forth.

Make mix and mult use longest sound.

For streams, it is more useful if mix'ed or mult'ed streams continue as long as there is at least one stream left, and this is more consistent with the behavior of these functions for discrete sounds.

Improve implementation of filters for streams.

Right now I believe the filter state is shared across all filtered streams. Furthermore, the output depends on lazymap's implementation. We should make a filter stream object that stores the filter state as the the iteration state.

Custom argument requests fail in Juno

No input prompt shows up for these custom arguments.

Test for memory leaks in streaming API

I'm worried the iterators interface, especially for filters, may cause memory leaks. I should write some tests for this.

Check for old build

If an old build is in place (with an old version of weber-sound), a cryptic error message occurs on the first call to running a sound. Check for the right version of the library and give a better error if it isn't available.

Cache visual and sound generation

If the same exact object is turned into a visual or a sound just use an existing object if it is present in an LRU cache.

Rare EXCEPTION_ACCESS_VIOLATION events

These always occur in SDL_mixer's Mix_Sound. I surmise there are cases where GC clears a sound object before it is played.

Create separate extension projects

Split the Cedrus support into a subproject on git. Make the DAQ plugin public.

Improve audio-playback latency

Currently sounds have a latency that depends on the buffer size of audio playback. However, in theory, one could zero pad samples just as they are about to be played to achieve more accurate timing regardless of how many samples are sent in the buffer at one time, by empirically tracking the time at which the audio samples are requested.

This could be achieved in one of two ways:

1. Wait for Julia to fully support multithreading (0.6.0? 0.7.0??), and implement a low-level playback interface in Julia.
2. Create custom C code to implement this playback feature.

Because of the potential issues with garbage collection the Julia implementation would raise, I slightly favor the C code approach, but it would demand a more complicated setup to make it easy for users to download and install Weber.

Fuse sounds into a single buffer.

When multiple sounds are presented in a single trial, with no expanding or response timed moments intervening, fuse those sounds into a single buffer, allowing them to play with more precise timing.

Support video playback?

It might be useful to allow the playback of videos???

I'm a little skeptical of implementing this because audio and video synchronization is often not very good, even in commercial video playback software. This makes me think good latency with this approach would be very hard to get right.

Precisely time audio streams

The old approach to stream timing doesn't work well, because it can pause non-swapped streams playing in parallel. I need to figure out some way to precisely time streams.

Possibilities

• Define precise times for streams that weber_sound then executes. This requires that I limit the set of mutations of streams that can be precisely timed to some limited grammar that weber_sound can interpret and execute on the fly.

• Provides some mechanism to retrieve the precise onset of played streams after the fact.

• In theory the precise timing of a stream can be predicted well before-hand by weber_sound. Based on the exact time when the next stream chunk should stop playing, the newly modified stream output can be offset. This would require some sort of filling-in procedure during the onset of a new, modified stream, in which the old, unmodified stream provides some fraction of the chunk size, and the new stream provides the remaining part. This requires that 'process' be able to specify what amount of audio data the iterator should provide. (Maybe have a new interface? Maybe just specify that the streaming_unit() can change on the fly and require streaming iterators to use the most recent value?).

Add image transformations

This would make generating images for an experiment easier. Alternatively we could get compose or some other vector graphics library working.

Improve usage instructions

Due to a bug in Atom, it is better to use include("filename") than to press the run button. Use this approach in the getting started section of the manual. Finish after #4 .

Create mockup for testing play and display

The play and display functions cannot themselves be tested, since their job is to play sounds and display images. However, there's a lot of infrastructure that doesn't actually interact with hardware that can be tested. I need to refactor this infrastructure to be independent of calls to SDL, and then have a NullWindow implementation of the infrastructure. Once this is done, I should be able to create tests of this infrastructure. It should also allow some more comprehensive tests of running experiments.

Use best possible `prepare!` time.

If sounds occur at the start of a trial, they cannot be precisely timed right now. Fix this by synchronizing the sound to the most recently played moment with a nonzero delta_t.

Improve debug with `@run_args`

Rather than just providing instructions in a warning, actually handle debugging with an extra parameter.

Allow special keys

Allow key codes for key combinations like "Ctrl-A".

Improve latency of stop and pause.

Currently weber-sound call stop and stop stream, but we need only change the length of the playing stimulus to stop and start, and this would reduce latency.

refactor FinalMoment

We no longer need this moment. We can just check if all queue are empty and terminate the experiment at that point.

Improve channel usage

weber-sound could better utilize its channels if all channels that were too late to play a sound were eliminated, and the channel finished latest were utilized.

Document Extensions

Requires #13.

Make a list of available extensions in Weber's documentation, and provide some usage examples (probably in the extension README.md).

Verify Cedrus keycode helper

Make sure the Cedrus extension shows keycodes when run_keycode_helper is called.

Drop playback of high latency sounds and flag a warning

Sounds that play way too late can really mess things up (e.g. creating a lot of undesired overlap). If a sound plays way to late, just throw a warning and don't play the sound.

Fix gaps in stream playback

When modifying a streams, sometimes there is a gap in playback. This could be fixed by addressing #46, but there is probably a simpler, temporary solution to this problem.

Get c code compilation to work on Windows

On Windows, compilation of c code creates a *.dll that Julia cannot run when compiled with msvc or msys's gcc. I'm completely lost as to what is going wrong, and have no good sense of where to begin troubleshooting. I've posted a question to stack exchange.

A possible workaround: define a simple inter-process protocol, compile the c code as an executable and then send the sounds to a child process.

Fix audio glitches with default windows settings.

We just need to set the environment variable SDL_AUDIODRIVER to winmm on windows by default.

Without this set, there are audio glitches. However, ideally, I need to make sure that, if this variable is set by the user, I respect the user's wishes.

Fix UI annoyances when using Juno.

Weber is kind of annoying to use from Juno right now, especially in Windows. The problems are:

1. Keyboard input is not captured. You have to alt-tab away from and then back to the experiment window when running from Juno.
2. On completion, the experiment window doesn't close. You have to alt-tab to get back to anything useful.
3. The input command in Juno sucks. It doesn't put the cursor in the input area, and if the Console is closed, it waits for input silently, leaving the user to wonder why nothing is happening.

This could all be fixed by:

• spawn a separate process for the experiment when in Juno (fixed by 6d76ff3)
• use a Gtk window to collect input (in its own process) OR implement simple UI in SDL.

Alternatives to this second step would be implementing some other OS neutral UI (electron???) or submitting a pull request to Juno that enhances the input function so it works better.

Note - some part of this should maybe be part of the 0.3.x milestone??

Add doc guide for Adapters

At the moment there is only a reference section, but a guide with an example would make this feature much more usable.

Implement slides

This would allow moving both forward and back when showing instructions

Fix or remove helpers.

Some of the helpers are outdated (run_calibrate). Make sure they work with the new API or remove them.

Fix missing functions in event documentation

This appears to be a doc generation error...

More documentation editing.

I have done an initial once through checking of documentation. There are probably still spelling errors and typos that I've not caught. Also, there are probably still places where the documentation is unclear and needs to be rewritten.