soundshader / soundshader.github.io

Visualising sound with ACF

1. The AutoCorrelation Function
2. Gallery of ACF images
3. Highres spectrograms with the DFT Shift Theorem
4. Wigner transform spectrograms
5. Vowels and colored ACF
6. Cat meow sounds as colored ACF
7. The Lesser-Known Side of Discrete Cosine Transform
8. Visual Morphology of Vowels

License

AGPLv3

soundshader.github.io's Issues

(Firefox) "Different sample-rate is currently not supported." when uploading song with 199kbps

Overview

An error regarding sample rates occurs given a specific mp3 file. I would share the file for reproduction but it's a copyrighted work.

MP3 file Properties

• filename: "retribution.mp3"
• bitrate: 199kbps
• length: 0:03:00
• size: 4.38 MB
• other info: converted from FLAC to mp3 using VLC's "convert/save" context menu and the "lame" codec.

Error message

Immediately upon upload a toast notification appears in the bottom left saying:

AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported.

After clicking the toast itself the notification's text changes to:

start@https://soundshader.github.io/audio/controller.js:108:33 setMouseHandlers/btnUpload.onclick@https://soundshader.github.io/index.js:129:30

which I assume to be just an unrelated click handler debug event that's now being shown because your toast is just a catch-all for debug logging.

Best guess

My best guess as to why this happens is because my mp3 is in an unexpected format due to me converting it from FLAC using VLC. I'll reconvert with different options and report back.

Unrelated, but it's very difficult to actually copy the initial message to the clipboard due to a click on it causing it to change. I had to use screen capture and re-type it. Keeping a persistent error message would be helpful for people who wanted to file issues. Having said that, thanks for putting some debug output upon error at all. That stuff is usually in a dev console and the increased visibility makes it easier for people who are "just stopping by" to fix their own problems/file an issue.

Edit: ^ This has since been fixed

Awesome project by the way. I felt like I was hitting a wall in finding creative techniques to visualize sound with the classic fourier analysis. Don't know how this technique has evaded me for so long.

MP3 plays but no visualization - Console error messages in Chrome

Selecting MP3 works, and MP3 begins to play. However canvas remains black. Console messages pasted below:

Chromium 83.0.4103.61

At startup:

Uncaught (in promise) TypeError: this.audioCtx.createAnalyser is not a function
at AudioController.init (controller.js:21)
at getAudioController (index.js:85)
at HTMLCanvasElement.canvas.onclick (index.js:36)

At begin play:

Uncaught (in promise) TypeError: this.audioCtx.createMediaStreamSource is not a function
at AudioController.start (controller.js:106)
at HTMLCanvasElement.canvas.onclick (index.js:51)

Delay / lag / synchronization

Fantastic visualizer and amazing work; thank you very much for making this.

Maybe it's just a placebo effect, but I feel like the phase of the visualizer isn't quite in sync with the music I've tried playing. I'm not sure, but I think it seems to be noticeably delayed, and the delay doesn't seem to be at a simple beat division, so the movement of the visualizer feels like it doesn't fit the rhythm of what I'm hearing. This is probably the most beautiful and intuitive music visualizer I've used, but due to this perception, when it's actually playing live I can't really appreciate it.

If it's doing all of the calculations in real-time, is there any way to run it ahead of time against an audio file and pre-generate the visualization so that it can play perfectly in sync with the audio?

If I'm just crazy or something, please let me know. I'm just trying to understand why it subjectively seems like a rhythmic mismatch to my ears.

Auto-detect sample rate of mp3 files

A note from HN:

What you can do though is look at the first few bytes of an .mp3 file (since it's a file drop/file load) to just directly read the sample rate from the MP3 block header[1], where you directly check the value encoded by [data[19], data[20]]: if it's [0,0] that means it's 44100, [0,1] means it's 48000, [1,0] means it's 32000 and that's it. There are no other sample rates allowed for MP3.

[1] http://mpgedit.org/mpgedit/mpeg_format/MP3Format.html for the full block format)

Motion Aftereffect can be disorienting

Not an issue so much as a notice to others who find themselves staring at visualizers for minutes on end.

Doing so with soundshader pretty consistently results in an optical phenomenon known as motion aftereffect in which static imagery appears to shift for awhile. It can be a bit jarring if you're not expecting it, so just letting folks know.

https://en.wikipedia.org/wiki/Motion_aftereffect has some info on the subject

with hss, option to generate whole spectrum of a long sound and save result

In the /hss view, Instead of using the A and D key to display part of the spectrum that is not visible on the canvas, would it be possible to generate the whole spectrum of a sound (of any length) and save it in a (png) file?

Downsample from stereo to mono instead of using just one track

I noticed here that it looks like you're only capturing the left (I think? or is 0 the right side?) channel of the audio track -

soundshader.github.io/audio/cwt.js

Line 22 in a72719c

this.audioSamples = this.audioBuffer.getChannelData(0);

Is it possible to downsample both tracks to mono instead in order to get any missing audio from the other channel for stereo-heavy tracks?

What about visualizing both channels simultaneously?