jurihock / voyx

Standalone real time dynamic vocal harmonizer

voyx

The project idea is essentially to harmonize the source vocal audio signal just in real time depending on the MIDI input. Each MIDI key determines the particular pitch shifting amount relative to the detected fundamental frequency of the source signal segment.

The required multi pitch shifting feature is already implemented in the stftPitchShift project. An additional idea is to utilize the Sliding DFT instead of STFT.

Let's get started...

voyx's Issues

Including OpenCL code

https://stackoverflow.com/questions/1415538/using-include-to-load-opencl-code

PDA

Approaches:

• Spectral peak picking / Vocoder / HPS (still insufficient spectral resolution)
  • Sliding Q-Transform + "Fast, Accurate Frequency Estimators" + HPS (better spectral resolution)
• Cepstral analysis (e.g. combined with formant extraction)
• Autocorrelation / YIN (e.g. via separate CPU task)
• F0 detection via regression analysis
• CREPE via Torch

See also:

• Comparison of Pitch Trackers for Real-Time Guitar Effects
• Real-time Pitch Tracking in Audio Signals with the Extended Complex Kalman Filter + orchidas/Pitch-Tracking
• sevagh/pitch-detection
• Traditional Machine Learning for Pitch Detection
• Fast, Accurate Frequency Estimators

TODO:

• https://dsp.stackexchange.com/questions/27029/fast-pitch-recognition
• https://dsp.stackexchange.com/questions/29962/how-to-deal-with-low-fundamental-when-using-amdf-for-pitch-extraction
• https://dsp.stackexchange.com/questions/22067/what-is-an-amdf
• https://stackoverflow.com/questions/973370/fast-average-square-difference-function
• https://stackoverflow.com/questions/3949324/calculate-autocorrelation-using-fft-in-matlab
• https://stackoverflow.com/questions/4583950/cepstral-analysis-for-pitch-detection

Consolidate audio pipeline

Destination project aubergine

Experiments on accelerating algorithms

• vDSP vector based processing (Apple specific)
• Metal GPU parallelization (Apple specific)
• OpenCL GPU parallelization
• Limiting signal bandwidth
• Utilize both CPU and GPU simultaneously
• atan2 approximation (instead of std::arg)
• Reduce sample rate (e.g. downsample input and upsample output)

trying to get this running, using msvc and /std:C++20

I have an application where this looks like a great starting point, have been trying to get it working in a windows Visual Studio environment. I have a number of small changes, but wanted to check with you to see if the latest issue may be impacting you as well -

In AudioSink.cpp

AudioSink::AudioSink(const std::string& name, double samplerate, size_t framesize, size_t buffersize) :
  Sink(samplerate, framesize, buffersize),
  audio_device_name(name),
  audio_sync_semaphore(buffersize),
  audio_frame_buffer(
    buffersize,
    [framesize](size_t index)
    {
      auto output = new OutputFrame();
      output->index = index;
      output->frame.resize(framesize);
      return output;
    },
    [](OutputFrame* output)
    {
      delete output;
    })
{
}

Is it possible that audio_sync_semaphore(buffersize), should actually be audio_sync_semaphore(1),

look forward to your thoughts.

Add vertical line plot

QCPItemStraightLine