urswilke / pyramidi

Generate & manipulate midi data in R data frames

pyramidi

Introduction

pyramidi is a very experimental package to generate / manipulate midi data from R. Be aware that a lot of the code I’ve written some years ago hurts my eyes when I look at it now :) Midi data is read into dataframes, using the python package miditapyr under the hood (which itself uses the excellent mido). The notes’ midi information (one line per note_on/note_off midi event) is translated into a wide format (one line per note). This format facilitates some manipulations of the notes’ data and also plotting them in piano roll plots. Finally, the modified dataframes can be written back to midi files (again using miditapyr).

Thus, you can manipulate all the intermediate dataframes and create midi files from R. However, you need to make sure yourself that the midi files you write can be understood by your softsynth. The data is not yet validated by pyramidi, but mido (also used to write midi files) already catches some of the possible inconsistencies.

If you’re new to midi, mido’s documentation might be a good start.

New since version 0.2

The midi data can now be

• played live in the R console OR generate a sound file and a html audio player when knitting rmarkdown documents thanks to the excellent R packages fluidsynth (see the documentation of the play() method in the MidiFramer class and its helper function player() which use
• fluidsynth::midi_convert() to synthesize midi to wav files (needs fluidsynth installed, but if I understand correctly R will do that for you)

Installation

You can install pyramidi from R-universe with:

install.packages('pyramidi', repos = c('https://urswilke.r-universe.dev', 'https://cloud.r-project.org'))

The python package miditapyr also needs to be installed in your python environment used by reticulate.

pip install miditapyr

But if everything works as I believe it should, miditapyr is automatically installed if you install pyramidi, as soon as you access the module for the first time.

Otherwise, you can also install it in your reticulate python environment with the included helper function:

pyramidi::install_miditapyr()

I’m not sure if that works on windows too. Perhaps there you have to manually configure your reticulate environment.

Usage

Generate a MidiFramer object

We can create a MidiFramer object by passing the file path to the constructor method (new()).

library(pyramidi)
library(dplyr)
midi_file_string <- system.file("extdata", "test_midi_file.mid", package = "pyramidi")
mfr <- MidiFramer$new(midi_file_string)

The object contains the midi data in various dataframe formats and an interface to the miditapyr miditapyr.MidiFrames object mfr$mf. You can write the midi file resulting of the MidiFramer object to disk:

mfr$mf$write_file("/path/to/your/midifile.mid")

Modifying midi data

In the MidiFramer object, we can modify mfr$df_notes_wide, the notes in note-wise wide format (note_on & note_off events in the same line). Thus we don’t need to worry which midi events belong together

Let’s look at a small example. We’ll define a function to replace every note with a random midi note between 60 & 71::

mod <- function(dfn, seed) {
  n_notes <- sum(!is.na(dfn$note))
  dfn %>% mutate(note = ifelse(
    !is.na(note),
    sample(60:71, n_notes, TRUE),
    note
  ))
}

When we call the update_notes_wide() method, the midi data in mfr is updated:

mfr$update_notes_wide(mod)

Writing modified midi files

Thus, we can now save the modifications to a midi file:

mfr$mf$write_file("mod_test_midi_file.mid")

Synthesizing and playing audio

See the vignette("pyramidi", package = "pyramidi") to see how you can synthesize the midi data to wav, convert to mp3 if you want, and then embed a player in your rmarkdown html document with

mfr$play("mod_test_midi_file.mp3")

*The player only appears in the docs.

Even if that sound is very weird, I was very happy not having to listen to the package midi file over and over again. :)

Documentation

You can find the complete online documentation of the package here.

• See the vignette("pyramidi") for a brief usage introduction how to manipulate midi data.
• The vignette("compose") shows a more extended example how to compose music and generate midi files from scratch.
• vignette("package_workflow") shows in detail the structure of the MidiFramer class and the functions of the pyramidi package.
• vignette("functions_usage") illustrates the low-level functions of the pyramidi package, that MidiFramer objects use under the hood.

pyramidi out in the wild

To see examples where pyramidi is used for midi mangling in R dataframes etc. (amongst plenty other of his awesome writings about music), please check out Matt Crump’s blog and his package midiblender!

Related R packages

• The tabr package is a massive music notation, transcription and analysis program allowing to create musical scores (using Lilypond). It allows to read midi files (wrapping tuneR; see below) and also to export them (also using Lilypond).
• The gm package also allows to create and show musical scores using musescore. It also allows to export the music to audio (also using musescore) and to embed the players in html documents.
• The noon package wraps node.js libraries and can be used to read live midi input port data. I wrote a small blog post how reading a midi port can also be done in R with mido. Interestingly, the node.js libraries and mido rely on a the same C++ library RtMidi.
• The tuneR package can also read in midi data. See the vignette("tuner"), for an example how you can transform the tuner format into the pyramidi format.

R packages used

This package stands on the shoulders of giants. A big thank you to the authors of the following libraries!

Package	Version	Citation
base	4.3.2	R Core Team (2023a)
details	0.3.0	Sidi (2022)
DiagrammeR	1.0.11	Iannone and Roy (2024)
fluidsynth	1.0.0	Ooms (2024)
glue	1.7.0	Hester and Bryan (2024)
grateful	0.2.6	Rodriguez-Sanchez and Jackson (2023)
htmltools	0.5.7	Cheng et al. (2023)
knitr	1.45	Xie (2014); Xie (2015); Xie (2023)
pichor	0.0.0.9030	Andersen (2024)
pkgdown	2.0.7	Wickham, Hesselberth, and Salmon (2022)
R6	2.5.1	Chang (2021)
reticulate	1.35.0	Ushey, Allaire, and Tang (2024)
rmarkdown	2.25	Xie, Allaire, and Grolemund (2018); Xie, Dervieux, and Riederer (2020); Allaire et al. (2023)
spelling	2.2.1	Ooms and Hester (2023)
testthat	3.2.1	Wickham (2011)
tidyverse	2.0.0	Wickham et al. (2019)
tools	4.3.2	R Core Team (2023b)
tuneR	1.4.6	Ligges et al. (2023)
usethis	2.2.2	Wickham et al. (2023)
zeallot	0.1.0	Teetor (2018)

Allaire, JJ, Yihui Xie, Christophe Dervieux, Jonathan McPherson, Javier Luraschi, Kevin Ushey, Aron Atkins, et al. 2023. rmarkdown: Dynamic Documents for r. https://github.com/rstudio/rmarkdown.

Andersen, Mikkel Meyer. 2024. pichor: Piano Chords in r.

Chang, Winston. 2021. R6: Encapsulated Classes with Reference Semantics. https://CRAN.R-project.org/package=R6.

Cheng, Joe, Carson Sievert, Barret Schloerke, Winston Chang, Yihui Xie, and Jeff Allen. 2023. htmltools: Tools for HTML. https://CRAN.R-project.org/package=htmltools.

Hester, Jim, and Jennifer Bryan. 2024. glue: Interpreted String Literals. https://CRAN.R-project.org/package=glue.

Iannone, Richard, and Olivier Roy. 2024. DiagrammeR: Graph/Network Visualization. https://CRAN.R-project.org/package=DiagrammeR.

Ligges, Uwe, Sebastian Krey, Olaf Mersmann, and Sarah Schnackenberg. 2023. tuneR: Analysis of Music and Speech. https://CRAN.R-project.org/package=tuneR.

Ooms, Jeroen. 2024. fluidsynth: Read and Play Digital Music (MIDI). [https://docs.ropensci.org/fluidsynth/ https://ropensci.r-universe.dev/fluidsynth](https://docs.ropensci.org/fluidsynth/ https://ropensci.r-universe.dev/fluidsynth).

Ooms, Jeroen, and Jim Hester. 2023. spelling: Tools for Spell Checking in r. https://CRAN.R-project.org/package=spelling.

R Core Team. 2023a. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

———. 2023b. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Rodriguez-Sanchez, Francisco, and Connor P. Jackson. 2023. grateful: Facilitate Citation of r Packages. https://pakillo.github.io/grateful/.

Sidi, Jonathan. 2022. details: Create Details HTML Tag for Markdown and Package Documentation. https://CRAN.R-project.org/package=details.

Teetor, Nathan. 2018. zeallot: Multiple, Unpacking, and Destructuring Assignment. https://CRAN.R-project.org/package=zeallot.

Ushey, Kevin, JJ Allaire, and Yuan Tang. 2024. reticulate: Interface to “Python”. https://CRAN.R-project.org/package=reticulate.

Wickham, Hadley. 2011. “testthat: Get Started with Testing.” The R Journal 3: 5–10. https://journal.r-project.org/archive/2011-1/RJournal_2011-1_Wickham.pdf.

Wickham, Hadley, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain François, Garrett Grolemund, et al. 2019. “Welcome to the tidyverse.” Journal of Open Source Software 4 (43): 1686. https://doi.org/10.21105/joss.01686.

Wickham, Hadley, Jennifer Bryan, Malcolm Barrett, and Andy Teucher. 2023. usethis: Automate Package and Project Setup. https://CRAN.R-project.org/package=usethis.

Wickham, Hadley, Jay Hesselberth, and Maëlle Salmon. 2022. pkgdown: Make Static HTML Documentation for a Package. https://CRAN.R-project.org/package=pkgdown.

Xie, Yihui. 2014. “knitr: A Comprehensive Tool for Reproducible Research in R.” In Implementing Reproducible Computational Research, edited by Victoria Stodden, Friedrich Leisch, and Roger D. Peng. Chapman; Hall/CRC.

———. 2015. Dynamic Documents with R and Knitr. 2nd ed. Boca Raton, Florida: Chapman; Hall/CRC. https://yihui.org/knitr/.

———. 2023. knitr: A General-Purpose Package for Dynamic Report Generation in r. https://yihui.org/knitr/.

Xie, Yihui, J. J. Allaire, and Garrett Grolemund. 2018. R Markdown: The Definitive Guide. Boca Raton, Florida: Chapman; Hall/CRC. https://bookdown.org/yihui/rmarkdown.

Xie, Yihui, Christophe Dervieux, and Emily Riederer. 2020. R Markdown Cookbook. Boca Raton, Florida: Chapman; Hall/CRC. https://bookdown.org/yihui/rmarkdown-cookbook.

pyramidi's Issues

Support native R backends for rendering midi

The R packages av (on cran) and fluidsynth (not yet on cran but soon) allow for synthesizing and converting audio without requiring the user to install any extra shell utilities. Perhaps supporting that back-end can lower the barrier to use pyramidi.

install.packages(c('av', 'fluidsynth'), repos = 'https://ropensci.r-universe.dev')

Example use:

midi <- system.file("extdata", "test_midi_file.mid", package = "pyramidi")
wav <- fluidsynth::midi_convert(midi, output = 'test.wav')
av::av_audio_convert("test.wav", "test.mp3")

Midi player in readme doesn't appear in github repo

The midi player on https://github.com/urswilke/pyramidi#synthesizing-and-playing-audio
doesn't appear anymore.

I'm quite sure it was there before.

data dictionary for dataframe objects in MidiFramer

Thank you for this package :)

I was an acoustic musician in a previous life, now a math nerd data scientist. This is the first time I've worked with midi data. Using music theory, I'm making sense of the note and b columns, however I think as I progress with this project, I suspect I'll end up reinventing the wheel without understanding all the columns output in the dataframes provided.

Could you please point me to documentation that will help me understand what the columns are in the various objects output by the midi transformations provided? Apologies if this is super obvious in the documentation and I failed to see it.

Control change specification?

Thanks a lot for the development of this package! been having a lot of fun exploring it and so far everything works great. I was just curious to know if midi cc has been implemented? tried to import a midi file with midi cc values that I exported from Ableton to see if I could reverse engineered , but got an error.