Force into Format Type 0 possible? about midilib

[mohanklein]

thanks jimm!!!

[jimm]

@mohanklein The Sequence.write method takes a second optional parameter that is the MIDI file format. Only 0 and 1 are supported, but if you pass in 0 then it will output a type 0 MIDI file.

[mohanklein]

@jimm This does force my test file into MIDI Format Type 0 but it seems in my test midi file all Note Events are now turned into repeatBits as children of a single NoteOn Event which seems wrong? Is this a desired behavior? Please see compare screens from "Synalyze It" Hex Editor compared.

I used this "Grammar" inside "Synalyze It" Hex Editor: https://raw.githubusercontent.com/synalysis/Grammars/master/midi.grammar
which states to implement this MIDI spec:
http://www.somascape.org/midi/tech/mfile.html

[mohanklein]

@jimm another screen showing only one single real Note On event is left - after using midilib ...

[jimm]

@mohanklein Hmm, you might have found a bug that the tests don't catch. Are you able to send me both MIDI files and whatever code you used to read the original file and output the Type 0 file?

[mohanklein]

@jimm I just tested this with another file and a "placebo" code not touching the midi other than forcing to type 0: same result. Attached in the zip file:

• an original type 1 file with multiple Note Events
• the processed type 0 file (after the midilib placebo code) having lots of repeatBit children for one Note On Event instead

#!/usr/bin/env ruby

require 'midilib'

class Processor
  def self.run
    seq = MIDI::Sequence.new()

    File.open('org.mid', 'rb') { |file|
      seq.read(file)
    }

    seq.each { |track|
      track.each { |event|
        # Don't do anything :-)
      }
    }

    File.open('processed.mid', 'wb') { |file| seq.write(file, 0) }
  end
end

Processor::run

midilib-format-type-bug.zip

[jimm]

Thank you, @mohanklein . I'll try to get to this bug, but I'm afraid I don't know how quick I'll be able to attack it.

[mohanklein]

hey @jimm! I just took another look into this and used the old version 2.0.5 which did not yet have the midi_file_format attribute for the sequence write method. The processed format type 1 file will have the same problem though (repeatBits and only one Note Event). Maybe this helps narrowing this bug problem down!

[jimm]

Thanks, that does help. I actually think that this isn't a bug (but I want to prove it as soon as I have time). I think it has to do with the fact that all of the note off messages in the original file are note on messages with a zero velocity. That is perfectly legal --- the MIDI spec says that note on messages with a zero velocity are the same as note off messages. This allows the MIDI to be compressed by outputting running status bytes, which I think are what those "repeat" messages are in the output you're displaying. Running status bytes means that if the same status byte would be repeated then it can be omitted. midilib does that when writing its output.

In fact, the issue describe with all of the notes being a "child" of one note on message might be perfectly correct. The first thing I have to do when I get to this is determine if the output you are seeing in fact perfectly legal and correct. There might not be any bug here. The output file is 96 bytes shorter than the input file, and I think that is due to midilib compressing the output by using running status messages.

Have you tried playing the file? Does it sound right?

[jimm]

I just re-processed the processed file, and the output is exactly the same. Even when the output type is switched, that doesn't make a difference --- the only difference in the two files is the single byte that stores the file type number. So at least midilib is being consistent :-)

[mohanklein]

Hey @jimm the org.mid file from the archive zip seems to have a strange structure (it is from a MIDI vendor I closely work together with). All their MIDI files have the same problem: When using them inside the drum software "Toontrack Superior Drummer" they will show up with a lot of more bars then the notes actually have, e.g. a 4 bar drum groove would show up as 4410 bars inside "Superior Drummer". When using midilib version 2.0.5 (until last summer when you fixed the note on 0 veloctiy behavior which I had found) the org.mid file from the zip archive would show the correct amount of bars inside "Superior Drummer" after running it through midilib. With the current version 4.0.2 this is no longer the case. That is why I looked into this and found this behavior. Attached a screenshot showing my original problem with the original "strange" MIDI files showing a lot of bars inside "Superior Drummer" and the processed v2 file which shows the correct amount of 4 bars and the processed v4 file which "keeps the bar problem". I do not know what exactly is going on inside "Superior Drummer" as I don't work with them - when I drag the "problematic" MIDI files directly to any DAW they are fine and only have the correct bars, e.g. 4. Attached also the org.mid and after conversion with midilib 2 and 4.

Archiv.zip

[jimm]

The original MIDI file's track 0 (all Type 1 files have a track 0 that contains metadata like the tempo and time signature) has an "end of track" meta event in it that has a delta time of 8467200, which is probably where the 4410 bars comes from. midilib keeps that end of track event.

[mohanklein]

but version 2.0.5 did not, correct?

[mohanklein]

so manipulating that end of track event's delta time is the solution to "fix" those files? is the end of track event mandatory for the meta track 1?

[jimm]

I'm not sure about version 2.0.5, but the method that adds an end of track event if it is missing was added early last year so all tracks that midilib outputs will have them. I believe end of track event is optional. If you remove that end of track event or fix it, you should be all good.

[mohanklein]

@jimm I don't think this explains the difference in behavior with the strange meta delta time file though ...

[jimm]

If the old version of midilib didn't write out the end track meta event then the drum software would have set the length based on whatever the last event was, probably a note. And perhaps rounded up to the nearest measure.

When I import org.mid and processed.mid into Reaper, they both show the same length of 4 bars, meaning that Reaper is most likely ignoring the end of track meta event. And the notes are all the same there.

Running both files through midilib's examples/reader2text.rb script shows that there is no difference in the actual note data at all. The only difference is in the number of tracks and where the end of track meta event is stored.

I believe that there is no midilib bug.

[jimm]

You could always use midilib to erase that "bad" end of track meta event before writing out the data, and then when the data gets written out midilib would add one that's based on the last note instead of being 4410 bars out.

[mohanklein]

I just did something like this in case of a strange long first meta track End of Track now:

if track_index == 0 && event.kind_of?(MIDI::MetaEvent) && event.meta_type == 47
  if event.delta_time > 0
    puts "Fixing strange Delta Time for End of Track event of #{event.delta_time}"
    event.delta_time = 0
  end
end

Bad idea? I guessed End of Track events should be 0 delta always?
How would I remove the event completely and do I have to recalc any time stuff?
No bug in midilib I am also pretty sure now sorry for wasting your precious time kind of :-)
Thanks!

[jimm]

That looks good. Since every track can have an end of track meta event, you probably shouldn't filter by track index. And you could optionally remove the event instead of changing the delta time. The end result should be the same.

To remove an event, you can call track.delete_event(event). If you're sure the event is the last event in the track already, you can pass in a calc_recalc_times value of false (that is, track.delete_event(event, false)) to avoid recalculating all of the events' start times. Those are calculated off of the delta times and they are ignored when writing out the data.

[jimm]

By the way, the delete_event method takes care of fixing the delta time of the following event if there is one. So you don't have to worry about that.