music-encoding / tablature-ig

Repo for The Tablature Interest Group

The Music Encoding Initiative

The Music Encoding Initiative (MEI) is an open-source effort to define a system for encoding musical documents in a machine-readable structure. MEI brings together specialists from various music research communities, including technologists, librarians, historians, and theorists in a common effort to define best practices for representing a broad range of musical documents and structures. The results of these discussions are formalized in the MEI Source and customizations, a core set of rules for recording physical and intellectual characteristics of music notation documents expressed in TEI’s ODD language (One Document Does-it-all, cf. amongst others: Viglianti, 2019). As such, the MEI Source contains both, the specifications that can be compiled to schema formats for validating XML files, and documentation in prose, the MEI Guidelines, which provide detailed explanations of the components of the MEI model and best practices suggestions.

The MEI Source is not a schema in itself; rather, it can be used to build customized schemas, such as mei-CMN, mei-Mensural, mei-all, etc. (also see Customizing MEI). This repository already includes several customizations. While these can form an ideal starting point for creating your own customizations, you should also understand customization and building processes.

In this document, you will learn how to contribute to the development of MEI by building the schema and guidelines (you should also consider consulting the tutorial on "Understanding ODD"). For the pre-built schemas of the latest release of MEI, please consult the "schemas" section of the music-encoding website.

Structure of this repository

This repository contains all the source code of the MEI Schema and Guidelines:

• .github: Configuration files for GitHub Actions workflows.
• customizations: TEI ODD files that allow you to build customized MEI schemata.
• source: Contains the source code, as expressed in TEI ODD. This includes the source code for the MEI Guidelines and the MEI schema modules.
• submodules: A container for Git Submodules: third-party developments that are needed, e.g., for building this repository's contents, but which are not part of our codebase.
• tests: Unit tests for the MEI Schemata.
• utils: Helper scripts e.g. for compiling schemata and guidelines from this repository.

Important

For the sake of the continuous integration (CI) workflow, the build artifacts of the schemata and guidelines are no longer included in this repository, but are generated on every commit to the develop branch and automatically pushed to their dedicated repositories. For more information please see the section Additional Resources below.

Nevertheless, it is possible to build any customization locally in your working copy of this repository.

Validating MEI files against an MEI Schema

One of the core strengths of the MEI Schema is that it allows an individual to validate an MEI file against an XML Schema to ensure the MEI file conforms to expected encodings and behaviors. To validate an MEI file you need an XML validation engine. XML Authoring tools, such as oXygen, might have built-in validation tools. There are also several command line utilities, including xmllint and jing.

For example, you might validate an MEI file from the sample-encodings project using the xmllint command line tool:

xmllint --noout --relaxng schemata/mei-CMN.rng "sample-encodings/MEI 3.0/Music/Complete\ examples/Bach_Ein_festeBurg.mei"

Or, the same command using jing.

jing schemata/mei-CMN.rng "sample-encodings/MEI 3.0/Music/Complete\ examples/Bach_Ein_festeBurg.mei"

Customizing MEI

The MEI model may be customized to express and validate different types of music documents. Customizations are configured with individual ODD files. This repository already includes several customizations:

• mei-CMN: Validates MEI files that express common Western music notation.
• mei-Mensural: Validates MEI files that express white Mensural notation (will raise validation errors if elements like "measure" exist in the MEI encoding).
• mei-Neumes: Validates MEI files that express Neume notation (like Mensural, will raise validation errors if elements that are not part of neume notation exist in an encoding.)
• mei-all: The full MEI Schema. This is the most permissive customization of MEI.
• mei-all_anyStart: A customization of mei-all, allowing every MEI element as the root element.
• mei-basic: The purpose of mei-Basic is to serve as common ground for data interchange, both between projects using different profiles of MEI, and other encoding schemes

Why Customizations?

For those who are used to having a single DTD or W3C Schema to validate music notation encodings, the customization process may seem to be a complex way of arriving at a schema to validate music notation. However, customizations are a vital part of the expressive power of MEI, and when used to their full extent, can assist organizations in ensuring the integrity and validity of their data.

When designing a music encoding system there are many contradictory and non-standardized practices associated with writing music notation. Different repertoires may have extremely different ways of expressing pitch or rhythm; for example, rhythm in Mensural notation is incompatible with the later systems developed in common Western notation.

Most attempts at addressing this complexity restrict a schema to only a certain subset of music notation and do not attempt to accurately represent the semantics of music notation that falls outside of its defined scope. So, for example, a system designed for common Western notation that depends on the existence of measures, duration, note shapes, or even staves, cannot semantically represent notations that do not have these features.

The MEI takes a different approach. With the customization system, schemas may be generated from an existing ‘library’ of well-defined musical behaviors, but each behavior may be mixed and matched according to the needs of the notation. In this sense, the MEI source functions more as a ‘library’ of music encoding tools by which many different types of notation can be expressed and not just a single monolithic schema.

Building MEI

The build process of MEI can produce several artifacts, as listed below. Depending on what artifacts you want to build, the build system has to meet additional requirements.

Artifacts	Technologies	Tools
Compiled ODD	XInclude, XSLT 3.0	Saxon HE with Xerces
Schemata	XInclude, XSLT 3.0	Saxon HE with Xerces
Guidelines HTML	XInclude, XSLT 3.0, MEI to SVG	Saxon HE with Xerces, Verovio Toolkit
Guidelines PDF	XInclude, XSLT 3.0, MEI to SVG, HTML to PDF	Saxon HE with Xerces, Verovio Toolkit, Prince XML

While it is possible to build the artifacts with other tools, the above are tested and thus recommended. Moreover, we try to support the building process on the most common operating systems (Microsoft Windows, Apple macOS and Linux), and currently have the following documented ways of building the artifacts:

• command line: If your build system meets the prerequisites as described in Building MEI on the command line, you can build the artifacts natively.
• Docker: MEI maintains a Docker image that meets all the prerequisites and can be used for building the artifacts via the command line. How to use the Docker image is described in the README of the docker-mei repository.
• oXygen XML Editor: When the command line is not your preferred tool and you do not want to build the Guidelines PDF, you might consider using alternative environments, such as Synchrosoft’s oXygen XML software family. A description of how to set up corresponding transformation scenarios can be found in Building MEI with oXygen XML Editor.

Additional Resources

A live version of the MEI Guidelines is available on the MEI Website in the ’Documentation‘ menu:

• MEI development version
• MEI version 5
• MEI version 4
• MEI version 3

In addition to the source files for MEI in this repository, there are other useful resources in other MEI repositories. The prebuilt release and development versions of:

• MEI Schema files: https://github.com/music-encoding/schema
• MEI Guidelines: https://github.com/music-encoding/guidelines

And moreover

• MEI Sample Encodings: https://github.com/music-encoding/sample-encodings
• MEI Encoding Tools: tools for modifying MEI files and conversion of MEI to/from other encoding formats https://github.com/music-encoding/encoding-tools

Referenced Material

• Viglianti, R. (2019). One Document Does-it-all (ODD): A language for documentation, schema generation, and customization from the Text Encoding Initiative. Symposium on Markup Vocabulary Customization, Washington, DC. https://doi.org/10.4242/BalisageVol24.Viglianti01

License

Copyright 2017-2023 by the Music Encoding Initiative (MEI) Board (formerly known as "MEI Council")

Licensed under the Educational Community License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://opensource.org/licenses/ECL-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

This is a derivative work based on earlier versions of the schema © 2001-2006 Perry Roland and the Rector and Visitors of the University of Virginia; licensed under the Educational Community License version 1.0.

CONTACT: [email protected]

tablature-ig's Issues

Modelling questions and details lute tablature

What?

This issue lists various questions and details regarding the modelling of lute tablature that are still to be addressed.

Questions and details

• Mensuration symbols (see phalese-1546_20-no_2 for an example of a slashed C). How do we do these:
  • Using a <mensur> element as a child of the <staffDef> (current approach).
    • The slashed C can be made using the attribute @slash.
  • Making it an attribute of the <staffDef> - but how?
    • Using @meter.count, @meter.unit (and, if necessary, @meter.sym)?
• Dots. Do we need to specify the vertical position of the right hand (fingering) dot (see, e.g., milano-1546_6-no_8)?
  • DL: Is @vo meaningful here?
  • TC: Well, there are examples of dots being written above a tablature letter/number; I don't think there is a universal understanding of the meaning of this. (It could mean staccato in certain cases.) And dots can be used as ornament signs or playing indications. So my overall feeling is that we should include this as a possibility. Default would be a dot under the letter/number to indicate r.h. first finger (= strike the course/string 'upward' = [usually] weaker beat).
  • DL: I don't think we can do this ambiguity in MEI - the editor probably has to get off the fence rather than encode a neutral 'dot'. If it's articulation, there's <artic>, if it's a fingering, then it's <fing>, etc. There is no generic dot. I think there's a compass point positioning for some things, but I can't remember what. Certainly it's used in mensural. That might be usable here.
  • RdV: Sorry, I wasn't clear. I meant the RH index finger/note-on-the-weaker-beat dot. That's <fing>, obviously - and I guess that makes that they will always be placed below the note by the renderer? I am not sure if there even are cases where this type of dot is not placed directly below the note. Probably...
• We need to include a @notationtype='tab.lute.spanish' for ‘Spanish’ tablature (Milan 1536, El maestro), which is currently missing.
  • DL: We should make an example at some point, but in principle, the only difference should be notation type.
  • TC: This is effectively what is used in modern guitar tablature - and flamenco 'cifras'.
• German tab: how do we deal with compound symbols, such as the letters or numbers with a line above them (see gerle-1552_1-no_4) which are currently done (incorrectly) with @symbol.mod='strikethrough'?
• How to indicate a tenuto sign preceding the glyph (see gerle-1552_1-no_4, m. 4 (2x) and 6)? Does it need some sort of position indicator as well? Currently, it is done using a new custom <tenuto> element that contains the attributes @startid, @position, and @style.

Enable rendering of Spanish tablature

Example:

milan-1538_1-no_2.png

Enable irregular barlines

Barlines in tabulature can be irregular, even though the mensuration does not change. We should have the flexibility to deal with this.

Example:
phalese-1547_7-no_21.png

Fix very large bar number rendering (?)

Verovio currently renders the bar numbers a bit too large.

Example:

Implement footnotes

In the case of adaptations, it would be good to be able to have the information that is enclosed in the <corr> part of a <choice> element shown in a footnote at the bottom of the page. Perhaps this is already possible in MEI/Verovio; I could not find anything about it.

Update, clean, and complete TabMEI documentation

What?

This issue lists information about the modelling of lute and guitar tablature that we might possibly want to include in the documentation - for example because it explains specific choices.

Example encodings

Lute tablature fragments:

• Italian tablature example: da_crema-1546_10-no_6 (= gerle-1552_1-no_4)
• German tablature example: gerle-1552_1-no_4 (= da_crema-1546_10-no_6)
• French tablature example: phalese-1546_20-no_2 (= milano-1546_6-no_8)
• Italian tablature example: milano-1546_6-no_8 (= phalese-1546_20-no_2)

Guitar tablature fragments:

• slurs example: van_halen-hot_for_teacher
• slides example: satriani-not_of_this_earth
• combined slurs + slides example: satriani-ice_9
• various techniques example: van_halen-panama

General

• Obvious errors (or editorial changes) are corrected (or realised) using a <sic>...</sic> element pair followed by a <corr>...</corr> element pair, so that all information remains in the encoding.

Lute tablature

• @dots='<n>' should go to the <tabGrp>: the dot applies to the rhythm (which gives rise to the symbol).
• n-line staves (where n != 6) are handled using @lines='<n>' in the <staffDef> (see, e.g., phalese-1546_20-no_2, where a five-line staff is used).

Guitar tablature

• We use @dur (rather than @dur.ges) in the <tabGrp> - the duration is unambiguous here.
  • DL: OK, my guess for why we did this: it's inter-chord interval, not duration. I can happily argue either way for this, and it might be worth revisiting in the SIG.
  • RdV: I thought the issue was that in the lute tablature the actual duration depends on interpretation (does |\ mean quarter note? half note?), while in the guitar tablature it is actually directly given in the CMN staff above (so no interpretation/ambiguity)?
• In the case of muted notes, the tab.fret value is deduced from the CMN staff above.
• ‘Single-note hammer-ons’ (see satriani-not_of_this_earth, m. 2; van_halen-hot_for_teacher, m. 2, 3): <slur> notes can have no @endid (cf. one-note slides) because they are 'single-note hammer-ons': notes not played with the RH but sounded by hammering 'out of nothing' a LH finger onto the fretboard. @show.dirmark='true' must in these cases always render 'H'.
• I left @startids referring to non-visible notes (i.e., notes just before or after the fragment) in for now (I assume that they won't break the parsing but just give a warning) (see satriani-ice_9 and van_halen-hot_for_teacher).
• Guitar tablature can have above the staff
  • No rhythm symbols (the rhythm indication is done in the accompanying CMN staff; see, e.g., all guitar examples in this repository). In this case, only <tabGrp> elements need to be used.
  • Rhythm symbols (see, e.g., https://nationalguitaracademy.com/guitar-tabs-for-beginners/). In this case, <tabGrp> elements with child <tabRhythm> elements must be used.
• Place attributes as 'high' as possible. E.g., on the <tabGrp> should go
  • @dur
  • @dot
  • @tie
  • @show.fret='true' @show.fret.enclose='paren'
• We must make sure that tied notes that are not repeated between parentheses in the tablature (which are usually found in the middle of the bar, and have the attribute @tie='m' or @tie='t') are not shown.
• There should be a default dirmark (default text for a technique) that is shown when @show.dirmark='true'. Candidates:
  • sl. for <gliss>
  • pick sl. for <gliss> with @pick-slide='true'
  • H or P for <slur> (hammer-on or pull-off)
  • T for <dir> with @technique='tap-fing' / @technique='tap-pick'
  • P.M. for <dir> with @technique='palm-muting'; followed by a dashed line in case of multiple notes
  • let ring for <dir> with @technique='let-ring'; followed by a dashed line in case of multiple notes
  • rake for <dir> with @technique='rake'
  • Fdbk. for <dir> with @technique='feedback'
  • w/ bar for <vibrato> with @technique='vibrato-arm'
  • Harm. for <dir> with @technique='natural-harmonic'; followed by a dashed line in case of multiple notes
  • A.H. for <dir> with @technique='artificial-harmonic'
  • T.H. for <dir> with @technique='tap-harmonic'
  • H.H. for <dir> with @technique='harp-harmonic'
• Bends are always indicated with text only rather than with @show.dirmark='true', as the @dis is indicated in semitones
  but the text may differ (additionally, at @dis='2', there is the option for either 'Full' or '1').
• Additional information can be added as text, e.g.:
  <dir technique='feedback' startid='#ID' show.dirmark='true'>Fdbk. pitch: D</dir>
• On using @show.dirmark='true' in combination with text:
  • @show.dirmark='true' in combination with text results in the default dirmark and the additional text.
  • @show.dirmark='false' in combination with text results in the provided text as dirmark. (TODO: Problem in van_halen-panama.mei at notes m4.n6-7: <pitchInflection> takes disposition as text, so what to do with the 'trem. bar' in the source?

Some lute tablature examples of mei contain duplicate xml:ids

A minor issue, but examples/lute/gerle-1552_1-no_4.mei and examples/lute/milano-1546_6-no_8.mei both contain duplicate xml:ids. xml:ids must be document unique. See https://www.w3.org/TR/xml-id/ and
https://music-encoding.org/guidelines/v4/content/shared.html#sharedCommonAttributes

gerle-1552_1-no_4.mei contains each of the following xml:ids twice: xml:id='m1.n3', xml:id='m1.n4', xml:id='m1.n5' and xml:id='m6.n6'.

milano-1546_6-no_8.mei contains each of the following xml:ids twice: xml:id='m3.n3' and xml:id='m5.n3'.

In all cases these duplications have occurred as alternatives under a <choice> element; nevertheless each xml:id must still be document unique.

Modelling questions and details guitar tablature

What?

This issue lists various questions and details regarding the modelling of guitar tablature that are still to be addressed.

Questions and details

• Instead of as a <slur>, we can also do single-note hammer-ons (see below) as a @technique attribute on <dir>: @technique='hammer-on'. But I like the current solution better (more in line with the single-note <gliss> approach).
• Slurs with slides in them (see satriani-ice_9, m. 1): do we encode this by nesting <gliss> in <slur>? One problem that I see here is that the 'H' dirmark that <slur> gives should be overridden with a 'sl.' dirmark that <gliss> gives.
• Sliding with LH hammering/RH taping finger (see satriani-not_of_this_earth, m. 2): this is a bit of an exceptional thing: all five notes are slid using the LH hammering (or RH tapping) finger. Wrapping <gliss> in <slur> (or <dir>) should reflect that. Is this done correctly?
• Syntax for <dir> when it encloses another element (see satriani-not_of_this_earth, m. 2, 4): where do we place the textual indication of the technique (here: 'T' for 'tap' and ‘Fdbk’ for 'feedback')? Right before the enclosed element?
• Wrapping an element in a <dir> element (see satriani-not_of_this_earth, m. 2, 4): is this OK or should it be the other way around? Or either way?

Place an image into the examples directory of anything we think we should cover (and add encodings later)

Implement right-hand fingering dots

Example of dots:

milano-1546_6-no_8.png

In German tablature, another system is used (little attachments to the rhythm flags) - we should include that too.

Some lute tablature examples of mei have startid attributes without the required # before the id.

A minor issue, but examples/lute/da_crema-1546_10-no_6.mei, examples/lute/gerle-1552_1-no_4.mei and examples/lute/milano-1546_6-no_8.mei all contain several startid attributes without the required # before the id. E.g. startid='m1.n3' should be startid='#m1.n3'.

See https://music-encoding.org/guidelines/v4/attribute-classes/att.startid.html att.startId takes a data.URI (rfc2396) as its value. So a # is needed before the id.

Implement ornamentation signs

Implement the various ornamentation signs found.

Implement (rendering of) German tablature

Some issues specific to German tablature:

• Beaming of smaller-value rhythm flags into what Newsidler in the introduction to his 1536 book calls Leiterlein (ladders). Although this feature is not unique to German tablature, it is done very consistently there (maybe even without exception? - I'm not sure). Marc Lewon noted, in another discussion, that they "group certain gestures in a suggestive way, which we should not lose in our transliteration."
• There are multiple ways to refer to the frets on the sixth course. Marc Lewon suggests to start with the four mentioned in Newsidler 1536, which he considers to be the most important ones:
  
• The relative vertical position of the symbols; we should have flexibility in placement. There are four options:
  • all the symbols are squished together as closely as possible so that they always align at the top (or bottom?) - see the Gerle example below;
  • the symbols are organised in polyphonic lines; this is more rare - see the Ochsenkun example below (note that some symbols are doubled as to indicate unisons);
  • a mix of both - see the Newsidler example below;
  • (in manuscripts) in cases where the scribe forgot a symbol, it is added below the others, leading to some sort of reversal/interruption of the 'logical' (i.e., from low to high) sequence of the symbols.

Examples:

gerle-1552_1-no_4.png

ochsenkun-1558_5-no_31.png

newsidler-1536_7-no_11.png

Clarify meaning of 'guitar'

In various places throughout the tablature MEI docs there is reference to 'guitar', almost exclusively referring to some generic 'modern' form of the instrument for which there is a large amount of tablature in 'internet' (ASCII) or proprietary formats. As acknowledged in the encoding details, this in turn may be an acoustic or an electric instrument. That's not the issue here.

There is also a large repertory of historical guitar tablatures, which have their own special features, including the use of 'alfabetto' notation, in which a single large capital letter is assigned to each of a set of chord-shapes rather than individual notes; sometimes this is combined with 'standard' tablature in a number of ways. These cannot be encoded in the same way as the 'guitar' tab referred to above.

So 'guitar' tablature is not standard for all time. Similarly, in MEI, the shorthand 'guitar' is not a sufficient specification of a tuning for all guitars (though as a default the modern 6-string guitar's e' b' g d a E is reasonable).

Implement vertical bars connecting symbols within chords

Examples:

morlaye-1552_4-no_7.jpg

narvaez-1538_1-no_18.jpg

Implement beaming of tablature rhythm symbols

Enable coloured tablature symbols

As seen in, e.g., the Capirola MS and some of the vihuela books.

Examples:
capirola-no_1.png

fuenllana-1554_3-no_31.png

Implement left-hand fingering (?)

This is not something one would find in the tablature, but something I, as a player, would like to be able to add. Not a priority now.

Modelling questions and details general

What?

This issue lists various questions and details regarding the modelling of any kind of tablature tablature that are still to be addressed.

Questions and details

• What is our policy on xml:ids? Empty IDs (xml:id='') are not valid XML (see here: an xml:id must have a value, an NCName, and be document unique).
  • We place them everywhere by default, adding an ID everywhere.
  • We only place them when the thing it identifies is acually referred to, and leave them out completely otherwise.
  • What will an ID look like? Some random character string, or something with more meaning, which makes things easier to interpret for people going through the encoding (e.g., I have used xml:id='m1.n.3' for the third note in the first bar).
• When modelling, I find it difficult to decide when to use an element ("if it feels like data" is an argument I've found) or when to use an attribute. In general, we should follow the practice from the MEI Guidelines as much as possible.
  • RdV: It seems to make sense to use
    • An element if the technique is something that may affect multiple <tabGrp>s in a row, e.g., let ring (cf. <beam>).
    • An element if the technique is something that affects all notes inside a single <tabGrp>, e.g., palm muting.
    • An attribute if the technique is something that affects only a single note.
  • DL: It would be good for MEI to have some guidelines for expansion writers for this. I suspect that Perry, Johannes and Laurent have other criteria. I'd add
    • Use an element if information is multidimensional.
    • Use a generic element with types if the alternative is a million new attributes.
    • Prefer subclassing something that already exists, so if pull-off looks like a slur, maybe subclass <slur>.

Enable rendering of French tablature

Example:

phalese-1546_20-no_2.png

Transfer information from online documents to the repository

What?

This issue lists the various online documents (meeting notes, technical notes, ideas, etc.) that we have created over the last years. All the information in these documents should be transferred to one central place -- the Tablature IG GitHub repository. This process is ongoing; the status ('ongoing' or 'done') for each of the documents is given below.

Once the contents of all these documents have been transferred to the Tablature IG GitHub repository -- i.e., once all have been given status 'done' -- this issue can be marked resolved (and the online documents deleted?).

Links

Meeting notes

• Tablature SIG meeting Tours 2017. Meeting notes from MEC 2017 meeting (Tours), May 2017.
  • Status: ongoing.
• MEIOX Tablature group. Meeting notes from initial meeting (Oxford), October (?) 2018.
  • Status: ongoing.
• Tablature SIG. Meeting notes from MEC 2019 meeting (Vienna), June 2019.
  • Status: ongoing.
• MEI Tablature IG agenda and notes. Meeting notes from MEC 2020 meeting (online), May 2020.
  • Status: ongoing.
• Tablature IG meeting. Meeting notes from MEC 2021 meeting (online), July 2021.
  • Status: ongoing.

Other

• mei-tmus. The previous repository. Although in principle this repository is deprecated, it must be checked whether it still contains information that is missing in the current repository, or that the current repository could benefit from.
  • Status: ongoing.
• TabMEI guitar techniques. Document started in the summer of 2019, attempting to list as many as possible guitar techniques.
  • Status: done.
• TabMEI progress notes. A list of things written down while encoding or thinking about MEI modelling in general.
  • Status: done.
• SMuFL for Tablature. A list of SMuFL-related issues.
  • Status: ongoing.

Staves with different numbers of lines / placement of symbols w.r.t. staff lines

Staves can have fewer (or more?) than six lines.

Example:

phalese-1547_7-no_21.png

Implement tenuto signs

Examples of star-like signs:

gerle-1552_1-no_4.png

rotta-1546_15-no_23.png

Examples of line-like signs:

vallet-1615-een_en_twintich-no_1.jpg

vallet-1615-secretum_musarum-no_1.jpg