For an HTML version check out the Manual under Software Projects.

 

MidGenusageexamplespackage/module structuretop level modulesDevicesSequencesinternal smf data structureMicro-SequencerTimestamps and DurationsDuration alignmentVisual beat/bar separatorEvents and event typesNote eventsabsolute note eventsrelative note eventsFlat (b) and Sharp (#) symbolsRestsrepetitions, sub-sequences and loopsrepeat operatorcontinue operatorsubsequence repetitionsadditional timestamp directivesPattern sequence templatesadditional note attribute datanote on velocitynote off velocitycontroller-type attribute dataStep-Sequencer (pre-processor)portamentotutorial pages

MidGen

MidGen is a Perl based standard midi file reader/processor/generator. It includes Perl modules and packages to create/read/write and process standard midi (smf) files based on Perl input scripts. Essentially this tool provides a framework, which is specifically designed for experiments with musical pattern, arpeggios, accompaniment styles, arrangements or entire scores. However MidGen is not only limited to musical data processing since the framework provides furthermore access to all smf event types such as Channel-/SysEx-/Escape- and Meta-events. In general there is no limitation since everything representable by smf can be generated, processed and provided by MidGen.

For musical data, a build-in "micro sequencer" function translates text-based micro-sequences into smf midi events. Since everything runs in perl environment, you can take advantage from programming languages like using variables for micro-sequences, sequence substitutions, storing arrangement parts in sub-procedures, repeat sequence iterations with loops, etc. together with convenient smf input/output functionality. This approach allows writing music in traditional sequenced format in combination with algorithmic or procedural programming functionalities. In addition, there are many functions for SMF data manipulation implemented such as copy/paste/insert/transpose which are working either on track level or across the entire arrangement. Also you can include and merge additional external smf midi data into your arrangement so that for instance live played parts get merged with generated accompaniment pattern. Essentially there is no limit for creativity since its an open framework were you can easily add features, functions, procedures or other extensions with your own ideas.

 

Example smf output arrangement in score representation:

 

The score was entirely generated from the example source code below using micro sequences:

 

To compile the midi output file, just run:

 

The output is typically a type 1 smf midi file saved in your current working directory from where the script gets executed. In addition, the console output provides an overview and displays general song- and track-information of your arrangement. For more detailed result debug, individual event lists per track are written as regular text files along with the smf output.

 

usage

Once you have installed perl, just run:

If there is no project-file specified, the default project (Projects\Current.pl) will get processed.

To get started quickly and to see how the output looks like, you can also just process a smf midi file thru the program or try one of the provided project examples. MidGen was tested with ActiveState Perl, Strawberry Perl portable and various Linux/Ubuntu Perl versions.

Example: read/write a smf midi file:

examples

The easiest way to explore MidGen in order to see how it works and what it can do, is to process some of the example files. Since some of them are made specifically for playsmf containing loops and arpeggios, it would be helpful to play the output smf using this tool.

 

package/module structure

top level modules

and much more.

Devices

The Devices sub-folder contains mainly device specific packages and modules.

Sequences

This sub-folder was originally meant to store a sequence library for musical pattern like bass lines, drum pattern, styles, etc., however it evolved over time to a library of tools, helper functions and other miscellaneous perl packages.

internal smf data structure

Internally the smf is nothing else than a multidimensional hash structure with integer key values across all hierarchy levels. The top level key represents the track number (except "track" -1 which is used for smf specific information like filename, smf type, PPQ, etc.), the 2nd level key represents the eventtime in ticks based on the smf PPQ setting and the 3rd level key is the event ID representing the event order within a given tick. This allows simple access and iterations across all smf events for easy data manipulation.

Reading and writing a midi file is as simple as:

In addition, you can also display general track overview information on the console output screen plus getting individual event lists per track by calling MidiDebug::PrintInfo() such as:

Since MidGen has already defined a default smf datastructure handle called %main::out runnig thru MidiDebug::PrintInfo() and MIDI::Write() functions at the very end of the program flow, you dont need necessarily taking care about those functions unless you work with multiple smf structures in parallel. Therefore its also possible to read and write a smf by just running the following instruction:

Make sure that you provide a new filename (smf "track" -1, argument 0), else the original smf will be overwritten.

track event list view output

As mentioned, each individual track provides additional event lists within separate text (.txt) files which are written in parallel to the smf output. Those lists might be helpful for debugging or just to get an event overview by track. The following information is organized in individual columns:

Example track lists:

special "track" -1

As already mentioned, there is one specific "track" (-1) holding generic smf information by 2nd level key values such as:

timestamps and durations

Timestamps and note- or continuous controller-durations for functions such as (insert, micro-sequencer, copy, etc.) are typically provided in whole notes as floating point values to decouple them from timesignatures. Internally all times are stored in ticks based on the smf PPQ setting. Therefore its important to define the required resolution in the beginning of a project.

supported event types

MidGen supports all types of midi events including channel-, SysEx-, Meta- and Escape-messages. NoteOn-Off pairing is done internally while reading smf files. That means all internal operations refer to Note- rather than Note-On/Off events unless they are inserted individually on purpose. Therefore each note is represented as a single event with a given duration and a optional note-off velocity.

Micro-Sequencer

The micro-sequencer is a central and important function especially written for convenient note- , controller- and other data insertions. Essentially it inserts a sequence of notes and/or controller/meta/sysex events into a specified track, based on a given start-time, scale (e.g. chromatic, major, minor, etc.) and base note (e.g. 60 - middle C). The sequence is provided as a consecutive, white space separated list of <duration>:<data> pairs representing duration timestamps with associated event data in text format separated by the colon : sympol. The micro-sequencer keeps internally track of timestamps, note-pitches and other data and typically advances in time with each provided event-duration value unless otherwise specified (e.g. for overlap notes or chords). In addition, the sequencer returns the total duration of the provided micro-sequence in whole note units. This allows the caller thread keeping track about the total timestamp if you simply sum up all micro-sequence durations in case they belong logically together. The micro sequencer function is called from Edit package as follow:

Actually there are much more arguments available, but those are the most important ones to start with.

Example - simple micro sequence:

The sequence above is mostly self explanatory.

Timestamps and Durations

Timestamps and event-durations are typically provided in whole note units either as integer or floating point numbers or as equations such as 1/4 or 1/4+1/8, 1/4+1/8+1/16 etc. To shortcut dotted note equations, you can just append tailing + signs to the duration value. For instance a dotted quarter can be written either as 1/4+1/8 or alternatively as 1/4+. Double or triple dotted notes are written respectively with tailing ++ or +++ signs. Similarly you can shorten a note by its half length with tailing - signs. The example below shows few different variants of duration timestamps.

Note: If there is no timestamp provided, the sequencer automatically applies the latest valid timestamp value.

Duration alignment

To avoid complex arithmetic equations and to keep sequences more readable, it is possible to align event durations to given timestamp grid boundaries by using the alignment operator | in front of the duration value. This advices the sequencer to proceed in time until the next grid boundary timestamp is reached. So in the example above, the |1/1:% event will simply insert a rest in alignment with the next whole note timestamp value. In this case it proceeds to the next bar since the time signature is 4/4 (whole note grid boundary).

Visual beat/bar separator

In order to keep the sequence more structured and readable, it is possible to insert additional | character symbols to visualize bar, beat or other timestamp separations. Semantically they dont have any meaning and are just ignored like comments or remarks by the sequencer as long as they stay in clear separation from events.

Events and event types

The microsequencer supports different smf event types such as

Events are always paired with duration timestamps even though some event types do not really require a duration such as marker or lyric events. In this case a zero duration can be used if the sequencer should not advance in time. Events can also get paired to insert multiple event types at the same time with the same duration in one sequencer event. For instance you can insert a note and in parallel a continous controller using the same timestamp and duration.

Note events

absolute note events

Note events are typically provided as numerical values rather than traditional musical symbols. This decouples them from scales and tonal systems and keeps the flexibility for additional arithmetic operations. They can get specified either in absolut- or relative (interval) values by preceding ^ (up) or v (down) symbols. Absolute values can be either positive or negative in case they refer to notes below the given basenote. Notes will follow the given scale and key provided by micro sequencer arguments <scale> and <basenote>. In result, note zero is basically the basenote.

Example - Two consecutive micro sequences with additional marker M<text> events and bar separators:

The example above just demonstrates how multiple micro sequences can get concatenated using a global timestamp variable.

relative note events

Since the micro sequencer keeps internally track about note values, it is possible to use relative note intervals in addition to absolute ones. Relative notes are determined by preceding up ^ or down v symbols in front of the note values. If there is no note number specified, the sequencer assumes just one relative note step.

Flat (b) and Sharp (#) symbols

Note values can be increased or decreased in semitones by putting either flat and/or sharp symbols after the note value. Traditional music puts them in front of the note, but here it is required to put them after the note value as additional attributes. The sequencer allows having multiple consecutive and/or mixed symbols by simply summing up all flats and sharps to get the final value.

Example - C major with few additional flats and sharps:

Rests

rests and pauses are simply represented by the % character symbol.

repetitions, sub-sequences and loops

repeat operator

To repeat single events or rests with the same duration and/or note values, you can just use the . (dot) symbol for either duration and/or event data values. The following scenarios are possible:

continue operator

The continue event operator > is similar to regular repeats, but ignores rest events and simply 'continues' playing with the latest note value. This event type is mainly used in combination with pattern sequence templates.

subsequence repetitions

In addition, the micro-sequencer comes with several build-in loop and sub-sequence functions for sequence repetitions. Essentially you can enclose sequences or parts of sequences within parentheses or braces n{<sub-sequence>} and put a repetition number n in front of them. If no repetition number is given, the sequencer assumes one insertion w/o repetition, otherwise the sequencer will insert the enclosed sub-sequence(s) n times. Since the sequencer allows the insertion of recursively nested sub-sequences, repetition pattern can quickly get large and complex by just a few instructions. The example below shows a small input sequence with two nested sub-sequences.

Finally, different types of parentheses have different meanings:

Since the sequencer keeps track of timestamps, durations and notes, you can preserve the previous duration/note and timestamp values when a sub-sequence get entered. In result, the sequencer can restore those values when returning to the main-sequence. This is especially important when building pattern templates using relative note values.

additional timestamp directives

Usually the sequencer takes care about timestamp values and advances in time with each provided event duration. This function is typically usefull for monotonic lines like lead melodies, bass lines, etc. with non-overlapping notes. However in many cases you need note or event overlaps such as in chords or many other situations. Therefore the sequencer provides additional timestamp directives by having additional < symbols in front and/or after the given timestamp value.

The examples above show 4 different scenarios:

The 1st one is a regular insert of a quarter note where the sequencer advances in time with the event duration. Therefore the total sequence length is effectively one quarter note.

The 2nd scenario shows that the return marker merges with the entry marker because the sequencer doesnt advance in time when the quarter note gets inserted. In result, the total sequence length is effectively zero although a note was inserted.

The 3rd scenario demonstrates how the sequencer goes back in time before the note gets inserted. Therefore the note appears earlier in time as the sequence entry point. Since the sequencer still advances in time with the inserted event, in result the return marker merges again with the entry marker and the total effective sequence length is zero.

The 4th scenario demonstrates the combination of both previous scenarios. The sequencer goes back in time before the event gets inserted, advances in time with the event and finally goes back in time again after the event insertion. In result, the note and the return marker appear earlier in time than the entry marker. Effectively the sequencer is running backward and returning a negative sequence length in this case.

Finally timestamp directives can be used to write overlap notes, chords or sub-sequence overhangs at the beginning or ending of an sequence. The example below shows a simple chord triad with 3 stacked notes. In this case only the last note get advanced in time while the 1st and 2nd notes doesnt advance. If there is no timestamp value provided, the given timestamp directive is still valid and the sequencer will not advance after the event insertion.

micro-sequence overlaps (overhang) Another example below shows how to setup sequence overlaps or overhangs using timestamp directives. In some cases it is required having extra events or overlap events either before the sequence actually starts or after the sequence has been finished. This can easily get achived by timestamp directives going back in time before the sequence starts or after the sequence has been finished. The example below shows two concatenated micro-sequences with additional overhang events on each sequence side.

Pattern sequence templates

Pattern sequence templates are typically sub- or partial sequences refering exclusively to relative notes based on the sequencers internal states. This decouples them from absolute note values and keeps the templates more generic. Usually they are assigned to perl variables for further usage in higher level sequence strings.

This allows for instance building different chord-type templates or small arpeggio pattern templates without attaching them to absolute note values for later instanciation within sequences.

The example above shows how a simple chord triad is setup as a template and beeing used in a small sequence.

Remark: Some strange looking events like 1/1<:0_% are so called NOP (no operation) events since they do not advance in time neither inserting a note since a rest is applied in combination with a note value, however they instruct the sequencer to set the current timestamp and the current note value for further sequence events.

The following example is very similar and shows how to insert a small arpeggio sequence using templates.

Now one template example demonstrating how to use subsequences with and without sequencer value restoration. Actually when we simply concatenate a sub-sequence template such as " > ^2 ^2 . " multiple times, the sequencer will increase continously all note values with each insertion. In contrast, if we put the sub-sequence into parentheses, the sequencer will take care about preserving and restoring note values when the template gets entered or left. Therefore the resulting sequence is totally different.

The next example shows the effect of additional square brackets around the template sequence and their effect. In this case not only note values, but also timestamp values get preserved when the subsequence get entered. In result, the sequencer jumps back in time once the sub-sequence template has been processed and starts over again from the beginning. Therefore the sequence appears now stacked rather than consecutive in time.

additional note attribute data

Each note- or pause-event of the micro sequencer supports additional (optional) attributes such as on/off velocity values and/or attached controller data events or event series. This way you can easily attach additional articulation attributes to each individual note in alignment with note-on and duration timestamps. Note attributes are typically appended by undersscore _ separations. For example " 1/4:0_.75_r.25 " inserts a quarter note with note value 0 (relative to scale and base note), NoteOn velocity 0.75 and release velocity 0.25.

note on velocity

Note-On velocities are simply written as normalized (0.0 ... 1.0) floating point values in adition to note events. If a note event is not specified, the sequencer will repeat the previous note with the provided velocity values. This can be useful to shortcut for instance percussion sequences repeating the same note with different velocities.

note off velocity

Note-Off velocities are specified by r<velocity> floating point values. They are rarely used and most sequencers doesnt really display them, but here an Cubase list edit example showing both Note-On and -Off velocities together.

controller-type attribute data

Controller-type attribute data are either additional single events inserted together with NoteOn or rest events at the current timestamp position in sequence or continous event series inserted along the given note duration. They are appended to note or rest events with additional arguments by one of the following syntax formats.

Single event controller (e.g. foot, switch, pedals, etc) are provided by either two or three additional arguments like:

<duration>:<event>[_C<Controller#>_<Value>]*

<duration>:<event>[_C<Controller#>_<Centered>_<Value>]*

For continous controller data series, the following format is used:

<duration>:<event>[_C<Controller#>_<Function>_<Centered>_<Value>_<Value1>[_<align>]]*

Controller type data can be one of the following event types specified by the Controller# argument:

The Function argument specifies one of the following data sweep shapes:

Remark: If the function argument is provided as a negative number, the data sweep will be inverted (up<--->down).

The Centered argument determines if the Value arguments are provided in centered (signed) or uncentered (unsigned) format:

The centered format makes especially sense for centered controller types such as pan, balance, pitch-bend, etc. while other controllers such as volume, expression, etc. make more sense in uncentered format representation - although there is no restriction for one or the other format type.

The optional aligned argument determines if controller data series get inserted continously (default) or in alignement with Note-On events. This feature can be used to reduce the amount of data in case controller changes are only required when notes get triggered.

Example - switch controller: attach additional sustain pedal press/release events to 1st and last note in sequence

Example - continous controller: sweep MIDI controller 7 (volume) using full swing sinusoid function along the given note

Below a few more examples inserting whole note rests along with marker text and midi controller 11 (0xb hex) demonstrating different swing and transition types.

Example - alignment: Insert 16 x 1/16th notes, go back in time and insert controller 11 (0xb) in alignment with notes, but 1/64th ahead of note events.

Example - tempo adjustment: Insert initial tempo and sweep +/- 25%

Step-Sequencer (pre-processor)

In general the Micro-Sequencer contains an embedded Step-Sequencer which kicks in if a sequence text string is enclosed within | bar character symbols. In this case the enclosed step sequences get internally converted to micro-sequences before the micro sequencer compiles the smf similar to a pre-processor. Step-sequences are basically text strings were each single character or symbol refers to an individual micro-sequence. Those individual sequences can be either quite simple just containing single notes or more complex containing chords, guitar strums or other pattern types.

A typical use case for step sequences are for instance drum patterns since they usually run on a regular fixed time base with static key->voice assignments. In this case a time base (or step granularity resolution) and a note definition is required before the step sequence is applied. This is simply done with a dummy rest event which does not advance in time (see example below).

Example: insert few micro-step-sequences with 1/16 resolution across multiple drum voices

Lets take a closer look at the example above: The initial dummy event 1/16<:35_% is just a rest event without advancing in time, but defines the resolution and the absolute key value based on chromatic scale with basenote=0. The remaining characters are all enclosed within bars very left and very right belonging to the step-sequence.

Remark: Additional bar | characters within a step sequence are just visual separator symbols for better readability without any semantic meaning. Internally they get removed before parsing the sequence.

By default, the step sequence characters have the following interpretation:

Example: demonstrating different velocities and stretching notes.

portamento

The Edit::Portamento() function emulates portamento functionality by collapsing monotonic melody tracks into single note pitch tracks with fixed keys while the pitches are provided by pitch bend events.

<Pitch-Bend-Sensitivity> = Edit::Portamento(<smf-hash-pointer>, <track>, <portamento-time>);

The function returns the required pitch bend sensitivity which must be inserted in the very beginning of the track, else the sequence will sound off.

eof

tutorial pages