High numbered keyswitches retuning

BETA SOFTWARE - WORK IN PROGRESS - comments invited.

Do you just want to get the script and add it to your Kontakt instrument? If so, skip to.

Introduction
This script is based on a totally unofficial spec. for retuning instruments via midi, developed by a group of microtonalists in the Xenharmonic Alliance facebook group. I wrote it to try out our ideas for Kontakt instruments.

We developed it as an alternative to the official midi tuning sysexes. Nowadays sysexes can't be used at all in many environments. For instance there is no way to access them in a Kontakt script. Ideally we want a spec that let's instrument designers and synth programmers add real time tuning table support in any environment, or the widest range of possible environments. For more about the motivation, see Tuning RPNs (specification draft).

What this script does:
 * lets you set up a tuning table on a Kontakt instrument via midi using "velocity keyswitches". It sends a rapid sequence of notes 126 and 127, with the tuning instructions and data coded into the note velocities. These notes are well above the normal playing range of most midi instruments.
 * Individual midi notes in the tuning table can be tuned to any pitch in the midi note range.
 * Velocity keyswitch retuning is switched on by sending the undefined controller 119 with data 119. It is switched off by sending Controller 119 with data 0.

This script can be added to many instruments in other formats so long as you convert them to the Kontakt format first.


 * Most sound fonts and .gig files can be imported directly into Kontakt.
 * Many other sample based instruments can be converted to Kontakt format e.g. using Chicken Systems universal translator - except of course for encrypted or locked instruments (e.g. the libraries of the Aria player are encrypted and can't be imported).
 * You can add the script to any of these instruments

You do need the paid for Kontakt rather than the free Kontakt player or the Kontakt demo to add a script to a Kontakt instrument.

This script works with many Kontakt instruments, but not all. Sometimes you need to disable existing scripts, and sometimes there are no free slots to use. For more about this see

Other elements of the user interface
This script also:


 * adds a series of knobs and other controls which give feedback about the tuning of the note as you play notes, and you can also use these to adjust the tuning of any of the notes of your instrument by hand.
 * adds a short list of example scales which you can access in the user interface.

Those extra controls are visible when you switch to the script editor. You can also make them the "performance view" in which case they appear on the front panel of the instrument.

How you can use this script
One way you can use this script is to insert these keyswitches into a score, just like the low numbered keyswitches used to change instrument articulation. The notes 126 and 127 are high enough so they are never needed in a normal score, well above the pitch range of a piano.

These instructions can also be sent to the instrument via midi in real time by retuning software - I have added the capability to Bounce Metronome for the next release and we expect other tuning software to support this idea in the future (or some version of it as the spec is still under discussion).

The instructions could also be sent on a "per key" basis for microtonal keyboards. The 0 to 122 note restriction would only be a restriction on the number of notes in a single chord per channel if the note numbers are dynamically assigned to keys. This could permit "monster keyboards" with hundreds of notes all playing midi notes in any desired tuning of the keys, on a single midi channel.

This script works for almost any instrument which can be played in Kontakt. Many other sample based instruments can be converted to Kontakt format - and are often available in it already - this means it can also be used for most sample based instruments. That includes all instruments in the soundfont sf2 format and in the .gig file format - this script can be used with any of those instruments. Some sample based instruments however such as Garritan Personal Orchestra and Vienna Symphonic Library are not available as Kontakt instruments. Other methods for adding microtuning capabilities would need to be explored for these, also for VSTi and software or hardware synthesizers.

Contributors
For the draft of the spec see Tuning RPNs (specification draft).

The main contributors were myself (Robert Walker), Kite Giedraitis, Rob Fielding, and Graham Breed, and Ozan Yarman came up with the idea of using high numbered keyswitches.

It's been through various stages. The original spec used undefined RPNs for retuning, and undefined controllers were also explored.

This script uses high numbered velocity keyswitches, because I couldn't get the tuning RPNs or controllers methods to work in Kontakt. This may be a good solution for other environments as well. Sysexes can't be used so the MTS sysexes are not an option in this particular environment. Comments on all this by synth authors and developers are welcome!

I developed a script for Kontakt instruments to explore our ideas. I tried various methods but only the velocity tuning keyswitches worked with Kontakt, at least on my computer. The other ideas using controllers and RPNs are included in the beta script for comment and experimentation.

This script will only work in Kontakt, not in VSL, GPO, etc
This script will only work in Kontakt. Other sample players won't understand it, and it is impossible to do the same thing via midi instructions or VSTi plugins or effects. VSTi for instance would need to be reprogrammed to understand these instructions.

This script won't work for instruments that are commercially encrypted and use their own sound engine such as Garritan Personal Orchestra or Vienna Symphonic Library. (It will work with the VSL instruments included in Kontakt).

The sound engines for these other instruments would need to have extra programming to respond to these high numbered velocity keyswitches.

Similarly this script can't be used for VSTi or other synths. Other methods would need to be explored to add this capability in other environments.

Why Kontakt only?
It needs the Kontakt scripting engine to work. AFAIK no other program can interpret this language. So this script will only work in Kontakt.

The script adds a tuning table to the instrument, and then changes the way the instrument responds to midi messages so that the velocity keyswitches can alter the pitches in the table.

It is not possible to do any of this with a conventional midi plug in, as it has no access to any internal tuning tables in the instrument.

Programmers in other environments would have to write their own code to do this.

For more about this see Why Kontakt only?

Alternatives
You can do a fair amount with pitch bend retuning. This uses "instant pitch bends" played before the note, and remapping notes to different channels. It can be surprisingly effective, and this is what most of microtonal retuning programs usually do.

However the result is that you need many more channels than you need for a normal midi piece. If you have different controllers, stereo pan positions, polyphonic instruments etc then you can easily run out of channels, especially with e.g. full orchestral work. It is also inconvenient because you have to load the instrument multiple times in different channels.

Why doesn't this script use the standard Tuning sysexes?
It can't use the standard tuning sysexes because there is no support for sysexes in the Kontakt scripting language.

Early ideas - controllers and RPNs
Our first idea was to use RPNs. See the Tuning RPNs (specification draft) for details. However I ran into issues implementing it as a Kontakt script. It only got about 12 of the RPN instructions out of a batch of 512.

Another idea was to use controllers. This idea seemed promising at first. But I ran into the same issue. If you send hundreds of controller messages, all for the same four controllers, in a short period of time, the script only gets notification of the last value of each of the controllers. That is understandable if you think of the controller messages as updating the value so the script only needs to know the latest value of the controller at any one time.

It was possible to get the controllers to work, but only if my program sent them at intervals of about 10 ms between each message, or about 40 ms between each occurrence of the same controller. Send them any more quickly than that and some got lost.

So, I was completely at a loss about how to proceed. This may just be a Kontakt issue, but it seems to me it might also be an issue with other synths as well. With controllers then generally your program wants to know the current value at any time and is less interested in the sequence of messages that lead to that value, so it is at least reasonable for a host or engine to discard controller messages if there is too much traffic.

The high numbered velocity keyswitches idea
This is the brilliant suggestion of Dr. Ozan Yarman of Başkent University. The topmost notes of the midi range are rarely used in instruments - are well beyond the top notes of a piano. Notes also are high priority so less likely to be dropped than controllers.

Keyswitches are often used to change playing styles of instruments, but typically they use notes below the natural range of the instrument to do this. The high notes are normally not used for anything.

After several revisions of this idea the latest version uses the note on velocities of the note numbers 126 and 127.

Note 127 is used to send "instructions" so 127 with velocity 4 sends the instruction "retune a single note with the next 4 data bytes". This is a similar idea to the midi "status bytes".

Then 126 is used for "data", for instructions that need data. So in the case of the instruction 127,4, then it will be followed by four more notes played on note 126 with the velocities used for data.

There is a minor problem here that you can't send a zero value as a velocity. To deal with that then the instruction 127,127 is treated as data value 0.

So we get four 7 bit bytes, to set the input note, the output note to retune it to, and the coarse and fine pitch bend. The pitch bend sets an offset from the output note in the range -50 cents to 50 cents. (Always the same range -50 to 50 cents no matter what range you have for pitch bends).

For details see

Sketch of how it works
This script monitors the note on event. It records all the note on velocities for notes 126 and 127, and uses information encoded into the note on velocites of the notes to build a big tuning table in the internal memory of the instrument.

Then when you play ordinary notes it then looks up the note in that table to find out how to retune it. Then, depending on the pitch, it may choose a different sample from the one that would normally be played by that input note, and it then adjusts its tuning further by up to 50 cents up or down.

How to add this script to a Kontakt instrument
Note, this is a BETA script - particularly since the spec itself is work in progress.

You need the full version of Kontakt to install this script. You can't use this with the free Kontakt Player as it doesn't have script editing capabilities (except briefly in demo mode).

Authors of instruments for Kontakt will also be able to add this script to their instruments. The license permits it to be used in this way, in any project, free or commercial.

You can get the script here: kontakt_high_keyswitches_tuning_no_release_retuning.txt

Here it is again, with syntax colouring for programmers: Kontakt retuning script

To add this script to a Kontakt instrument:


 * 1) click on the wrench icon
 * 2) then on Script Editor,
 * 3) then look for an empty slot labelled Click on tab to select it and
 * 4) then click Edit.
 * 5) Then copy and paste my script into the edit field. You may need to use Apply From Clipboard to achieve this.
 * 6) Then click Apply (button to far right)

The title of the script gets filled in automatically, the button and knobs will appear, and your instrument can now be micro-tuned in Kontakt.



You will be able to retune the input notes straight away using the knobs. The tuning is persistent, so you can edit the tuning by hand and save the tuning, and it will remain tuned like that next time you use the instrument.

How to use the controls to retune individual notes by hand
This script adds various controls to the user interface for your instrument.

To retune any note on the keyboard to anywhere in the midi range, you


 * 1) Use the "Note" knob to select the note you want to retune
 * 2) Use the "To" knob to select the note to retune it to.
 * 3) Set the amount of the cents bend with the Cents, Centicent and Last digit knobs
 * 4) Alternatively, double click on the text field below the Cents knob and enter the amount of the bend as text.

How to retune the instrument via midi
You can also insert the necessary velocity keyswitches into a score by hand and then play them via midi to retune the instrument.

The next update of Bounce Metronome when ready will be able to send high numbered keyswitches to retune these instruments via midi, so e.g. you just need to select a SCALA scale from a drop list to retune all the notes to that scale.

Some other programmers of retuning software will probably do the same.

NOTE - BETA, WORK IN PROGRESS

Test midi files
This test file retunes all 128 notes of the instrument to 120-et using the velocity tuning keyswitches 126 and 127.

It then plays a 128 note glissando in 120-et. Added an example .wav recording so you can hear what it sounds like.

120-et-complete-with-128-note-glissando.mid (wav)

It retunes all 128 notes, adds a pause of 50 ms before switching off the velocity retuning method, and then plays the notes with a glissando up and then down.

Here it is without any notes played, just updates the tuning: 120-et-complete.mid

This version retunes the notes 0 to 125 - and so doesn't need any pause afterwards.

120-et-to-125-glissando.mid (wav)

Here it is without any notes played, just updates the tuning: 120-et-to-125.mid

Both of them also have velocity keyswitches to update the tuning description as well.

Minimal script for programmers
This is a minimal version of the script. It removes all the user controls except the "reset to 12-et" button. It removes all diagnostics and it removes all the retuning methods except for the keyswitches using notes 126 and 127, and removes the release sample retuning (not normally necessary as most instruments tune them correctly when you change the tuning of the note).

The result is less than 300 lines of code.

This is easier to read, so may be useful for programmers who want to see how it works.

It may also be useful if you want to add the script to your instrument but have no spare slots and so need to merge it with your existing code.

kontakt_high_keyswitches_tuning_minimal.txt or with syntax colouring kontakt_high_keyswitches_tuning_minimal.htm

As you see, then most of the work is done in the function process_tuning_keyswitches plus a small amount of code needs to be added to on note and on release, some variables need to be added to on init and you probably will want to add a "reset to 12-et" ui control, the script handles this in on ui_controle ($ResetTo12EQ)

Free script slot issues
Some instruments have no free slots for new scripts. Many of the Kontakt pianos are like this for instance. If so, AFAIK, the only way to use this script is to contact the original author of the instrument and see if they can modify it.

Assuming you do have free slots, it is probably best to install this into the first available script slot to prevent interference with other scripts. Unfortunately often the slots are locked or password protected.

As a result often you are forced to install this script after the other ones. You may find that it only works if you disable other scripts. Try disabling each in turn until you find out which one causes the problem. Then you might find you can copy / paste those problematical scripts into a later slot than the one used for this script.

If the script slots are not protected, or you are the original designer of the instrument, you might be able to merge my script with the script already there. This would need some knowledge of how the scripts work.

The minimum you need to add is the script in the previous section,. This will permit velocity retuning via midi and adds a 12 equal reset button, but no other user interface controls.

Out of tune release sample issue
Most instruments work just fine with the default script. However, you might find that the note changes pitch at the note off, when it triggers the "release sample". If that happens, try this script, which disables the old release sample, and triggers its own new release sample, tuned to match the output note + pitch bend:

kontakt_high_keyswitches_tuning_with_release_retuning.txt

It is the same script as before, just changed declare $retune_release:=0 to declare $retune_release:=1

It is a bit of a kludge because the only way you can retune these release samples in Kontakt is to DISABLE ALL AUTOMATIC RELEASE SAMPLES and retrigger a new release in the script when you get notification that the note needs to be released.

Luckily most instruments don't need this tweak. If you do need it, it might work fine, but is not guaranteed to work.

In some situations you get interference of this script with the other scripts, such as an extra "double" note at the note release, which never switches off.

This release tuning work around probably wouldn't be needed if the script could be put into the first slot. However, the slots are often locked with existing scripts, so you can't do that.

This method doesn't put any restrictions on the output frequencies of the instruments
Suppose that your midi instrument is able to play midi note 127, then you can set any note you like to play midi note 127, e.g. you could set note 60 to play 127. Usually note 127 is too high in pitch to be of any interest, but you also get other "unorthodox" ways of using the midi range such as repeats of the same instrument with different articulations etc. So note 127 might be needed, if so no problem.

Monster keyboard instruments with hundreds or thousands of keys, using this keyswitch method
Some keyboards available or under development have huge numbers of keys such as the Tonal Plexus Keyboards by Hi Pi Instruments, with up to 1536 keys.

Other instruments with more than 123 keys include Terpestra keyboard with 280 keys with plans to make it microtunable in its kickstarter project, Starlabs Microzone U-648 with 288 keys, their Ztar with 144 keys plus 6 note-assignable triggers, the four quadrant Lambdoma keyboard with 256 keys, and the Opal sonome with 192 keys.

Usually these work by splitting the notes across several channels. The TPX variety of Tonal Plexus uses 105 and 106 notes of each MIDI channel in an octave. The Lambdoma keyboard splits over 4 channels each with 64 notes.

However, the [https://www.facebook.com/photo.php?fbid=589357484447453&set=pb.414318895284647.-2207520000.1380210065. Megaplex, only available by special order], uses two channels per octave, and 128 notes per channel, and has 1536 keys in the six octave version.

You might wonder if this keyswitch method would let you use large numbers of keys in a single channel, either for the Megaplex or any other keyboards that might be built using similar ideas, where the keyboard has keys that send midi note numbers 123 to 127.

It would depend how they are used.

Keyboard connected directly to the velocity keyswitches retunable synth via midi
If the idea is to connect the keyboard directly to the high numbered velocity retunable synth - then indeed you can't use the midi notes 123 to 127 (or whichever notes are set aside for velocity keyswitch retuning).

With this way of using the keyboard, then you would set up the tuning with some other software application, and simply connect your keyboard to the synth to play it. The high note numbers on your keyboard would change the tuning on the synth, so you would need to avoid using those.

Keyboard connected to the velocity keyswitches retunable synth via retuning software
If the idea is to send the notes of your keyboard through retuning software however, then all 128 keys can be used so long as the software can do dynamical re-assignment of the notes.

For a 128 key keyboard, the notes 0 to 122 would doubtless be preset to play the same numbered output notes via the software. But when you press e.g. key 123, then the software will find a key that hasn't been used recently, and use that to play the pitch. So you would dynamically re-assign the tuning the synth as the notes are played.

To take an example, suppose that 100 is the note that hasn't been used recently and 67.02 (decimal midi notation) is the desired pitch

This might sound a bit complex, but the programming is straightforward, and a program can easily keep track of things like that.

In Tune Smithy I already do dynamical reassignment of pitches to input note numbers with MTS sysexes. It is easy for the fractal tunes to need more than 128 distinct pitches per part. Also, if you have ways of doing tonic shifts or whatever with the input keyboard is another way it can go over. This works just fine with an MTS retuned synth set to play notes on a single channel, and would also work fine with a velocity key switch retuned synth.

With this approach, the main difference is that with the MTS sysexes you have a maximum of 128 simultaneous notes held down simultaneously. With the note velocity keyswitches, you have a maximum of 123 notes in your chord.

Full range of notes on the synth still playable
With all these methods you have the full range of the instrument available for output notes on the synth. If the instrument you are playing has samples or sounds for midi notes 123 to 127 you can still play those, by setting other input notes to play them.

Custom built monster keyboards
What if you want to build a monster keyboard able to do velocity keyswitch retuning directly, with its own built in software to convert key presses into appropriate midi messages to play the desired pitches?

At first it might seem that such a keyboard would be limited to 123 keys per midi input channel. But actually you could design a keyboard with any number of keys, say 1000 keys, all playing distinct notes and all played on a single channel. The only restriction, again, is that you can't play chords with more than 123 notes held down simultaneously all in the same channel.

I thought it might help to explain how this is possible, by sketching a possible design for a monster keyboard like this. I am assuming that the keyboard has an embedded microchip which is used to translate the key presses into a stream of midi notes.

I have no idea how such chips are programmed, but the concept behind it is simple. The idea is to dynamically reassign the midi note numbers to keys. Only the keys that are actually held down at any time need to have a midi note number assigned to them.


 * 1) Internally each key has its own id, not limited to the 0 to 128, as it is not using midi at this stage. Could be voltages or 16 bit numbers or whatever it is the key sends to identify itself. So for a 1000 key keyboard, then each key sends a different number from 0 to 1000 to the internal chip that converts the key presses into midi messages, this is the key's id.
 * 2) the chip has an internal table of 1000 entries which keeps track of the midi note numbers assigned to these ids. It doesn't matter how this is done, but to take an example, to start with, it could be set to 0 to 122 for first 123 keys, and then remaining keys assigned number -1 meaning not in use.
 * 3) The chip has another internal "lock" table of 1000 entries, initialised to 0. As soon as you play a note, that number is marked as locked - its entry in the table is set to 1. When a key is locked, its assigned midi note number can't be changed.
 * 4) When you release the key it unlocks its number so it can be reassigned.
 * 5) When you press a key that doesn't have a midi number assigned (current note number -1), then the program on the microchip will look through the table for a key that is not currently held down. It then will then swap with that key. It sets that old key's note number to -1 meaning not assigned any midi note number, and use its number as the midi note number for the new key.
 * 6) Before it sends the new note number as a note on event, it sends the sequence of high keyswitches needed to retune that note to the correct output note on the destination synth.
 * 7) At any time no more than 123 of the keys have midi note numbers assigned, and the remaining keys are assigned -1 = not assigned.

Then we can assign any pitch in the midi range to any of these 1000 keys. The only limitation on the keyboard is that you can't hold down more than 123 notes simultaneously.

There might be other ways to do it, but this method would work. The only question is how easy or hard this is to program. That takes us beyond my realm of expertise, would be easy in C code such as what I write, and surely is possible at least.

C-code sketch for monster keyboard
Here is an idea of how you might do it - monster keyboard c-code sketch. It is just done in code as a way to present the idea which may be more readable for some programmers, not actually tested.

Useful links to download free Kontakt instruments
These instruments work with the full version of Kontakt. Most of them can be microtuned using this script with a simple copy / paste of the script into one of the free script slots in the instrument as in the screenshot above.

http://rekkerd.org/freebies-for-native-instruments-kontakt/

Also most of the Kontakt Factory preset instruments can be scripted in this way.

Instruments that need work or can't be scripted in this way
Some of the pianos have no free scripting slots. With all the scripting slots occupied and password protected, there is nothing you can do. But the editors of the instruments might be able to merge my script with theirs.

Some of the scripts interfere with this one. The bassoon for example in the factory presets collection did this. I think it probably generated extra notes during receipt of the high numbered keyswitches which interfered with their effect.

Anyway whatever the reason, it needed to have my script before the Instrument script. As none of the scripts could be edited, I bypassed both the existing scripts bypassed, put my script into the next free slot and then put copies of the existing scripts in the remaining slots.

So that is a possible solution if you have enough free slots.

The gnarly Kontakt scripting language
I thought I'd say a bit about the language for programmers who may be new to it. Some of my script may seem strange, and hard to follow, if you don't know about this.


 * You can only declare variables in "on init". All the variables are globals, and they all have to be declared inside of the "on init" handler for some reason. Which as you might expect gets called when the instrument is initialised - also when the script is loaded.
 * All the variables have to be integers or strings; it has no floating point format. You can display these integers as fixed point decimals to the user however, so for instance declare ui_knob $Cents(-50000, 50000, 1000) declares a ui knob with a range of -50000 to 50000 but the numbers are all divided by 1000 before they are shown to the user, so giving a display that shows millicents.
 * Integer variables all must begin with the symbol $, arrays must begin with %, and strings must begin with @
 * It uses # for !=, := for assignment and = for test of equality.
 * Most user declared functions can't have arguments passed to them
 * However you can also have "inlined functions" which have parameters and are inlined during compilation, see inlined functions
 * Some predefined variables such as $NOTE are valid only in the original handler e.g. that one is only valid in "on note" and "on release". They are also valid in functions called by those handlers.
 * You can declare variables that create knobs and other controls in the ui, e.g. declare ui_knob $Note(0, 127, 1) This means that the variable $Note has a range of 0 to 127, and no other values can be stored in this variable, even temporarily in the process of working things out. So you need to declare other variables if you want to do calculations with these numbers.
 * Some instructions can't be skipped with if end if conditions. The declarations obviously, but also "make persistent" is always executed. The only way to not make those variables persistent is to remove that part of the code or comment it out. The same also applies to SET_CONDITION(NO_SYS_SCRIPT_RLS_TRIG) which is why I comment it out in the version without release triggers.

Useful external links for the Kontakt Scripting Language
This is for developers who want to write their own Kontakt scripts, I used some of these links when writing this script

http://www.sandymilne.com/category/kontakt-scripting/

http://nilsliberg.se/ksp/

http://www.soniccouture.org/tutorial/scriptorium-manual.pdf

http://www.audiotek.cz/produktyATK/ni/kontakt3_p2.pdf

http://www.soundonsound.com/sos/jul10/articles/kontakt1.htm

Undefined controller 119 to switch on and switch between retuning methods
There, tuning software would send 119, 0 to switch off tuning, but synth would switch it off in response to any unrecognized value.

Synth could default to not retunable and require CC 119,119 (or vals 120 or 121) to switch it on.

Advantages:

Helps prevent accidental retuning
To retune the synth you have to


 * send the undefined controller 119
 * send it with a value of 119-121
 * send a recognised note 127 status byte
 * immediately follow it by note 126, or another 127 with velocity 127

This is also why I chose 119 here for the value. It is easy to remember, send avalue same as the controller number. Also isn't a normal value for an on / off controller such as 0, 64, or 127

Some hardware keyboards are preset to send undefined controller values for some reason. So there is probably nothing we can do to logically prevent accidentally retuning. But this way it is at least highly unlikely to happen. 119 seems an unlikely undefined controller for a keyboard to be preset to send automatically, as it is the highest numbered undefined controller.

Does anyone reading this know of any hardware keyboard which is preset to send undefined controller 119 when you adjust one of its controls?

Lets you retune and use all 128 notes for the velocity tuning keyswitches
Example, with the 126, 127 velocity tuning keyswitches, this retunes the input note 127 so that it plays middle C

Now because the velocity retuning is switched off we can play midi note 127 on our array keyboard, and it will play a middle C, and not change the tuning of the synth.

Delay needed before switch off retuning for 128 notes retuning - bug?
Note though that in my tests with Kontakt, for some reason a gap of about 50 ms minimum was needed between the original 119,119 and the final 119,0, otherwise the script never receives the 119, 119.

I tested this by sending just two messages to Kontakt, 119, 119 and 119,0, the script never gets notificaton of the 119 119. If you put note ons between the two messages, the script is told that the controller value of 119 for those notes is 0. If you have a delay of 50 ms between the 119, 119 and 119,0 then everything works fine. The same thing also happens in VST Host and Cantabile, the two hosts I've tested so far.

Suggests there must be some pre-filtering of controller messages somewhere along the way. Anyone got any ideas about this?

Tuning velocity keyswitches with just two note numbers 126 and 127
The idea is, to treat the note velocity for midi note 127 as a kind of "status byte" and 126 as the "data bytes"

Like this (I leave out the note on byte):

This should be followed by

(behaviour is undefined if this protocol is not followed)

Zero data handling then become easier

Could have running status for our new "status bytes", so if you want to send a series of pitch adjustments, you don't need to keep resending the "status byte", like this:

That's 12 bytes total per note retuned, or using normal running status for the note ons, 4 for "header" + 8 bytes per note.. Or, if you make sure to switch the notes off, 24 bytes, and with normal running status, 4 or "header" + 16 bytes.per note

With note offs as note on 0, is

If one of the values is 0, e.g. f is 0 you just send a 127,127 for that data byte.

Pitch bend format
The pitch bend follows the same format as a standard midi pitch bend, except that the pitch bend range (just for these messages) is hard coded to +- 50 cents. So there are 16384 steps for the range -50 to 50, giving a pitch bend resolution of 0.0061 cents.

That is good enough for even the most demanding of microtonalists in almost all situations (in theory, you could notice a difference in tuning of less than a hundredth of a cent on a super accurately pitched instrument if you had polyrhythmic beating partials beating at different rates from each other but this situation is likely to be exceedingly rare).

To calculate the pitch bend in cents use the formula 50*(n - 8192)/8192

So in our example the bend for our example of 4160 is 50*(4160-8192)/8192 = -24.609375 cents

So now, when you press midi note 60 on your keyboard, the output note is midi note 64 bent downwards by 24.6 cents.

I've added the extra fine pitch bend here mainly so that the numbers can be displayed accurately in Kontakt, and because Kontakt works internally with millicents, or a thousandth of a cent. This divides each of the pitch steps into a further 128 finer steps giving a resolution less than 0.00005 cents. Most programmers will neither need or want to do this I imagine.

This +- 50 cents pitch bend range is just for the purpose of this tuning message. The normal pitch bend range is unaffected, and you don't need to know what it is (in the Kontakt script below, I have no idea what the pitch bend range is for the instrument, don't know of a way to find it out, but don't need to know it).

The reason for hard coding the pitch bend is because it adds no benefit to have a wider pitch bend range. We already have unlimited range pitch glides throughout the entire range of midi notes, through a sequence of note on changes, and coarse and fine adjustments. If you make the pitch bend range adjustable then this introduces possibilities of intonation errors due to mismatch of range between the retuning software and the synth - and doesn't seem to add any benefits when we already have pitch glides through the entire midi range of pitches. 50 cents is a natural standard to use as it is the smallest pitch bend range which makes it possible to achieve all possible notes.

Instructions to reduce amount of data for pitch glides, midi controlled pitch vibrato and other frequent retunes
These instructions would reduce data for pitch glides over midi cables. Over USB cables and internally within a computer the data rate is so fast that the difference is not likely to be noticeable, probably talking about nanoseconds rather than ms (USB midi transmits data at a rate hundreds of times faster than a midi cable USB Midi spec).

So, I am not sure if there is enough demand for this to make it a requirement of the synth. If not could be just optional instructions that a synth can program or not as desired and retuning programs would normally do a full retune of every note using 127,4 only.

However, the amount of extra programming for a synth to support these is likely to be low once you have programmed 127, 4, just a few extra instructions. So for that reason might be reasonable to make them a requirement.

What do you think? (After reading the suggestion below)

For pitch glides, to reduce number of messages, could have:

So the idea here is, first you do a full retune of the note with a 127, 4. After that you can do further retunes of the same note using 127, 3. The input note of the 127, 4 is remembered by the synth until the next 127,4 or until a 12-et reset 127, 126.

This should be followed by

Similarly if you just want to adjust the bend for the most recently retuned input note, and want to leave its output note unchanged:

(with switch off and normal type running status, 4 for header + 8 bytes per note)

And, if we like, could have:

(with switch off and normal type running status, 4 for header + 4 bytes per note)

So - the idea is, multitimbral synths can be in mono tuning mode only, multi channel tuning mode only, or optionally both, and if both modes are supported, can, again optionally, respond to these instructions to change the tuning mode via midi.

For technical reasons, this script does not currently permit switching between mono and multi channel tuning mode via midi, and these instructions are ignored.

This Kontakt script acts on the instrument instance as a whole. So, it will be in multi channel tuning mode if you load a different copy of the instrument in each midi channel, and mono tuning mode if you load a single instance of the instrument set to receive midi notes on all channels.

(Q: Is there any way for a Kontakt script to check which midi channel the notes were received on? If so I could make it switchable between the two modes in case where user has it set to receive notes on all midi channels).

Multi channel and mono tuning mode
Could have e.g.

How to ignore incomplete and non well formed instructions
If you receive any midi note apart from 126 or 127 on a channel, that counts as an "end of instruction". If the previous instruction hasn't been completed at that stage the whole thing should be ignored.

So for instance

The 127, 4 should be ignored as an incomplete instruction. You only do the retune when you receive the fine pitch bend value.

The same thing happens if the 127, 4 is interrupted by another instruction byte

The first 127, 4 should be ignored as an incomplete instruction.

How to ignore unsupported instruction bytes
If you receive an instruction byte your synth doesn't recognise (e.g. could be the 125 or 124 above) then you should ignore all the "data" received until you next encounter an instruction byte you do recognise.

An unsupported instruction byte will also interrupt the data so

All the data from then on is discarded, until the next recognized instruction.

Also, the two bytes of the incomplete 127, 4 are discarded.

"End tuning" instruction
Any unsupported instruction will stop the "running status".

Let's set aside 100 as an "end tuning" instruction.

Optional instructions (most synths and most retuning software will ignore these)
With 126 instructions available 1 to 127, and only eight of them used so far (including the 127 for data 0), there is lots of room for expansion if more instructions are needed.

Optionally (would be useful for Kontakt) where e = extra fine pitch adjustment (gets you to millicents precision).

Since this instruction sets the extra fine adjustment for the NEXT NOTE RETUNE it has to be sent BEFORE THE NEXT NOTE RETUNE.

So to send a note with extra fine pitch resolution:

where n=input note, m = output note, c = coarse, f = fine pitch bend, e = extra fine pitch bend

Indeed if you want to permit transmission of the data at very fine resolution you could also have

and so on up to as many bytes of extra resolution you want. Nobody else needs to understand these messages, so if anyone wants to add this capability, probably so they can display the incoming data in destination synth, causes no problem to anyone else.

Similarly you could have extra instructions to set the pitch of the 1/1, or ratio value, or indeed, can send a description of the note

Here if you don't want the 1/1 to be any of the notes in the tuning, e.g. for a CPS set, then is no problem, you can just temporarily retune e.g. midi note 0 to the desired pitch for the 1/1 and once it is set, retune it again to whatever you want it to be).

The ratio would not normally change the pitch of the note, just change the way the note is displayed, and should be ignored if the note is not at approximately the correct ratio from the 1/1 to play that pitch. If the note is already at the correct pitch to a reasonable tolerance (say within a tenth of a cent or whatever), then it is permitted to retune it to the exact ratio on receiving this instruction.

If you want to reduce the amount of data for 12 tone tunings, you can have

I'm not sure it is worth the extra complexity of coding to make this a requirement of a velocity retuned synth however, what do you think?

Sending the instructions using a single high numbered velocity keyswitch
Actually you could reduce it to velocity instructions on a single note number, midi note 127. But that would get more complex and need more data.

One way to do that would be to use the. highest 3 bits set the "status" and lower 4 bits set the "data". This time you need 4 messages to send 2 bytes.

I'm not sure it is worth the extra complexity and the increased number of bytes needed.

Velocity tuning keyswitches (original method with notes 122 to 127)
Here the note 127 is used for "extra instructions".

So for instance Now input note 60 will be retuned to midi note 62, with pitch bend, 64 + (32*128) = 4160

Zero values
All this works fine except for a minor complication here that the note velocity has to be greater than zero, because a note on with velocity 0 is treated as a note off in Midi.

This was solved by adding some more messages using the 127 (extra instructions) note.

Each instruction should be followed by a note release, which can be note on velocity 0. With running status this means it takes 8 bytes to retune a note.

Persistent output note
When you change the input note, the output note gets set to the last used output note for that input note. This may be clearer with an example:

So for instance in

This helps to reduce message traffic. For instance if you just want to slightly adjust the tuning of all the notes, e.g. several notes gliding simultaneously, then you only need to set the input note and the fine pitch bend of each note, most of the time.

The RPNs idea
This was our first idea. However it didn't work as a Kontakt script. It worked by using undefined RPNs.

With the coarse part of the RPN set to 100 and 101 and fine part of the RPN set to the note you want to retune, this is what you send:

There tuning happens during receipt of the pitch bend LSB. Only the input not and the pitch bend LSB are required, so a minimal message if you keep the current data values for all the undefined RPNs is:

where the - means that no data is sent.

However for this to work you need to be able to inspect the raw data stream. Is not enough to just be able to interrogate and find the current data value for any of the RPNs. If your script or code can only inspect the current data value of the RPNs, then it is necessary to send all the data.

In hex:

bug for RPNs idea
I got this working, successfully retuned a Kontakt instrument to 120-et using RPN retuning sending the RPN messages from a debug build of Bounce Metronome.

But ran into a curious issue. I have to leave a pause of 40 ms between each change of rpn like this:

RPN coarse: 101 RPN fine: NoteToRetune Data entry coarse 0 Data entry fine: NoteToRetuneTo

Sleep(40)

RPN coarse: 100 RPN fine: NoteToRetune Data entry coarse : Pitch bend MSB Data entry fine: Pitch bend LSB

Sleep(40)

If I set it to less than that e.g. Sleep(10) then it gets mixed up.

It will use e.g. the Pitch bend LSB as the NoteToRetuneTo, as if it never got the intervening RPN coarse and RPN fine messages.

Anyone know why that is necessary and whether there is a work around?

Note - never found a solution to this. Same thing happened with the undefined controllers retuning. Needed to have a sleep of about 10 ms between each controller sent. If sent without sleeps, then I found that the script only received 12 controller messages out of a batch of 512 controllers sent to Kontakt, in a test run.

Note also I don't know for sure if this happens in Kontakt or in my DAW (Cantabile).

But - the keyswitches idea solves this. Notes are high priority and don't seem to get lost in the same way.

The controllers idea
Our first idea was to use RPNs. See the Tuning RPNs (specification draft) for details. However I ran into issues implementing it as a Kontakt script. It only got about 12 of the RPN instructions out of a batch of 512.

Another idea was to use controllers like this:

The pitch bend fine is an "end of instruction" so no retuning is done until the pitch bend fine is reached. The extra fine pitch bend was an extra thing I added myself to get millicent precision in the display in Kontakt (sent before the fine) - most programmers would not need this.

For choice of controllers I used the last six of the "undefined controllers" (UPDATE - now using 119 to set the tuning method - so will use 113 to 118 instead of 114 to 119, will update script soon).

Another approach, to use undefined system common messages
Midi has several undefined "system common" messages which are just like the Note on, aftertouch etc messages but are left undefined for future use. It seems unlikely that these will ever get defined.

These messages include: 0xF4, 0xF5, 0xF9, and 0xFD

If we could use two of the undefined system common messages, that would be ideal for data, as that would let us send two bytes of information with a single message and you could retune a single note with just two messages, six bytes

as before retunes m to n with c, f for the pitch bend

Unfortunately you can't access them in a Kontakt instrument script. I would imagine in many environments you can't access them.

It could be done in Windows, sending messages from one Windows program to another one coded at a low level with midiOutShortMsg