Meta-trombone - part II

These last few weeks have seen much development on software side of the meta-trombone controller.  I created a monstrous Bidule patch to instantiate my vision of what this integrated controller/instrument should be.  

Here's a top level view of the patch:

The audio signal from the trombone goes to an audio to midi converter that I developed to extract midi notes from my performance.  It also goes to external outputs and/or Mobius looper through an audio matrix that makes possible all the complex routing that my setup requires.  Finally, trombone sound is also fed to an envelope follower that, depending on performance mode, can modulate the playback of Mobius' tracks (more on this below).

Here's a look at the top level of the audio to midi group:

This group is based on an audio to midi converter I found on the Bidule forum.  From what I gathered looking at this group, it seems to determined pitch by comparing incoming notes to a delayed version.  The amount of delay is based on frequency and sampling rate.  When the two signals are combined, if the frequency matches what was fed to the delay, they are phase-cancelled to near 0.  This process is done for every note we want to evaluate and the note closest to 0 is the one returned.  

The original group work well enough, but it was a bit sluggish for my taste and used up considerable system resources.  My approach, as I indicated earlier (and as implemented by students at Cornell), is to reduce the number of notes the system has to evaluate by tracking the position of the trombone slide and evaluating only for those notes that can be produced at the current slide position.  I'm still working on a reliable way to track slide position, but right now this audio to midi converter works quite well for all notes in first (closed) position.

The midi notes that it generates are sent to Mobius to trigger various scripts depending on the performance mode currently selected.  Before I describe these various modes, let's take at the control interface that I created in TouchOSC that allows me to control this complex patch with my iPod Touch.

The first column to the left selects the performance mode.  The one immediately to its right selects the Mobius track that is effected by the performance mode.  The red column selects the Mobius track that will record performance from either the trombone, an effected Mobius track or both.  The last column on the right turns the audio output for that track on and off.  Finally, the purple button on the bottom determines wether trombone sound is sent to external outs and Mobius tracks.

While this interface may look simple enough, there are many logical conditions that are evaluated behind the scenes each time a button is pressed.  For instance, if Play 1 and Rec 1 are selected, the audio from track 1 is not actually fed into itself.  If Rec 2 is selected and Out 2 is off, the system turns it on automatically.  And so on...  these logical conditions are the core assumptions I built into the system that will make it behave as I intend it to.  A big part of that, was defining how the performance modes would affect the various system states to produce the required results.

Performance modes

Starting with the simplest, in Off mode the midi notes interpreted from the trombone performance are discarded.  In synth mode, the notes played on the trombone modify the playback rate of the Play track in Mobius.  Also, the playback of that track will be modulated according to the trombone performance by virtue of the envelope follower.

In Envelope mode, the track playback is also modulated to the envelope follower, but the midi notes no longer change the playback rate.  Instead, each note played on the trombone changes the playback location of the loop.  Trigger mode is the same as envelope mode, but without the envelope!

Things to do

I still need to complete the pitch recognition system by developing hardware to track slide position.  I also need to develop the midi module element of the system.  Presently, I'm thinking of adding an arpeggiator, a quantizer and perhaps a midi looper.  Next, I will add effects!  Not only do I want to modify the input trombone sound and the loops, but I also would like to map trombone performance to changes in effect parameters and effect selections.  For instance, I could map a filter's cut-off frequency to trombone notes so that playing a certain note would emphasize a specific spectral aspect of the effected loop.

How will it sound?

While there's no way no know exactly what kind of music I'll end up producing once the whole system is up and running, below is a recording of a recent practice sessions that used only the Synth and Envelope mode.  No effects were added and it is rather poorly mixed, but it does, I believe, show promise.  One thing I'd like to note about the performance is that everything you hear, with the exception of the solo later on in the piece, is created from "performance" of the first recorded loop (what you hear twice in the beginning).