M1D1 Animatronic Music Robot

M1D1 is a project I’ve been working on in my free time over the past year. It is powered by an Arduino DUE microcontroller and programmed in Arduino C++. It takes some of the core code library I developed for my Music Spiral project, and greatly expands it with a deep implementation of music theory, new control interfaces, bass accompaniment MIDI output and some machine learning capabilities to listen in real time and interpret musical context and key signatures.

From an animatronic and visualization standpoint, it incorporates:

An Anthropomorphic Robot Head
About 8 inches high, built from custom 3D modeled and printed parts, aluminum, servos, fiber optics, and electronics components. Inside the cranium is a little fake plastic garden.

LED Lights: The robot head has around 90 RGB addressable LED lights in its cranium, eyes, mouth and internally to light up fiber optics. The LEDs are programmed to respond dynamically to associated chromatic note mappings, over multiple octaves, and from multiple devices, with a variety of selectable rendering modes which alter how it operates. There are modes to color map different music scales/keys, color map different instruments with separate colors, color map based on synesthesia frequency relationships, etc…

Ultrasonic Mist: Hidden inside the head of the robot is a modeled reservoir that holds water, and a capillary action tube with sponge material. An ultrasonic mister can be engaged by a variety of inputs (usually I map to MIDI CC for channel pressure). When the user plays a key on the synth and applies aftertouch, the head generates ultrasonic mist, which comes out of the top of the head/cranium.

Servo Controlled Eyes: A servo mechanism connected to the eyes allows them to move back and forth to the beat.


A Chromatic, Archimedean Spiral Light Display
The body section is an Archimedean spiral measuring about 12 inches diameter. I designed it and 3D printed it in White PLA Plastic. It has a translucent white mylar sheet on the front to hide the LEDs and diffuse the light.

LED Light Spiral Array: The spiral has about 100 LED lights embedded in its construction. They are all grouped chromatically in arrays and mapped to visualize over 5 octaves of MIDI musical notes. Various different display algorithms can be set like on the robot head and previous logarithmic music spiral project I built.

OLED Display: The OLED display above the body section renders out realtime, instantaneous information on what is being musically input and/or output from the M1D1 robot. It has an extensive music theory library built in, and when you high basic, or very complex chords, it instantly recognizes what chord is being played, along with inversions of chords, and displays that in the interface. Ie: if you hit a bunch of keys and don’t know what you would call that chord, it shows you “ie: A min 7 over E”. It does this even with multiple instruments being played simultaneously. For instance, if a bassist is playing a note, along with three notes from a keyboard or guitar midi system, it can categorize chords based on the combined musical signature of the instruments.

In addition, this is where data on the M1D1 active listening is shown. The M1D1 bot catalogs what is being played over time, and has some machine learning type of intelligence built in to determine overall key signatures for a song and key changes.

An Intelligent Bass Note Accompaniment System
The third element shown is the control interface enclosure with Five Large Tactile Faders, Three Potentiometer Knobs, and Two Buttons. The enclosure was designed and 3D printed with White PLA plastic.

MIDI Output: A MIDI bassline accompaniment is output in real time. The root note and key signature for the groove intervals is constantly adjusted in real-time, as you play an instrument. M1D1 follows along with your changes, even when you modulate out of key to more dissonant notes or chords. The bass line is very dynamic and acts like as if you were jamming with a real bass player who is listening and watching your changes.

Control Interface / Mixer: The various faders and knobs allow the user to control how the bass accompaniment is played, stylistically, and controlling overall intensity, dynamics and density of notes. It can also lock into an incoming MIDI drum groove.

The Knobs: The three potentiometer knobs adjust some key settings and behaviors for the Bass Accompaniment.

Pattern: The Pattern Knob selects from 15 different built in patterns/grooves that I programmed in that represent a wide variety of popular bass styles/genres. A 16th pattern slot is open for user customizable grooves, which can be taught to the device with real-time playing.

Interval: The Interval Knob determines how M1D1 takes the pattern/groove template of intervals, and transposes it according to what is being played live from other instruments. When the knob is fully rotated counterclockwise, the MIDI bassline output is the least adventurous, tightly adhering to the pattern/groove selected, and following tightly along with quick note/chord changes... it very closely mimics your playing changes in real time, including if you drastically change the key. As you rotate the knob clockwise, the MIDI bassline output becomes more adventurous, and is more likely to follow the overall key signature that has been detected over recent memory of notes... you will get more bass exploration of fifths, fourths, octaves, and some more complex intervals that are in the overall key signature.

GateLng: The Gate Length Knob dynamically adjusts the lengths of held bass notes for the accompaniment. The pattern/groove still controls the overall length of notes held, but this potentiometer scales the lengths. Counterclockwise is more stacatto, and clockwise is longer legato.


The Faders: These five faders can be set at specific settings, or dynamically mixed in real time to completly alter the content/character of the Bass Accompaniment.

PtnDepth: The Pattern Depth Fader Adjusts how much of the selected Pattern is played. At maximum it plays the entire recorded pattern/groove. As you dial it back down, it dynamically removes some of the density of note intervals for off-beats. At very low settings, it will just play the intervals which fall on main beats in the pattern. The notes / transposition always follows along with the current played input from whatever device is connected (MIDI Guitar Pickup, Synth, Piano)

Steady: The Steady Fader introduces steady notes on standard Whole, Half, Quarter, Eighth, or Sixteenth Notes. As you dial the fader up, the note density increases.

DrumSync: The Drum Sync Fader listens to incoming MIDI Channel 10 (default for drums), and can be used to lock in a groove based on the incoming drum pattern. At maximum, it will play note intervals for every cymbal, ride, hihat, bass, tom and snare hit. As you dial back the fader it dynamically removes some of the instruments that it locks on to. At around a quarter way up, it just locks into the kick drum. At about half way up, it locks the bass groove into kick drum and snare hits.

Doubled: The Doubled Fader can be used to have the Bass Accompaniment output a bass line that is one or two octaves below whatever is currently being played/input to the device. While playing, you can quickly dial this up for a section of a song, and play a riff on synth and M1D1 will output a bass line that is dynamically one or two octaves beneath your riff, for a nice doubled sound.

Random: The Random Fader adds some additional notes into the groove, mostly on curated off beats, and at the unison or octave intervals of whatever you are playing. It can be dialed up to add a bit of extra Bass Accompaniment variation and flair.

The Buttons: The Program Button enters a menu on the OLED screen above the archemedian sprial, where various deep settings can be adjusted. The shift button can be used in conjuction with other knobs to access some deep settings.


Footswitches: In addition, regular ¼ footswitches can be plugged in, offering extra controls, including the ability to tap in and out to teach M1D1 a new progression / groove. A recorded groove is automatically normalized to the key signature its played in, and transposed appropriately as the player makes music. The recorded groove can also be density mixed, or quantized to certain scales.

YouTube Video Demo of the M1D1 Head Functionality with Sequential Take 5 Synth:

For more information contact:
Jason Cooper - jason@jasoncooper.com