Data Knot – Machine Learning tools for low latency real-time performance
I am excited to finally release Data Knot, a package for Max based on FluCoMa. It’s something I’ve been developing over the last 5 years or so, the last 3 of which were under the name SP-Tools.
Data Knot is a set of machine learning tools that are optimized for low latency and real-time performance. The tools can be used with any type of audio input, including optimizations for Sensory Percussion sensors and ordinary drum triggers.
It is made up of over 120 objects actress 13 categories spanning low latency onset detection, onset-based and real-time descriptor analysis, classification and clustering, regression and prediction, corpus analysis and querying, synthesis and modeling, audio, data, and CV processing, and a slew of other abstractions that are optimized for low latency applications.
Ambisonic feedback experiments
In late 2023 I had the idea to expand the kind of feedback-based playing I was doing with pieces like Kaizo Snare by having some kind of robotic microphone arm holding an additional microphone that would be controlled by the computer while I was doing other things. Like some kind of 3rd arm thing.
Well after much experimenting and testing I’ve gotten it up and running and it looks/sounds something like this:
(more performance videos below)
Continue reading »
Kaizo Snare
I spent the better part of last year working on a performance that was a strange combination of things. It pulled together turntablism, feedback/friction, machine learning, signal decomposition, 3d printing, robotics…and a snare drum. All of these were ideas I was working on and exploring separately, but as things can sometimes do, they ended forming into something that was much more than the sum of its parts.
You and Me and Us and Me and You
I like to picture that there are an infinite number or realities, all running in parallel. The entirety of my existence, everything I have ever known is but one of these. I represent a single point on this infinite line.
There are moments in your life where you feel that infinity collapse. The things, moments, and people that exist across the multitudes of possibilities. These create a mirroring, an echo, a shimmer across these realities. You are no longer you, you are the infinite you. You stand across all time. You see the infinite.
Sometimes you live this without even knowing it.
This is a blog post about infinity. Specifically infinity minus one.
Friends. Objects. Lights. Feet.
Over the years I have had many wonderful discussions with people whom I have never met. Either through emails, forums, or even chatrooms I have talked about all manner of things with people scattered around the world. Sometimes these virtual friendships manifest in the physical world and a thing is born. An art thing. This is one of those times.
Black Box project
The Black Box project is a project that involves Pierre Alexandre Tremblay on bass/electronics, Patrick Saint-Denis on robotics/electronics, Sylvain Pohu on guitar/electronics, and myself on drums/electronics. It’s a four-way collaboration that has gone through two residencies (in Montreal and Huddersfield), to work out the finer details of putting a large-scale show together.
It looks and sounds something like this:
Lights
Here are a couple of performance videos using an approach I’ve just recently started calling Light Vomit.
What you see in the video is a combination of automated processing (via The Party Van/Cut Glove) with a variety of DMX light interactions and behaviors. Everything is controlled from a Max patch I made specifically for the gig (and specifically to test these behaviors), most of which use audio analysis to dynamically record/play/process incoming audio and trigger a variety of light behaviors. (Click here to view my moment to moment analysis of the performance using my Making Decisions in Time improv analysis framework.)
dfscore 2.0
dfscore 2.0 is here! dfscore 2.0 is a much improved and completely rewritten version of dfscore that I started working on a couple of years ago. The dfscore system is a realtime dynamic score system built to display a variety of musical material over a local computer network. The primary motivation for building the system was to allow for a middle ground between composition and improvisation.
But before I get into all of that, here is a video showing the latest version along with its premiere at the Manchester Jazz Festival:
Everything. Everything at once. Once. (3)
Here is the 3rd iteration of my ongoing series of pieces ‘Every. Everything at once. Once’.
Before I gib gab about it, here is “Everything Everything at once. Once. (3a)”:
The idea for these pieces is to focus my creative process and energy on the curatorial act of choosing the instruments I will improvise with. This has now extended into the choosing of the location, look, and visual approach used in each video. Each video in the series has had its own distinctive look, and has also come to include new collaborators alongside my long-time collaborator Angela Guyton.
Going into this third video, I knew I could try something different, because of the acoustic (and battery powered) nature of the instruments chosen. A melodica (which I bought over 15 years ago at a flea market for $5!), and my trusty ciat-lonbarde Old Mr. Grassi. I first thought about using some kind of outdoor location, but after seeing David Pocknee and Michael Baldwin‘s basement, I thought that it would be a wonderful place to film.
The original idea was to use some dripping paint, with David and Michael each having a squirt bottle, and then adding some vertical glitching artifacts similar to the language Angela had explored in her latest video. But after doing a take with the paint, and seeing how amazing the lighting by David and Michael was, the paint/glitch idea was scrapped altogether. Angie thought that with David and Michael enacting two tasks, what their role meant within the work changed. It created a different context and gave the work a different character, but one that didn’t seem as complete as when they were focused completely on their dispassionate navigation of the space as they sensitively manipulated the lighting.
I was initially hesitant about this change, since I was quite excited about the paint/glitch idea, but I trusted Angie’s judgement on the matter. And in the end, the videos came out better.
The choice of instruments for this particular version of the piece is far smaller than the other versions, with just two instruments:
1 x ciat-lonbarde Old Mr. Grassi
1 x Artist Ltd. Pianica
The first version had 10 instruments, and the second version had 18.
That being said, this particular combination of instruments is one of my longest standing ‘pairings’, going to back to when I initially started performing solo improv. In that sense, it served as an almost proto version of this creative thinking. Some of the language in the third video (3c), particularly the near-unison sustained pitches, diatonic-y material (Gminor?!?!), is something I strongly associate with this particular combination of instruments.
Even though the combination of instruments is quite old for me, I wanted to incorporate that specific pairing into this compositional framework, where the choice of instruments defines the piece.
I am still, however, surprised at the endless source of inspiration these instruments provide. It almost feels like cheating, at this point, given how interesting the instruments sound, but I felt I found lots of new and fresh angles on them. My work on the Battle Pieces, .com pieces, and dfscore system has given me such a fluid understanding of form, gesture, and pacing, that I can really focus on multiple formal levels while attending to the micro/developmental nature of the sounding materials themselves.
The visual identity of these pieces is becoming something that is increasingly important, which in this case specifically involved dynamic manipulation of light. This tendency has been apparent in all of my work over the last few years, but each one of these videos, particularly, has a very considered visual identity and approach. I am curious to see where this will go, and if the general idea of these pieces (to draw a metaphoric circle around the part of the creative act I want to focus on) will explicitly be extended to the entire ‘art object’.
When asked, after filming the videos, to describe what we thought was important using only three words (Michael had asked David this question before), Angela and I shared two of the three words: creativity and love. We disagreed on sharing vs freedom for the third word, but what can you do. I am comforted by the fact that I’m right and she’s wrong.
In progress
The last few months have been both slow and busy at the same time. In that time I’ve been slowly working on multiple pieces/projects simultaneously. I figured it might be interesting to make a blog post about things in progress.
After finishing a piece for good friend Linda Jankowska and the end of last year I started working on two other pieces. The first one being a piece for skipping CD player and drumset. The working title is “Rhythm Wish”, but I’m not fully settled in on that. It looks/sounds something like this:
The piece, compositionally, will fall somewhere between two approaches I’ve been working on recently. First the kind of open/memory/improv structure of pieces like the .com pieces, and to a lesser extent ‘an amplifier‘, and the game/challenge/etude modes of interaction found in the ‘Battle Pieces‘ from a duo (strikethrough me & you) with another good friend Sam Andreae.
The piece will be constructed in a way that before the performance the performer (either myself or David Meier) has to improvised some material, given certain prompts. Those materials will then be burned onto a CD player (with the un-muted modification ala Nic Collins CD Hacking). The CD player will sit on the snare drum (as in the demo video I made above), which will cause the CD to skip around and act unpredictably. That glitchiness (along with the order of the tracks on the CD) will form the structure of the piece at a larger level. The more local interactions will be defined by challenge/interactions that the performer will have to engage with, along with the recorded version of themselves.
There will also be some real-time ‘nudging’ of the CD player via a footswitch that I hacked together from an old remote control for the CD player.
As evident in the video, the sound of the CD player blurs with the sound of the acoustic drumset, which is very much intentional. Some of the seedling ideas for the piece were of a sort of temporal duality. With a protagonist/antagonist fighting it out musically, one being live and one being recorded. I like the poetics of it being the same person/instrument with the conflict coming only via the displacement in time (then vs now, past vs present). I took these ideas and played around with a ‘3-act play’ structure, which is still somewhat present in the larger structural framework, but is less important than it originally was.
I will also play with illusion in terms of what is real and what is recorded. This kind of interaction was a mainstay in the performances of the performance art duo I have with Angela Guyton, Takahashi’s Shellfish Concern. During the era that relied on live painting + live sampling, the idea of hearing the sound of the lines being put down, then not hearing them (or hearing a different recording) when seeing it happen again proved to be very powerful. This was very much inspired by the ‘Club Silencio‘ scene from Mulholland Drive. (more on Takahashi’s Shellfish Concern below)
In addition to the ‘Rhythm Wish’ I am also working on a piece for tuning forks and computer hard drive. It looks/sounds something like this:
The piece will be for four performers all sitting around a table, with the computer hard drive in the middle. The piece is written for a new ensemble I’m performing in called bird rat centipede and will be called ‘the keys to everything that has ever existed’.
The main compositional material of the piece will be ‘privacy’. In the sense of ‘hitting the tuning fork on the table and pressing it to the table’ being a ‘public’ sound, and ‘hitting the tuning fork on your knee and holding it near your ear’ being a ‘private’ sound. I very much like the idea of multiple perspectives on the same piece, particularly since there is no ‘real’ perspective. Each performer hears a, literally, different version of the piece, and same goes for the audience and/or recording.
The computer hard drive is modified with an ‘analog HD’ by Gijs, which is essentially 3 oscillators and an amplifier driving the voice coil in the hard drive (so it acts like a speaker).
I’ve also started working on my dfscore system, which is a networked laptop score displaying software system I’ve been working on for a while (which has taken a bit of a back burner position for a bit). I used a very stripped down version of it in ialreadyforgotyourpussy.com with Richard Craig, which looks/sounds like this:
I had pitched the idea to the Manchester Jazz Festival last year, and although it was not accepted, I did continue talks with them. The project is now moving forward again and is, quite unusually for me, being financially supported by Jazz North. I’ll have more details on this soon, but it is very exciting to be working on the system again.
It will eventually be a software system that you can quickly install on any number of laptops which are connected up via ethernet cables (for minimal latency). This will allow for a more dynamic/adaptive type of composition which would work very well with the kind of music I’ve been writing as of late. I will also be openly sharing all of the code and try to make it as user friendly as possible.
I’ve also finished putting the finishing touches on this MIDI to relay system built for me by Circitfied (Dan Wilson). I commissioned Dan to build a box that would take MIDI messages and turn on/off relays which I would connect to a bunch of ciat-lonbarde instruments. The box on it’s own is pretty cool/exciting (not to mention great looking!) but it will be much more so once I pair it with some crafty programming.
I plan on building several modules to drive it, including an attack detection to random re-patching, and a much more challenging/exciting analysis-based system where I can analyze the audio descriptors (using Alex Harker‘s externals for Max) of each combination made by the box, then by analyzing incoming audio, create a resynthesis of that audio using repatching/reconfiguration! Very similar to the synthesis used in my CCCombine Max patch, but instead of using a bank of samples, it would use hardware synthesis/sounds.
Lastly (but not leastly) is a continued development with the next phase of Takahashi’s Shellfish Concern. We’ve moved away from an amplified canvas and towards a DMX lights based setup. That looks something like this (though not directly focused on Angie):
The trajectory of the work has been moving further and further away from an ‘art object’ in the last few years, so this is the final ‘jump’ away from physical stuff. Most of it is still based around an image, but now it’s smeared in time/space by using flashing lights as the only visual component. The sonic component is still being worked on, though it’s mainly been sampled voice/breathing.
strikethrough me & you – live
Some more me & you action, this time live. Oddly enough, even though we’ve been at this for a while, this was our first live performance. The set went really well, to a full house at The Noise Upstairs.
Still working on the mixes from the recording session at the end of last year and will release all the tracks for free once they are ready. For now here are a few of the performances from our first gig.
.com pieces
The .com pieces are a series of pieces I started in early 2013 which are centered around snare drum feedback and form/memory as composi(/improvisa)tional material. So far I have finished two of them, iminlovewithanothergirl.com (for snare, microphone and DMX lights) and ialreadyforgotyourpussy.com (for amplified wind instrument and amplified snare). The third and final .com piece is in the sketching stages and is as of yet untitled.
photo by Nick Rutter
Shbobo/Shnth/Ciat-Lonbarde
So back in March I got some funding from my school to bring one of my favorite instrument makers to the UK.
Peter Blasser has been making crazy amazing instruments for years now, and I’ve been using them from the point they were first available as kits (the original Fourses/Fyrall).
Peter has started a new company focusing on digital/embedded instruments, the first of which is the Shnth. I can attest to it’s kickass-ness.
Check out all the links to learn more about it, but this post is mainly about two videos. The first is an excerpt from a performance Peter and I did. The second is an amazing documentary (by Angela Guyton) about the workshop itself.
Snare + Microphone
Before I talk, here is some sights and sounds:
During my summer tour last year I improv-ed my way into using a condenser mic as a friction implement. The sounds were harsh, but very controllable and with a great continuity between vastly different types of sounds/playing. I instantly knew I was on to something, and wanted to explore this further.
Fast forward to one month ago.
I was starting to plan/conceive my next composition, having recently started a PhD at the University of Huddersfield, and wanted to explore this microphone/snare thing further. At the same time, I had decided that I was going to move away from the “middle man” of writing a composition, for myself, to perform. (More on this, and the implications of it in a future post). I decided to use this very limited mode of playing as the backbone of this exploration. No other implements, no electronics (other than amplification/distortion), no “easy” solutions to the problem I was trying to solve.
This video is not that solution. It is just a document of some of the sounds and playing techniques I’ve been exploring.