Experiments and One-offs: Creative Coding and VJ'ing with Processing and OpenFrameworks

Role

  • Animator
  • Creative Coder
  • Video Artist

Timing

2008 to 2013

When I first started performing as a VJ at a series of underground parties in Chicago, it only took one event to find that I rapidly got tired of working with existing footage. I had heard of Processing, and was familiar with the basics of computer animation from my day job as an interactive developer at Arc Worldwide/Leo Burnett.

Initial Experiments

Initially I created a series of sketches in Processing, and rendered them out as static clips that I then reassembled on the fly in the live video tool Resolume. These sketches were necessarily primitive, because I was learning the concepts at the time and because I needed to be able to mask individual layers, change colors or add effects, and simple monochromatic sketches made that easier.

I found that building my own source material in Processing went a long way toward increasing my enjoyment of the performance, because I "owned" what was playing on the projectors - similar I suppose to a DJ playing his or her own material!

PulsingStar - Processing Sketch Experiment from Nathan Koch on Vimeo.

ModulatorP5 and performances built on Processing

In 2011, I performed at the Logan Square Arts Festival in Chicago, and I wanted to have total ownership of the process, with a higher level of modulation of the individual elements in my sketches. Because I was performing along side an audio artist, I wanted to be able to have sketches that responded to different frequency ranges of the sound.

Over the course of building the performances, I ended up moving the shared code to a Processing library I wrote called ModulatorP5. ModulatorP5 gave me mouse control, Low Frequency Oscillators, and audio analysis for all my Processing sketches that evening. I ended up giving a talk on ModulatorP5 at the Chicago Art and Code Club.

TriangleFlow - Processing Sketch Experiment from Nathan Koch on Vimeo.

The Nature of Code and more experimentation

After my Logan Square Arts Festival performance, I didn't VJ again for some time, and instead focused on broadening my knowledge around creative coding and animation. I experimented with OpenFrameworks and Cinder, and read Daniel Shiffman's The Nature of Code.

I gave a talk in early 2013 at IDEO (hosting a new edition of the Chicago Art and Code Club) on Modeling the Natural World in C++ using Cinder. I was primarily using Cinder as the example because the lessons were easier to build- the concepts were mostly using C++ Standard Template Library data structures with just a touch of Cinder for display.

Weave - Processing Sketch Experiment from Nathan Koch on Vimeo.

Reflections

Unfortunately, a lot of my creative coding projects were one-offs and I just considered them fuel for VJing at the time. With OpenGL in Java being a moving target I was shocked to find how few of these sketches continue to work six years later!

At the same time, this was a great opportunity to focus on the process over the product, and my knowledge of Processing and OpenFrameworks will certainly inform future creative projects.