Reel Ear: Music Theory / Ear
Training / Improvisation
Created for Reel Space Music School,
Valencia, Spain.
|
Reel Ear: Music Theory / Ear
Training / Improvisation |
Words: A soundscape application used in performance installations by the artist duo greyworld. The application uses camera and manual tracking of up to 6 participants in a square space, and based upon their position, creates audio for each person to experience. The space used in the first showing was about ten meters on a side, though this could be a square of any size. Rectangle spaces could also be used with a change in how the numbers are scaled.
The sound heard by each participant comes from three layers:
---The Participant Words Layer allows each participant to record their own phrase
that they can place somewhere in the environment. Once placed, these phrases
are played back from disk, so more recordings can be made.
---The Pre-recorded Words Layer consists of words that “live” at various
points in the space, and can repeat or change the words spoken. Any folders
with audio files can be used for playback.
---Four underlying Environments (Cave, Forest, Water, Wind) add appropriate
background sounds as well as moderate the effect levels used on each person's
audio. Again, the controller can load whatever sounds they wish by choosing
a folder with .wav or .aiff files.
Participants wear wireless headphones and explore the space through movement;
what each person hears depends upon their proximity to the sounds in the other
layers. There are global “falloff” controls for each layer, so each
sound group has its own “proximity effect”---distance away versus
volume level. There are 12 separate channels of audio going out, grouped into
6 stereo pairs; the controller person can monitor any pair or pair combination,
or all 6 pairs at once, and can save audio recordings of the overall mix.
Effects (delay and panning) are moderated by the person's position relative to the four Environments, but can also be controlled individually and globally. You can also generate background noise, which really adds to some of the Environments. There's also a 6-voice synthesizer (carrier/phase/ring mod) you can mess with...it's pretty simple, but it has timed ramps for its frequencies, so some really nice evolving sounds are possible.
A visualizer (created using Jitter's OpenGL functions) shows the location of all sounds and participants, and allows manual tracking of participants with the mouse, or with multiple mice. Keyboard controls are also provided for the first four participants. A multi-touch interface could also be used for tracking...I can't wait!
Musical Skates: An application which uses sensors to turn ice skates into musical instruments, for skating practice, musical experimentation, and performance.
My client has developed a system for sensing a number of physical parameters
of the movements of ice skaters: their speed, direction, the proximity of the
left and right skates to each other, being on the inside or outside edge of
the skates, the angle of the edges, the skater's arm position, and so on. The
data streams from these sensors are transmitted wirelessly to a receiver at
a central computer. The application gathers these streams (from up to 8 skaters
at once) and transforms them into various musical sounds, with control over
instruments, pitches, volumes, and so on. In this way, a team of skaters can
CREATE the music they are skating to, rather than following pre-recorded music...or
venture anywhere in between, as each experience can inform the other. The sensor
data can be used to control lighting or projections in the ice arena as well,
so the skaters could control these aspects of the performance if this was desired.
Combine a really talented team of skaters with a few choreographers, musicians,
show designers, and producers, and this instrument could really be something.
We're hoping it will be premiered soon!
////////////////////////////////////////////////////////////////////////////
United Taps Tap Dance Analyzer: An application which senses incoming foot-tapping data from tap dance students, then displays the taps in separate tracks for each student, for timing, rhythm practice, experimentation, and performance.
The music and dance possibilities for this project are really exciting. Similar
to multitrack music applications, this program will allow users to create patterns
of note beats, then play them back in a variety of ways, with numerous real-time
controls over them. Their incoming tap beats will appear on top of their beat
track, so they can see how their timing compares to the set pattern. Up to 16
tracks will be supported, with control over appearance, motion, beat patterns,
and sequences of these patterns as larger units.
Many combinations of tracks will be useful---for example, having 8 tracks could
mean you have 8 people, it could mean 4 people with 2 tracks each, it could
mean 2 people with 4, or other combinations. Different movements can be matched
to different tracks, or maybe each of one's limbs follows its own track...like
you're tapping on a drum kit, or just on your thighs! The possibilities for
rhythmic practice and experimentation, whether through dance movements or other
means, really are endless. And the interplay between using the program as a
“follower” and as a “creator” is so interesting to experience,
especially in groups.
The grid-type of track in the program is purposefully limited in one key way:
it uses a sixteenth note as its smallest unit of rhythmic time. This means that
rhythms like triplets and other more complex subdivisions are not directly possible,
at least the way other applications generally handle this. However, if one thinks
additively---build complexity by combination, rather than subdivision---many
fresh rhythmic ideas are possible. The controls for the beats in each track
allow you to make your patterns very dynamic: able to be changed / shifted /
randomized / sequenced easily, freely or in time with the overall tempo. This
interaction is much like playing an instrument or a video game, and can add
another dimension to the experience: with a practiced teacher operating the
program while others follow along, that person can can act as coach, facilitator,
challenger, conductor, co-player, and more. Or, you can set up your own set
of challenges by "playing" a session, then follow along with your
motions as it plays back. And you can always change what track or tracks you're
following simply by where you look, so actions like having all members of a
group do something in unison is as easy as saying "now everyone follow
track 4", then "back to your own tracks". In this way, even the
simplest beat pattern can be used for many different purposes.
The program is in an early phase right now, but what I've seen through some
experimentation and testing is very encouraging. I hope to have Version 1 done
before the 2009 holidays!
United Taps Tap Dance Analyzer Video ::
3D Model Viewer: An application for orthodontists to view and manipulate 3D virtual models of upper and lower sets of teeth. While designed for this purpose, the viewer can be used to view up to 6 .obj (Wavefront Object Model) files at a time, in different window layouts.
This project is in its beginning stages; however, it should progress nicely,
as I have lots of the pieces already made in other projects. I'm looking forward
to building this one, because it will be a useful and fun tool for all kinds
of 3D exploration, especially using models and grids. Plenty of interesting
free models are available online, and free converters can turn lots of formats
into .obj (which is required). And you'll be able to texture the models with
any image or movie file, which can look amazing...lots of room for creativity
there!
Astronomy Project: An application that calculates the planetary positions for a range of years, listing one measurement per day.
Using a series of equations, this program determines the heliocentric positions of all 9 planets (yes, Pluto is back!) and stores these "Latitude/Longitude" pairs in a text file. These values can then be read by a spreadsheet or database program.
More details to come.
Other Projects
bailacon video -- 121 MB ... more about this project to come. Maybe you can figure out where I'm headed with this one, based on my other projects and how the video is put together... :)
16-step sequencer designed for touch-screens
Evolution UC33e Controller Patches
About Max
What can I say? I've found my software of choice. The more I use it, the more convinced I am that it can really and truly *do just about anything* in the digital realm ... limited only by one's imagination, patience, and dedication. My experimentation with digital media-manipulation / program writing in Flash and Director was enjoyable and fruitful while I did it, but looking back I really wish I had discovered Max first. It pays off to research these things and to keep up on what's going on...
Cycling '74 :: the great folks behind the magic
For what I wanted to do---design interfaces for the creation, manipulation, and recording of media---it has been a far more intuitive, flexible, and extensible program to use. Of course there will be pros and cons to any program, and these can change from project to project. But in my experience, Max/MSP/Jitter (MMJ) is great for 90-plus percent of my experimentation---which itself has expanded tremendously, largely due to the ease of working in the environment.
Is it "programming"? Of course. But there's a much bigger discussion there. You could say that writing an email is "programming", but perhaps many don't see it like that. Using software to create new media types, even a simple instant message, is *programmatic interaction through a tool* which has already been programmed (in that sense of the word) by the creators. But inside the program, there are a ton of options for the user, and in large applications, this is pretty much infinite. Areas such as "Customize", "Preferences", or "Macros" push the fuzziness of these boundaries even further ...
... so, my interests have been to explore that spectrum (or network) of possible "connection levels" between the user, the program, and the programmers. Using MMJ allows me to play all three roles. Just like the potentially infinite choices one could go through when organizing a kitchen, planting a garden, decorating a dorm room, writing a song, painting a picture, or typing a letter---to name a few out of a billion---you can choose how you want the interface to look, what you want it to do, what freedoms and limitations it will have, and so on. The most interesting (and challenging) thing is that you are constantly learning while you're doing it :: learning how to make the logic behind this or that interaction, what interface elements are more useful than others (as you play the "user" role and test it), what additional media could be utilized, how best to explain the patch with hints or examples, etc... etc...........
See this caveat for a comparison between programming in code-based versus graphics-based environments. Yes, code absolutely is necessary for all things computer---MMJ itself rides on top of C, and can utilize Javascript and Java within it, as well as several other languages. My problem with code is that I've seen the experience of reading and writing it be too intimidating, painstaking, and inscrutable to too many people :: people who might otherwise have strong interests and abilities in things like making interfaces, managing and morphing data, and creating "logical functionality" which can be utilized by themselves or others. In other words, exactly what code-based languages have *always* done, just like the HTML behind this page, the browser's capability to turn it into the web page as you see it, the tools and add-ins to the browser...giving you the freedom to bounce all around, grab what you want, mark things to come back to, communicate with others, etc... all riding on multiple sets of code which the vast majority of users know virtually nothing about (and don't care to know). The usefulness is based largely on how accessible it is, so users generally interact with it on its top layer (what you see), with tools that access other layers or functions "below the surface".
What I love about MMJ is that you can start at any layer and build your own networks of information and functions, gradually digging into more complex possibilities as you learn or think of other things to add. All the while, the interface is growing along with the functionality; each dynamically determines the design of the other, in many interesting ways ... paralleling the multiple roles one plays as designer : programmer : user.
Often, the design of an interface is quite important in determining how usable it is -- removing extraneous functions can clean things up and make the interfaces more intuitive, even at the expense of features. It's always tough to determine the right balance between something that does a simple function well versus something that does a whole lot, but in a complex interface. This is an eternal trade-off, but one that constantly pushes me to evolve in many areas of creativity and craft.
If you're interested in the truly fascinating programming environment that is Max MSP Jitter, there is a free, fully-functional 30-day trial available on the Cycling '74 site. Students can also get a 9-month fully-functional version very affordably. Thank you Cycling!
Also definitely check out the forums :: they have been invaluable to me for years, being the source of many fresh ideas (not to mention clever techniques or solutions to problems). There's the complete range of experience and expertise there, and many example patches get posted all the time. My username on the forums is seejayjames.
If you do get into using Max, do yourself a huge favor and take *extra time* to read the manuals, try the tutorials, check out every help patch as you use new objects, and read what messages can be sent to each object. Don't worry what you do or don't understand at a given point; the more possibilites you try, the more connections you'll make between your ideas. Don't fall into the trap (as I continue to do in spite of myself) of too much bit-by-bit experimentation. Some of this is obviously necessary to understand how each piece works, and lots of times you just want to play around with a very specific idea (like testing the range of effects on a piece of audio). Ideally, creating patches should approach 90% thought and 10% programming, where you know what you can achieve and how, then you think through the best way to do it before you grab the mouse. Once it's made, a good patch is 98% play and 2% tweak.
When you get small functions working, make them nice-looking and Save them into a "snippets" file. Then use copies of it in your patches as you need them. This really will save a lot of time, but it will also help you understand an absolutely essential concept :: processes can be as modular as you want. In other words, how you connect controls together can exist in any configuration, but at each point of connection, *that object is simply doing it's thing*, based upon the interaction or messages it receives. And even more basic than that, it's always operating on numbers :: single numbers, lists, streams, numbers that represent text / pixels / samples / MIDI notes / 3D positioning / effect levels / and / pretty / much / everything / else ... so if you understand how to work some number magic, you'll be a true DJ ... ... data jockey!
