Progression

Getting the Wiimote recognized on a windows machine was an interesting experience, let's just leave it at that. Then, after much work trying to get the tk.wii patch I gained some success only to realize that calculation of the angle of the points the Wiimote is tracking is not provided. In fact, the Wiimote in it's intended usage never uses the IR leds as a reference for rotation. Doh! This is all done through use of the accelerometers which makes sense...if you're moving the Wiimote. Since I'm interested in moving the actual leds and the tk.wii patch provides no data concerning the position of each point being tracked I had to look elsewhere.

I just happened to be trying out the GlovePIE application (for programmable input emulation) with the Wiimote to see what functionality it had when I came across documentation about how to get the individual point data I needed. Not only that, it could provide the size of the points being tracked which could also be useful. Nasty! From there it was just some simple trig to find the angle between the two points. Handily enough, GlovePIE supports the same OSC protocol that Max also has externals for, so getting the data into max isn't a problem.


Now that I have the angle of the LED's piping into max, the data is playing nicely with the panhandler patch which allows setting the rotation of a sound source.


Everything seems to be working as intended. Now it's a matter of selecting sounds and deciding how to manipulate them so I can start on a more solid implementation. Speaking of which, I made a special hat for head tracking. It's mostly for looking cool but it's going to help with development a lot.

Technical Progression

As of now various hardware bits are on their way in order to make this project a reality. Making their way to my house are a set of speakers, a wiimote, a bluetooth adapter, some infrared leds and some AA battery holders. This didn't put me out too bad considering I was in need of speakers anyway.

In practice the use of the wiimote with max/msp should not present too much of a problem. Masayuki Akamatsu's aka.wiiremote patch will handle interpretation of data streaming from the wiimote. There is plenty of support for this.

This rotational data from the wiimote can be fed into any number of multi-point panning patches that I've come across. The best I've found so far is Michael Zbyszynski's Panhandler~ patch which in conjunction with the VBAP object makes setting up multiple outputs and panning a sound through them pretty painless.

All of this gives me a pretty good start on the actual implementation of the back end of the project. The only real complications may come from trying to manipulate the sound in a customized manner that does not involve simply following the user directly. Some other ideas that I'm toying with that will complicate things (yay) include 3D panning of sound, ie an additional speaker above the person which will allow tilting of their head to correspond to alterations in the panning. Also, I'm very interested in allowing the user to effect the actual audio indirectly through the nature of their movements in order to add another layer of interactivity and weirdness to the project.

New Project: Auditory Illusions

Proposal for project #2/possibly 3 can be found here

This project formalizes ideas I've been toying with for some time. The basic aim is to create an environment in which someone can experience disorientation through sound manipulation. To accomplish this I will be making use of surround sound and overhead head tracking by way of the wiimote and two head mounted infrared led's. Based on a person's orientation sound will be panned through the speakers to either give the sensation that their movement has no effect on their audio-sensory experience or that this experience is unreliable and/or not to be trusted as a true representation of their surroundings. Those experiencing this installation will be blindfolded to produce a reliance on sound to determine their positioning.



For now this project is as much curiosity as artistic ambition. I'm set on creating a specific experience; one of confusion and disruption. Beyond this the idea of auditory illusions is the driving force beyond this experiment. It is my hope that through experimenting with this system new insights will arise and allow it to progress further into a more fleshed out idea.

Project 1 Development

So far I've managed to successfully connect and use a distance sensor and stepping motor with the Make Controller. The motor progresses to precise numeric positions based on the distance of an object to the sensor. More soon!

Project 1 Musings

So I've been developing ideas which will make the second idea I proposed below more physical. Below are links to a few sketches.

Basically a small plant/tree will be placed on a table and made to react to the presence of viewers by way of distance/motion sensors and motors. These motors will be attached to the tree or plant with fishing line and will cause both movement and shaking depending on how they interface with the fishing line. Think of it as puppetry. I propose two versions in my sketches. The first, the most simple, uses 1 sensor and 2 motors for only one direction of movement and shaking. The second uses 3 sensors and 4 motors for more realistic movement in 3 directions as well as shaking. All sensors and motors will be placed beneath the table so they're not seen and the lines will be as hidden as possible.

The idea is that this plant/tree will react to the user in a way that is appropriate to its fragile and subservient position to those who interact with it. It cannot run or hide and is at the mercy of those around it and therefore is quite scared. It will shy away from those around it and start shaking if you come too close. It make even react audibly to your presence. I may have to use a fake tree to avoid the unintended message that I am in some way trying to turn a living thing into a puppet although that could be another interested take on this project.

Ideas and Musings

So it is definitely easier to develop ideas for a medium you are familiar with. I've had some trouble coming up with physical computing ideas simply because I don't really understand the full range of possibilities. I can propose a few though:

1. Rework the Drum Circle arcade game/interactive table that was developed by Dylan Moore, Andrew Gardner, and myself. It never really became a game and I'd like to get it to that point as well as make a more manageable and portable version. Also, we never implemted a solid form of beat detection which would be crucial to it operating as a game. I've already noticed several Max/MSP scripts that function as beat or even rhythm detectors. I also have a dead toaster lying around that is just dying to become a game 'cartridge' reader.

Original Drum Circle

2. Interactive Tree Studies installation. The 3d trees I created last semester are begging to be made into an interactive piece. My first inclination is to use motion sensing to allow the mere presence of a viewer in the same room as it to cause a tree to start failing in health and such, shown by diminishing posture and wheezing, etc. As soon as the viewer leaves the installation area the tree starts to recover its health.

3. The last idea I had would be to create an intuitive game controller for the concept I proposed in CAR340, regardless of whether or not it gets picked to be worked on. The game is designed to operate using some sort of kinetic input (proposed as the wiimote) but could really stand to have an even more physical and realistic simulation of the motions displayed in game. The game characters are battling, vertically oriented, plants which operate in a spring-like fashion, duking it out using darts and dodges. They never leave the spot they start on and simply swing about. The controller I would develop for this game would be a real world interpretation of your character (a dolled up spring?) which utilizes motion tracking along two axis' to communicate where the top of the spring is at all times in x and y coordinates. This info would be relayed to the game engine which would translate the characters 'head' to it's correct position, allowing the darts and dodges to be controlled directly. If the actual game was not chosen to be worked on I would have to create a simplified version which mostly demonstrates the use of the controller.