Spear

I'm tired of Max/MSP and PD. Graphical programming just isn't as powerful as programming, and at the moment it isn't that intuitive either. So I've been looking for other software to use, and one of them I've found is called spear.

Spear uses fft analysis on any wav file of your choice to graphically display all the frequencies that make up a complex sound. Sort of like ms paint for sound, you can then manipulate the waves any way you desire.

On the y axis is the frequency of the sound, and the x axis is time. The darker the line, the louder that frequency is. Of course, the analysis itself isn't perfect, but it's more than enough to be able to mess around with the sound. You can select harmonics, scrub, pitch shift, and time stretch (among other things).

I mention spear because I was originally going to use it in conjunction with cubase to write this song that I'm working on. It turns out that my samples (namely the human voice) are really hard to transform with fft analysis. Even though they may not make it into the final cut, here is a voice sample and its manipulation in spear.

Post Semester Wrap-up, Pre Summer Ramp-up

Played for Cornell's Graduation, flew to New Mexico to pick up my car, and drove to Mountain View, CA. Started work this week at Palm, working on devices I'm not allowed to blog about because of three separate confidentiality agreements. That's a shame.

Instead I've finally uploaded some of the CU Wind Ensemble concert from spring semester. Overture to Candide was conducted by a guest conductor (and as a result was really slow imo). The piano concerto was composed by Norbert Palej, and this was it's premiere performance. Fog tropes is actually an electric piece with a brass sextet (2 trumpets, 2 french horns, a trombone and a tuba). The result is pretty cool... it's hard to tell what is in the recording and what was actually performed live onstage.

Just ordered my midi controller, so I expect it should be here by the next post. I'm so excited :)

Nothing New to Report

Just finished finals last week, so I wasn't in a position to churn out anything very creative. I'll just let you know what's in the works.

I tried to upload a video of my second project for music class, but ripping a dvd into avi takes more skill than sb had time to learn this week. It's too bad: the project consisted of me playing the tuba, Ricardo playing the electric guitar, Stephen playing the midi keyboard, Cameron manning the computer, and Henrik 'playing' the wiimote. The wiimote was hooked up to some special graphical and particle effects using PD's Gem. It was a spectacle.

Speaking of Cameron, his final project for the class is on youtube, music thing, and gizmodo. Props to him for that. Where's my project? I didn't get to perform it due to technical difficulties. Transferring my patch from Max4.5 on PC to Max5 on Mac turns out to be pretty near impossible. (remind me to rant about how much I dislike Max/MSP later)

Our video game Stage IV is complete and the showcase went well. As soon as they put it online I'll post a link so you guys can check it out. I just had a tuba recital last week, and we (hopefully) got that on tape, so I'll be looking into that as well.

Looks like I didn't get a midi keyboard controller for my birthday, so I'm gonna have to purchase one myself for the summer. If anyone has one they love, let me know about it (I'd rather not spend more than $200)

Numerical Analysis Renders

End of the year projects are coming and going. This one in particular is for my numerical analysis class. Here we use ODE solvers to guesstimate where particles will end up in a vector field. That's boring, but the results are very pretty pictures


Oh, and here's our particle video, made in Matlab ^__^


ODE Particles from esbie on Vimeo.

Melodyne: I'm not Impressed

When I first saw Melodyne, I had the same reaction that I had towards Guitar Hero: an emotional one...almost a little offended even. DDR doesn't make you a dancer, and Guitar Hero doesn't make you a rockstar (also: being a rockstar doesn't make you a musician).



If you haven't seen Melodyne, you should check it out now. Melodyne decomposes any audio file into distinct voices using Fast Fourier Transform (fft) analysis. You can then use their editor to distort these voices in any way you choose. The most popular transformation? Auto-tune.

Let me explain why I don't like Melodyne.

The Hype: people are hailing this as a new technological breakthrough that will speed us into the new age of amazingness. Let me inform you that fft has been around for decades. People are entranced that Melodyne can pick out individual instruments from the mix, but that's not very mind-blowing, considering each instrument has it's own unique timbre and wave structure. Melodyne can't distinguish between two of the same instrument, as we might expect (Sorry folks, we haven't taught the computer any counterpoint yet).

The Abuse: people are going to use Melodyne extensively and explicitly for autotune. Listen, if you can't sing on key, there are other ways to fix that... like singing lessons. FFT will produce artifacts, no matter how good the software is. Visit my tumblelog for examples of blatant autotuning.

What do I think is awesome about Melodyne? The interface gives musicians leverage. Cubase is to Audacity as Melodyne is to Spear. Spear is a dead simple fft decomposition and resynthesis software. Everything you can do in Melodyne you can already do in Spear (Spear is also free, like Audacity). It's the interface that can really make this software powerful.

What else do I like? Since I'm into the world of electronic music, I plan to exploit Melodyne in a different way... by exaggerating its artifacts or transforming audio samples into something completely unrecognizable, not by pretending that I can sing.

Fewer Words, More Pictures

Look Up by ~esbie on deviantART

Diana.setEmoState(5);

I've been hella busy this semester, and finally you get to see one of the reasons why! Above is a composite of just a few of the facial expressions for a character named Diana (played by Cornell's RPTA, Carolyn). These, and many other character pictures will be used in the game that I'm working on for Advanced Game Design called StageIV.

Stage IV is a story about a father who is diagnosed with cervical cancer, and how the family deals with the knowledge of his approaching death. The players chooses which character to play as and then interacts through dialogue graphs with other characters while the story unfolds.

I say graphs because trees become exponential in the amount of content needed, and a content heavy game isn't really feasible in one semester between Ben and I. Although your choices within the dialogue don't directly affect the story's outcome, your words are highly influential in how the other character feels towards you and in general. Thus the other characters (NPCs since it's a single player game) can react in a multitude of ways depending on their mood.

Doesn't sound like the most exciting game you've ever played? It's not supposed to. We're thinking of calling it 'Interactive Fiction' instead, so that the intention is clear. Ben and I are more interested in creating an artistic experience for the player than an action packed game.

Ok so what on earth am I doing for the game? All the photographs you see were taken by my Sony Cybershot. I then went into photoshop and cropped out every single background so that it's only the character. Then after some resizing, I also cranked up the contrast and turned down the saturation. Original is on the right:


The reason I crop the photos should be obvious; I want to place these characters in several different locations in the game, not just one. Resizing should be self explanatory too. I need the transitions between facial expressions to be as smooth as possible, so it feels like you're looking at a person, not a bunch of pictures of a person. Then of course they had to be scaled down to fit onscreen.

But why the contrast and desat? The desaturation is partly a design choice: Ben and I wanted the game to have a certain feel. But additionally, some of the base pictures have different amounts of lighting. Desaturating the picture helps hide that inconsistency (which means less work for SB). And why the contrast? The effects of the contrast are hard to see, but I found that it just helps the desaturated image from becoming too blah.

I'm also doing music for the game. Future post in the future :)