... .  .    .       .joeberkovitz.com

Audio-enabled Physics Engine Example

I thought I’d put up a fun demo SWF from my “Flash Bang” talk of a couple of weeks ago. This is an example of what can be done with the new FlashPlayer 10 (”Astro”) Sound API; it synthesizes sounds on the fly that are synchronized with colliding objects in a simple physical simulation. The volume and pitch of the sounds are controlled by the collision force and the nature of the colliding objects.

To run this demo, get hold of the FP10 standalone player, then use it to open this URL:

http://www.joeberkovitz.com/projects/StandingWave/physicalSound/physicalSound.swf

(If you try this with FP9, you’ll get a runtime error dialog, or the app won’t open. And at some point soon the FP10 API will probably change, invalidating this app and obliging me to rebuild it, so in the near future YMMV.)

The sync between the sounds and the visuals is quite delicate and if it is off by even a little bit, the results look terrible. On my MacBook Pro, the auto-calculation of the audio latency is near perfect, but on Windows the player API’s Sound.position property seems to be off by about 150 msec, enough to ruin everything. You can play around with the latency in this example by moving the slider.

I didn’t post source because I am using a proprietary library, but the sound code in the example is not very interesting or complex actually. The trickiness in this app is all in the audio/visual synchronization. I am actually running two identical physical simulations, one of which runs ahead of the other, generating each collision sound exactly the right time interval before the other simulation displays the same collision. You can see both simulations by clicking the checkbox in the upper left of the example.

I used an open source physics engine called APE (Actionscript Physics Engine) available, at http://www.cove.org/ape/index.htm. I picked it because it was very easy to modify the collision detection mechanism to add audio hooks, but I haven’t done enough research to compare it to other engines for this purpose.

Here’s a screenshot (without sound of course!):

No Comments

“Flash Bang”: upcoming talk on Flash audio synthesis

This Wednesday evening (5/28), I am giving a talk on bleeding-edge Flash audio and music synthesis at the Boston Flash Platform Users Group . Jeez Louise, I must be about the worst self-publicist ever… I’ve been so busy that I forgot to blog my own speaking event. Anyway, the meeting is at the NEIA Annex building, 303 Boylston Street, Brookline MA at 7 pm.

I think this will be a fun one — my computer will be making a lot of noise and playing some tunes. Some of the highlights include:

• The strange and scary things you have to do to synthesize audio in Flash 9
• The big audio improvements in Flash 10 (code and demos will be shown!)
• Architectural approaches to dealing with latency and synchronization
• Adding simulated sound to a physics engine
• The brave new world of Flash applications for creating music, not just playing it back

Hope you can make it!

2 Comments

StandingWave: Live Musical Audio Synthesis in AS3

I’ve been working on a funky new homebrew software project called StandingWave: a musical audio processing engine built entirely in Flash. My goal in doing this is to explore a world of online interactive music applications in which audio is not merely played back, but generated on the fly — performed, in fact — from an underlying representation of musical events. Such applications might range from a traditional music notation editor to game-like music composition environments to… knows?

Computer music performance is hardly something new, of course. But embedding the capability in Flash, at this point in the world, can make musical applications accessible on the web and amenable to community use in a way that’s never been possible before. Think about what applications like Buzzword and Google Spreadsheets are doing for traditional “productivity apps”.

I’ve started with the audio engine because it’s an interesting technical challenge, although I’m working on some of the other pieces concurrently. I’ve put up an initial crude example that demonstrates sample-based waveform synthesis. This toy application can play back single notes, a chromatic scale and a sample MIDI file at various transpositions, tempos and volumes, and all of this is accomplished by actually synthesizing digital audio signals on the fly, starting from a set of recorded guitar samples and applying gain envelopes, frequency shifting and mixing. Musically, it’s hardly exciting, but it’s a start on the capabilities needed to concretely deliver music in the Flash Player with no external add-ons, and without leaning on the crappy, highly variable MIDI playback delivered by the browser’s native OS.

There’s no waveform audio output API in Flash, so how is this done? Read on…

[Continued...]

15 Comments