Controlling Audio Latency in Flash 10

October 15, 2008 on 12:04 pm | In Flex, Music, Programming | 13 Comments

Adobe threw a slight curve ball with the long-awaited release of Flash Player 10 today.  The release itself wasn’t the curve ball (although not everyone expected it to happen on thisdate).  The surprise was the audio latency using the new dynamic sound generation APIs.

Audio latency, for programs that synthesize sound on the fly, is roughly defined as the amount of time between the program creating a sound and the computer’s audio output actually playing that same sound.  Latency is important because it affects the user experience.  If a user hits a “play” button and waits 1 second before hearing anything, that can be annoying.  If a user types a note into a music notation editor and has to wait 1 second before hearing anything, that’s downright maddening.

Latency isn’t all bad, though. The higher the latency, the more “give” the sound output pipeline has, and the more resistant it is to disruption by other processes running in your browser and on your computer. Ideally you want to strike a balance between latency and robustness in your sound output.

In Noteflight, the latency on Mac OS X had been at a comfortable 30 milliseconds — about a 30th of a second — using the Flash 10 beta.  But in the released production version, Noteflight’s audio latency suddenly jumped to about 850 milliseconds.  Other folks with audio synthesis apps reported other numbers, all of them different.  Whoa!  What happened?  (Before going further, let me say that our latency is now back at 250 milliseconds, so things aren’t as bad as they first seemed.)

Adobe introduced a new behavior into the released player, where the latency depends on the number of samples you provide to SampleDataEvent. If you provide 2048 samples per callback, your latency will be the minimum available on the platform (30 ms on Mac, ~250 on Windows). If you provide 8192 samples per callback, on the other hand, your latency will be in the neighborhood 1000 milliseconds. The amount of latency for in-between sample block sizes is a complex function of this number. Tinic Uro will hopefully publish more information about that function, which apparently involves a power-law nonlinear function of the block size combined with the native sound driver buffer size, but experimentally I have seen these numbers on Mac OS X:

block size latency (ms)
2048 30
3072 170
4096 250
8192 850

This is good stuff to know, and it’s something of a surprise to us all. I notched the sample size in Noteflight back down to 4096 in a hurry, and latency is acceptable once again. Or, semi-acceptable. I’d like to get it even lower, but I don’t want to do that in too much of a hurry without adequate testing and tuning of the synthesis pipeline.

[Important footnote (10/16/08):  You might think that by adjusting the sample block size dynamically, you can change the latency on the fly for a single Sound object that is streaming samples.  The answer is… sort of, but not in a useful way.  You can make the latency increase by making the sample block size larger, but if you make it smaller again, the latency will not go back down: it will stay at the higher value.  What is reported to work, but I haven’t tried it yet, is to output sound using more than one Sound object, each with its own block size (and hence its own latency value).  The usefulness of that approach is also not clear.]

13 Comments

13 Comments »

RSS feed for comments on this post. TrackBack URI

  1. […] actual sound. It is important to keep this value low for live experience. After hours of research some people figure out, that the amount of samples you pass to the sound card is changing the latency […]

    Pingback by Andre Michelle » Blog Archive » FP10 SoundAPI changes — October 16, 2008 #

  2. […] information Andre Michelle blog Joe Berkovitz blog Kai-Philipp Schöllmann […]

    Pingback by Adobe made some creepy noise at Adobe, MAKE SOME NOISE — October 16, 2008 #

  3. […] with my local beta standalone player yet. the release version is out since yesterday or so and the bad news already spread – there are again changes to the sound api causing huge latencies depending on your […]

    Pingback by tickytickyturntable — October 16, 2008 #

  4. […] die breite Masse, sondern eher für 0,0001 % der Entwickler, also André, Joa und einer Handvoll anderer Entwickler. Aber egal, man freut sich über den Erfolg der […]

    Pingback by Blog von Jens Franke » Das Arbeiten in einer Welt voller APIs — October 16, 2008 #

  5. […] http://joeberkovitz.com/blog/2008/10/15/controlling-audio-latency-in-flash-10/ […]

    Pingback by Dealing with latency in FP10 : pixel to noise — December 19, 2008 #

  6. Hi
    We wanted a new version with dynamic sound and we got it. But I ‘m also not satisfied with the very long audio latency using the new dynamic sound generation APIs. Ok, I can unterstand the fact that the version is only the beta one not the final but we have expected a better version from Adobe.
    I think Adobe should work on decreasing the Audio’s request-response time, which is very important especially for any electronic musician.

    Comment by Flüge USA — February 20, 2009 #

  7. cool… And can we save the generated audio in a WAV file? Is that too much to ask?

    Thanks,
    Paul

    Comment by Paul — March 17, 2009 #

  8. You can indeed save the generated audio in a WAV file if you write the code to do so! Noteflight does this, using a Flash 10 FileReference.

    Comment by joe — March 17, 2009 #

  9. and when I said correct I of course meant “what Adobe has suddenly decided is the correct sample size without telling anyone about it advance”!

    Comment by Gummistiefel — May 26, 2009 #

  10. Hi! Outputting sound with more then one object seems to work, i just got the same problem on one of our sites. Regards from germany, marco

    Comment by Marco Pe — August 18, 2009 #

  11. Hey, i got the same problem than Marco on one of my site. The solution seems to be taking more than one object, then it works. greetz

    Comment by Peter — August 25, 2009 #

  12. Hi, I propose to check the soundcard settings. Maybe you’ll achieve improvements then. Best regards

    Comment by Etikettendrucker — November 2, 2009 #

  13. Great article :-) Thank you for posting and best regards!!!

    Comment by Netyu — March 11, 2010 #

Leave a comment

XHTML: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Entries and comments feeds. Valid XHTML and CSS.
All content copyright (c) 2006-2007 Joseph Berkovitz. All Rights Reserved.