Today, Apple released a beta of their iPhone OS version 4.0; in order to access the release and start developing their applications for it, iPhone/iPad developers were required to accept the following restriction:

“3.3.1 … Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).”

This is a fascinating and wacky position. Much of the outrage over the above statement is coming from developers who would like to use tools such as Adobe Flash CS5’s iPhone packager, Unity and others to create applications — these tools produce intermediate translations from their own representation of an app into a form that is at some level compatible with Apple’s own standards. With this restriction, Apple appears to be attempting to outlaw the use of such tools, presumably to monopolize not only the ends but also the means of iPhone development. One can only speculate on the possible reasons: an Apple/Adobe vendetta-in-the-Valley, a mistaken belief that consistent developer tooling means a consistent user experience, a desire to control every aspect of the value chain.

Personally, I think the outrage should be over the metaphysical angle here, not the business angle. Apple is implicitly taking a position that apps are not “originally written” in the minds of developers, in the form of cognitive representations of problems and their solutions. They are taking a position that the brain is not a translation tool for mapping from these representations into C, Objective-C, or what have you. They are subscribing to the theory of a “ghost in the machine”, implying that at some point an app crosses some magical boundary from being an mental thing into a physical thing that is “written” in some definite programming language. They are maintaining this because, if they weren’t, every single iPhone app would violate their licensing agreement by virtue of the developer’s mind itself being a tool that produces Objective-C as an “intermediary result”. Apple may thus be the first company to bet the farm on Cartesian dualism.

If this seems like a really nerdy joke (which it could be), try a few thought experiments. What if Ben writes a Flash app, shows it to Amy, who codes it up in Objective-C, compiles it and submits it to Apple? Should it be rejected since it was not “originally written” in Objective-C? If you think Apple’s answer would be “no” — a good guess — then substitute Adobe’s iPhone Packager for Amy. Now should it be rejected according to the rules? What, at the end of the day, makes Amy different from a machine translation tool? (Personally, I’d rather hang out with Amy than with iPhone Packager, but that’s another story.)

You may be thinking that what makes Amy different is that she could at least in theory translate the idiom of the original application to the idiom of the iPhone, which would provide a better user experience. Maybe so — so let’s look at another example. What if Ben’s application is an adventure game, and he wishes to describe the behaviors of the game objects and rooms using a simple scripting language that is translated to C as part of the process of compiling his game. He’s not using Flash or Unity or anything like that. Coding game-behavior scripts in C is a pain in the butt, and Ben’s a better programmer than to waste his time on that. He’d rather create the ideal one-off translation tool for the job. Sorry, Ben — clause 3.3.1 says that your program must be “originally written” in C, not translated to it. Maybe you should hire Amy to translate your scripting language by hand, that might be acceptable (since Amy will be “writing”).

Doesn’t this start to seem like backward progress, rather than forward progress?

Programs aren’t “originally written” in any definite language, or in any definite location. Programs are encoded information that is contiguous and continuous with the information in our heads, and in the world. Programs are often generated by other programs and tools, to which the same difficulties in definition apply. My programs are originally written in the shower — at least according to my definition. We could engage in a long argument about whether these shower-programs are really “code” or not, but my main point is this: it seems silly for a company like Apple to take a definite position on exactly what “originally written” and “intermediary translation” mean. (They declined to define these terms in the agreement, of course.)

I leave you with a final quote from the license agreement:

“10.4 Press Releases and Other Publicity. You may not issue any press releases or make any other public statements regarding this Agreement, its terms and conditions, or the relationship of the parties without Apple’s express prior written approval, which may be withheld at Apple’s discretion.”

Oops, close call. Good thing I didn’t click “Accept”.

Libby Freligh was the senior product manager for the Flex platform at the time of its introduction by Macromedia, and the Boston Flex User Group is lucky to have rounded up Libby as a speaker this coming Tuesday, October 13 at One Broadway, 5th floor in Kendall Square, Cambridge.

This won’t be a techie talk for a change: instead, this one’s all about the business landscape inhabited by Flex then and now. If you don’t know Libby, I can tell you this: she is a speaker with a lot of humor, intelligence and energy. Not only will you not be bored, but I expect you will be treated to some inside-type material about the origins of Flex that you’d simply never hear anywhere else. I think it’s fair to guess that you will walk away with a broadened perspective whether you are a technician, a business person or just plain curious.

Hope to see you there!

“Won’t anyone think of the children?”

Oh, snap. Apple may be our techno-culture poster child for great design and great execution, but they appear to have flubbed it badly while trying to keep the children sitting in front of its computers safe from whatever’s out there.

We recently were dealing with a support call concerning a number of Noteflight scores created in a particular school classroom. These are XML documents, and it turned out that they had been corrupted by the injection of web proxy HTTP headers in the middle of the document, rendering it unparseable. The corruption now appears to be the handiwork of Mac OS X 10.5’s Parental Controls option, which has been reported to have a buggy interaction with Firefox that can insert this garbage into the content of any HTTP POST. And indeed, all the cases we found occurred with Firefox (various versions) and Mac OS 10.5 with Parental Controls in effect. Safari reportedly has no problem.

The garbage typically takes the form Proxy-Connection: keep-alive\rCache-Content age=0 and appears a little under 1500 characters into the POST — probably not coincidentally this is approximately the length of a network packet. Parental Control’s web aspect is implemented as an HTTP proxy server, no doubt one with a bug in it.

What’s really amazing is that this bug has been around since the beginning of 2009 or so, and little has been said or done about it. But if you Google for the garbage, you’ll see that this garbage has made its way into every corner of the web.

Now, that’s what I call viral!

You heard it from me: these folks are writing some of the most badass music-related code on the planet.

The Echo Nest are experts in “machine listening”: they have developed a set of algorithms that crunch through raw audio media and extract a set of distinctive musical features. These features roughly describe what is happening in the music at a hierarchy of time durations (beat, measure, section), and from the features they can compute a notion of similarity between different pieces of music. This similarity metric drives the recommendation aspect of their business.

Naturally enough Noteflight and The Echo Nest have some mutual interests, hence our visit. Audio media and music notation are both descriptions of music, so our companies both think a lot about how those descriptions are related. It’s a tough problem to go from either description to the other, and no algorithm can perform either task anywhere near as well as a human musician.

Anyway, while I was over there they showed off a very cool music hacking tool called Remix which you can grab from Google Code. It’s basically a Python library that takes an audio file, analyzes it using Echo Nest wizardry, and then returns a data structure describing the audio down to the beat level. You can then mess with these beat-length samples based on their descriptive data, and reassemble them in bizarre and unexpectedly musical ways.

As an example, they played me a version of “Here Comes The Sun”, in a strangely filtered version in which only beats in the same key as the opening intro had been retained. The result was a odd, drone-like modification of the song in which the intro itself was intact, but then unfolded into a sequence of snippets from the song that were completely familiar but from which all harmonic motion had been precisely excised.

I then heard a Hall and Oates song that had had beats 2 and 4 surgically removed from every measure. The result? A weird double-time version in which the song form progressed at twice the normal speed, the lyrics were mostly unintelligible but with many recognizable syllables, and the entire song’s length was chopped in half. The latter aspect could be viewed as an improvement on the original.

As a code-on-the-spot challenge, I asked if they could put together a version that sorted all the beats by amplitude, putting the softest ones first and the loudest ones last. 5 minutes of Python hacking later, we were listening to a bizarre, long crescendo of segments from the song, seamlessly reassembled into a whole. The beginning consisted mostly of quieter instrumental chords or the unaccented syllables of words, while the end was a kind of synopsis of all the climactic moments in the song with kick drum or vocal accents. The whole song turned into a single musical gesture, reassembled from its fragments into something completely different but still wholly familiar sounding.

After months of waiting for an opportunity to open up in my schedule, I’ve finally managed to create and package the StandingWave2 audio synthesis engine for Flash. It’s now up on Google Code at http://code.google.com/p/standingwave/. Phew… about time!

People have been asking for an open source audio library for a long time. Because the original StandingWave1 became an integral part of Noteflight, I could not simply give the code away. StandingWave2, on the other hand, is a clean subset that the Noteflight team is happy to make available to the Flash community under the MIT OS license.

The basic ideas behind StandingWave are sources, filters and performances. Sources and filters are simple, self-contained objects that can be hooked up to create a kind of audio-processing/sequencing pipeline, and then rendered by a “player” object that encapsulates the Flash Player 10 audio API. Performances allow source/filter combinations to be delivered in a continuous sequence, with extremely precise timing. Obviously this can be used to play music, but it can also be used for all kinds of dynamic sound creation. And it’s relatively easy to extend StandingWave to add your own kinds of sources, filters and performances once you read the code and get the idea.

All the DSP algorithms are in pure AS3. They would certainly be faster in PixelBender or Alchemy, but we’ve opted to keep the approach simple and flexible for now so that it’s easy for people to extend.

I will be talking much more about StandingWave at 360Flex and Flash on Tap, so hope you can make it to either of those conferences. I’ll be posting the slides here.

About its quality and performance: as useful as this library is (thousands of people use it on Noteflight), it could be so much better. It’s fast, but it should be faster; it has a basic repertoire of sources and filters, but should be richer. The need for improvement is one of the main reasons we’re open sourcing it: we very much want others to contribute.

Happy audio coding!

Important update: After a lot of amazing work by Max Lord, who rearranged SW2’s internal organs to create a new, bionically altered library, StandingWave 3 was released in June 2010. This update to StandingWave uses Adobe Alchemy to compile C audio processing code directly into the Flash Player for maximum speed, and is generally much faster and more flexible than Standing Wave 2.  It also includes a notion of modulators that allows for continuous variation of many filter parameters.  Like this idea? I do, which is why it’s now the production audio library in Noteflight.  Go to the StandingWave 3 repository on GitHub.

http://www.vilnashul.com/calendar/event/the-state-of-startups-using-cloud-computing/

This should be interesting; I know a couple of the other companies represented on the panel, and I am sure some diverting and unpredictable discussion is in the works! Also I’ve never been to the Vilna Shul, a much-neglected historic synagogue in Boston.

I’m showing a few cool demos of this technology at the conference, including a Moccasin-based visual editor for a “musical shapes” world in which shapes represent tones, and a set of data entry forms that progress smoothly through an accompanying musical form as the user navigates. It’s been loads of fun working on this stuff, which is to say it’s not anything like work at all.

I guess I have to go back to work now.

Gosh darn it.

http://joeberkovitz.com/projects/FITC2009/BuildingVisualEditors.pdf

Overall, I am really enjoying this conference purely as a conference. It’s a much broader mix of creative and technical people than I usually see at software-related shows, the organization is great, and the quality of the presentations has been overall very high. A lot of the best Flash coders in the business are here, as are some folks from the Adobe Flash Player team. It’s a great bunch of folks to hang out with and talk to. As one highlight, it was nice to meet Robert Penner, a well-known name in tweening-weenie circles and one of the few other people who has worked on music notation in Flash (he did a lot of work on the mediaRAIN music viewer).

Toronto is a pretty hip and fun city too, as I am finding out. Maybe more on that later — although I didn’t bring my camera, stupidly.