... .  .    .       .joeberkovitz.com

Going Back to Cali

Although not exactly the Cali represented in LL Cool J’s immortal deadpan number.  (Also check out the wonderful sax solo at the end, playing a minor 3rd up from the key of the tune.)

On Friday, I fly to San Francisco, rent a car, drive to San Rafael, meet spouse who is already there, and then we’ll spend 6 glorious days moseying around the wilds of the Mendocino County and northern Marin coastlines.  Goodbye, work, for a while.  I predict there will be some kind of trip report!

No Comments

Live (or not so live) on The Flex Show

Starting today, you can catch me on episode 63 of The Flex Show, in which the effervescent John Wilker and Jeffry Houser interview yours truly about Noteflight, Flex, and I forget what else — it was taped a few weeks ago, which feels more like several years given the wealth of activity that was packed into it. I think I might take a listen right now, to find out the exact manner in which I embarrassed myself.

1 Comment

A Flash “Clipboard” using Local Shared Objects

I spent a good part of yesterday implementing a new global clipboard feature for Noteflight using local shared objects or LSOs.  These cookie-like constructs are often used to persist application-specific data across multiple sessions, so that when you close an application down and then restart it later, it resumes in the same state you left it.  What I haven’t seen much of (and haven’t done before myself) is use LSOs to communicate data between completely different instances of the same application running at the same time.  It does work, and seems to work well.

Noteflight is a Flash-based music notation editor, and one of the most often-requested features has been the ability to cut, copy and paste between different documents.  These “clipboard” operations already worked within a single session editing a single document.  However, the clipboard contents did not survive across edits to a series of documents,  nor could they transfer between different documents open in different windows.  In effect, each copy of the application had its own private copy of the clipboard, initialized to emptiness.  Not surprisingly, users don’t like these limitations.

It seemed clear that LSOs would be a good vehicle for this enhancement.  We would write the clipboard to an LSO on every cut or copy, and read it out of the same LSO before every paste. This would not be exactly like a real native system clipboard (since it would be separate from the system clip that contains text, images, etc.) but for the purposes of Noteflight it would seem a lot like one. The challenges we expected to solve (and did) were as follows:  serialization, keeping LSOs small, handling LSO failure gracefully, and application version skew.  There were of course a couple of major surprises, too!

Serialization.  All LSOs are stored in files, and must serialize their data.  They do so using the AMF3 protocol. ActionScript 3 defines how one can manage the serialization of arbitrary Objects using the [RemoteClass] and [Transient] metadata tags (be sure to look those up if you are thinking of serializing objects in a Flex application).  I’ve used them before, but the special problem here was that the clipboard data structures had already been designed and were quite complex, with a number of potential references to objects sitting “out in the application”.  Serializing any of those would result in lots of irrelevant data getting “dragged in” to the LSO and bloating it, so they’d have to be marked as transient.  It would be also necessary to make sure that each and every class requiring a [RemoteClass] tag received it.  Getting all this right in a short amount of time seemed unlikely.

Fortunately I had plenty of existing code to serialize the objects in the clipboard to XML and back, since Noteflight persists musical scores as XML.  So I used Flash’s handy IExternalizable interface to take advantage of this. This is a great technique that allows you to decide how your objects will be serialized. Basically, what I did was kind of like this:

[RemoteClass]
public class AppClipboard implements IExternalizable
{
  private var _data:AppData;  // contents of the clipboard...

  public function writeExternal(output:IDataOutput):void
  {
    output.writeObject(XmlWriter.toXML(_data).toXMLString());
  }

  public function readExternal(input:IDataInput):void
  {
    _data = XMLReader.fromXml(new XML(input.readObject()))
  }
}

(Note the [RemoteClass] tag — you still need it even if you use IExternalizable, or Flash won’t reconsititute the class of your object when you read it back in again.)

With this code written, when you setting one of your shared object’s data properties to an AppClipboard, like this:

  mySO.data.contents = myAppClipboard;

then the writeExternal() method is implicitly used to serialize that object. Same deal goes when reading the shared object back in: readExternal() gets used.

Next challenge:

Keeping the data size low. since there is a default 100K limit on LSO storage. XML has its disadvantages, and verbosity is certainly one of them. I needed to compress my serialized objects somehow. I found a nice way to do that, which works for any serializable objects at all (whether you use IExternalizable or not). Here’s the “write side” of the technique:

  var so:SharedObject = getClipboardSO();
  var bytes:ByteArray = new ByteArray();
  bytes.writeObject(_clipboard);   // write our object to a ByteArray
  bytes.compress();    // compress the ByteArray
  so.data.contents = bytes;   // and write the ByteArray instead
  so.data.version = NotationXmlReader.CURRENT_VERSION;
  so.data.creationTime = _clipboardCreationTime;
  so.flush();

The “read side” works similarly:

  var so:SharedObject = getClipboardSO();
  if (so != null
      && so.data.version == CURRENT_VERSION
      && (_clipboard == null
          || _clipboardCreationTime < so.data.creationTime))
  {
      var bytes:ByteArray = so.data.contents as ByteArray;
      bytes.uncompress();
      _clipboard = bytes.readObject() as AppClipboard;
      _clipboardCreationTime = new Date().getTime();

      // Flush the object after reading it so it doesn't get cached!
      so.flush();
  }

Here you can see the version skew detection, as well as a guard against using a “stale” shared clipboard that is actually older than the local version. That can happen if the “write side” fails to save an LSO due to size limitations, in which case we have to degrade gracefully and use the local clipboard instead of the stale shared one. This means that a single instance will still behave consistently even if it fails to transfer data to other instances.

Surprise #2: note the call to so.flush() on the read side. I did not expect to have to flush() a SharedObject after reading it. However, before putting this flush in, I found that the player would cache the SharedObject and refuse to check to see if it had been updated when reloading it by calling SharedObject.getLocal(). This caused clipboard-style transfer of data between different application windows to fail sporadically. It took a long time to figure this one out.

No Comments

Minimal Apache/Rails configuration on small virtual server

As I’ve mentioned before I’m running a couple of WordPress blogs and a Rails application on Slicehost, using the smallest slice size of 256 MB RAM. Apart from Rails, this is a typical LAMP setup with Ubuntu Hardy Heron, Apache2-prefork, MySQL 5.x, PHP5. It took a little while for me to find a robust Apache and Rails configuration, though. The out of the box Apache2 config would periodically eat up all the swap space and die. So my hope here is to save others some newbie-sysadmin headaches.

Apache2. Apache2 prefork is configured to keep a certain number of processes running to respond to HTTP requests that come in. These processes are reused across a number of requests and thus tend to grow over time in their memory consumption to reach the largest amount of RAM demanded by any page served. When using mod_php, the RAM consumption is related to PHP application resources, in this case WordPress. I found that my Apache processes generally were in the 20m range for resident memory size (the RSS column in the ‘top’ display). With the out-of-box Apache2 configuration allowing as many as 15-20 processes, that meant that any large batch of concurrent requests (such as those originating from a search bot) would totally nail my machine. So the most important change I wound up making was to reduce the number of processes:

<IfModule mpm_prefork_module>
    StartServers          3
    MinSpareServers       3
    MaxSpareServers       3
    ServerLimit          10
    MaxClients           10
    MaxRequestsPerChild   1000
</IfModule>

This generally causes me to see around 5-6 processes running, with the possibility of as many as 10.

A number of folks have suggested I run nginx or lighttpd, which I’m sure would be a great idea, but I was strapped for time and in a mood to stick with things that have lots of online recipes and that I already somewhat understood.

Rails. Various approaches are available here as well. At first I was running the mod_passenger module, but I found this gave me a few 30m processes as opposed to the single 50m process I could get by running a single mongrel. So now I’m running a single mongrel and it’s working fine.

MySQL. Haven’t tuned this much yet, but I’m not seeing any problems.

WordPress. Caching is enabled on the advice of a number of people that this will decrease MySQL resource consumption.

No Comments

Flexcover 0.81 is out

A quick update: Alex Uhlmann and I have just released the latest Flexcover edition on our Google Code Project. This release supports the newly announced Flex 3.2 SDK, Flex Builder 3.0.2 and AIR 1.5.

No Comments

Vote for sampled audio to not crash IE7

Turns out there’s a nasty bug in sampled audio playback in the newly released Flash 10 player. Found by Tyler Wright of Legato Media, this bug takes down your entire IE7 browser if you open simultaneous windows with Flash content, one of which is playing back audio using the new SampleDataEvent API. This is a crippling problem for any ambitious Flash content on the web incorporating dynamic audio.

Fortunately, we now have the prospect of an American President who can speak in intelligible sentences… which means that you can turn all of your positive energy to getting Adobe to fix this bug. Please vote on it at:

https://bugs.adobe.com/jira/browse/FP-985

Future generations thank you! (Future generations of what, you may ask?)

1 Comment

Flexcover upgrade for Flex 3.2.0 SDK is looking good

Alex and I — lately, to be honest, it’s mostly Alex — have been working on the forthcoming upgrade release of Flexcover, the code coverage instrumentation tool for Flex/AS3. I’m feeling pleased because I finally merged up the modifications to the Flex SDK compiler to the 3.2.0 branch, in preparation for an expected release of a new Flex Builder in the MAX timeframe.

The merge went smoothly and the 3.2.0 compiler seems to work like a champ — well, at least, it does everything that the 3.0.x version did. What’s more, a sneak peek at Flex Builder 3.2 revealed that they’ve fixed an irksome bug in the Flex Compiler project options dialog, and it’s now possible to enable and disable coverage instrumentation using the compiler options field.

That’s just the part I did. Alex has been busy working on some very nice changes to the CoverageViewer, which should greatly reduce the performance problems people were seeing during data collection before. We’ve also made it much easier to merge reports from multiple test runs. A few more fixes are needed but things are looking up…

2 Comments

Host of Broken Dreams

This site has been up and down like a yo-yo since Friday. All is better, though now that I’ve moved this domain and the Noteflight blog to a new hosting company: Slicehost. Slicehost seems very cool — they offer what seems at first like a bare-bones service (VPS hosting starting with a minimal Ubuntu Linux distro, no one-click installs or control panels). But it’s not so bare-bones once you realize that Slicehost provides really, really clear how-to articles that make it very straightforward to set up all the same stuff that you’d normally get from a more standard hosting service. And when you’re done doing what these articles explain, you understand how it all works: a great benefit. And you’ve got total control over the system you build, with root access and all. No one can @#$% with it. For an example of being @#$%ed with, you may continue reading. [Continued...]

No Comments

David Coletta at Boston Flex Users Group 11/11

A quick squib here to note that my friend David Coletta will be talking about building unified codebases for AIR and Flex at the November Boston Flex User Group meeting.

Go see David.  Go see David.  Go see David.

Is there anything about “Go see David” that you don’t understand?  He’s a dynamic, intelligent and very, very informative speaker.  Hope to see you there!

No Comments

Strongly Typed Proxies in AS3

Something I’ve often wanted in AS3 is a strongly typed proxy: an object that implements a known interface with typed methods, but where every method results in the invocation of some method like invokeMethod(methodName:String, args:Array). AS3 does have a Proxy class, but unlike Java’s class of the same name, an AS3 Proxy doesn’t implement any particular interface. One can extend Proxy into a subclass that has an interface, but by that time the Proxy superclass isn’t delivering any real value. The AS3 Proxy has been designed for loose, dynamic typing — which after all is one of the strengths of the ActionScript/ECMAScript languages.

Why use a strongly typed proxy? A common use of such a proxy is to create a “smart wrapper” around some underlying object with a known interface that looks and acts (and code-hints) exactly like the underlying thingamabob, but does some other stuff around every method call or property access. You might want a smart wrapper that logs or records all the function calls (as well as executing them). You might want a wrapper that does a try/catch around every call. You might want a wrapper that serializes the call over a stream to some remote destination.

(If this sounds like what some call “aspect-oriented programming”, well, it is similar to one aspect of aspect oriented programming :-)

Anyway, we can’t use AS3’s Proxy to solve this problem, so what to do? We’ll start with a simple approach, but make it more general. If our interface looked like this:

public interface ISmashable
{
    function smash(force:Number):String;
}

and we wanted to create a smart wrapper that logged all function calls around any ISmashable implementation, we might create something like this:

public class SmashableLoggingProxy implements ISmashable
{
    private var _smashable:ISmashable;
    public function SmashableProxy(smashable:ISmashable)
    {
        _smashable = smashable;
    }

    public function smash(force:Number):String
    {
        trace("smash():", force);
        var result:String = _smashable.smash(force);
        trace("    result:",result);
        return result;
    }
}

Then, if we did something like var proxy:ISmashable = new SmashableProxy(realSmashable); we could treat the resulting proxy as an ISmashable everywhere in our application, Every time we called, say smash(5) method on proxy, not only would realSmashable.smash(5) be called but we’d see some nice trace output as well.
If you have the time to create a lot of wrapper methods like this (or write a fancy code generator to save the trouble), that’s fine.  Another way to go, however would be to create a “generic” smart wrapper base class: [Continued...]

9 Comments