FlexibleFactory

Some of our readers might have noticed a slightly downbeat tone to our recent posts, especially concerning everyone’s favourite web player! Here’s a more positive post to put the record straight.

Things have been a little quiet here at FlexibleFactory over the last year or so because the team have been busy with the launch of our new website over at TunaMedia Ltd.

Although we’ve kept the Flash content fairly discreet, it really deserves pride of place on the site. Without the Flash Player, this project would not have been possible. Even though we’ve been through a bumpy ride together, now that everything on the site is (quite literally) singing in harmony it gives me great pleasure to introduce you to radiotuna.com. Search thousands of free online radio stations by artist or genre… we’re pretty sure you’ll like it!

…and, despite the bumps and bruises, we still love you, Flash!

Over at tunamedia.co.uk, we’ve released a demo of Flash movie playing Shoutcast streams using the new Sound API features available in Flash Player 10. Take a look.

Last week, I updated and rebuilt the Flex SDK from Subversion, and was pleasantly surprised to find that new players had been included in the repository. It was followed by disappointment that my pet project, which leans heavily on the new SAMPLE_DATA events in the Sound API stopped working.

It turns out that in player 10,0,0,591, the SAMPLE_DATA event string, which up to this point has been a property of the flash.events.Event class has been moved to now exist upon the flash.events.SampleDataEvent class. I’d always wondered why the Flash Player team had chosen to create a SampleDataEvent class, but to put the string that defines the event on the Event class instead of the SampleDataEvent class. It appears that this inconsistency has now been cleared up.

It does, however, lead to a situation where a software product labelled as Release Candidate is still having significant changes made to it that will break software developed against it. It leaves me with the conundrum of having examples of the new Sound API extensions on our blog that will either work with the latest (but somewhat obscure) version of the player and leave most users with a broken experience, or to develop against the old API in the knowledge that the pieces will eventually break when newer versions of the player are officially rolled out.

I think it’s time to redefine my understanding of the term “Release Candidate”!

Over the weekend I was taking a look into whether Silverlight might be a better bet than the Flash player for delivering streaming audio (Shoutcast in particular) to end-users. I built an extremely simple application that points a MediaElement at a Shoutcast stream (using the same url tweaking trick oulined in this previous post) and plays it back. Once again, looking through the documentation, it becomes apparent that Microsoft are trying to steer developers towards the use of their own streaming server solution.

So bearing this in mind, is Silverlight a viable browser solution for delivering legacy streams that fall out of the bracket of their own “streaming solution”? It can play back Shoutcast streams across domains in just the same way that Flash can but suffers the same fundamental problem suffered by Flash. There is no means of informing the player that an mp3 stream delivered over http is not intended for download, so the player keeps the whole stream in memory with the expectation that the download will end at some point… with Shoutcast it never does and the player consumes memory (ultimately ALL of the memory, if left playing long enough). So, both Flash AND Silverlight end up in the same basket on this front, slowly consuming all system memory. I think that this is stupidly limiting, and an API extension to drop played data would be a pretty simple enhancement. At the cost of promoting and protecting their own server products, both Adobe and Microsoft are missing a very fat slice of consumer ears.

Here’s a verbatim copy of my reply to a question asked on the FlashBrighton mailing list about online radio stations and Flash:

Question:

Hey,

I’ve been asked to build a little widget that streams a selection of free-to-air online radio stations. I know that this is possible, but does it require a specific server setup or can I just point a netstream object at a URL?

TIA

Answer:

Currently there are 2 options (neither of them particularly satisfying):

Option 1:

Sound.play can be used on audio/mpeg content type Shoutcast/Icecast streams (i.e. standard mp3 Shoutcast). You can establish the stream URL by opening the published .pls (Winamp playlist file) in a text editor.

Shoutcast servers (DNAS) sniff the User-Agent header, and if you are going for the root of the streaming server (e.g. http://shoutcastServerLivesHere.net:8000/), then it will inspect the User-Agent. If it matches a browser type, then it will serve up a page of HTML rather than the stream itself. Unfortunately Flash in the browser does indeed send a “bad” user-agent, as this comes from the browser, not flash. You can work around this by adding a semicolon to the path (i.e. http://shoutcastServerLivesHere.net:8000/; ) which fools the DNAS server into handing you the stream. I’m not sure what the deal is with Icecast (this uses something called mountpoints, so I don’t believe that the root ever resolves to a stream).

When using Sound.play, you can (with limitations), use this across domains… however some of the API that allows inspection of the data is disabled, such as computeSpectrum.

Because the ICY protocol (used both by Shoutcast and Icecast) is infact a superset of HTTP, when you call mySound.play on a shoutcast stream, it is in all respects the same as a normal file download. However, in this case the download never ends. So, somewhat problematic is the fact that a playing Shout/Icecast stream will eventually consume all memory on the client (although I’ve heard that Flash conks at about the 1 hour mark, but I have not proved this). This means that periodically, you will have to kill the sound, drop all references to it and reload it. You can either do this while the current stream is playing (and risk maxing out the 2 connections per domain browser limit), or completely stop and restart the sound (with a short break inbetween). Whichever method you use, the 2 soundswill very rarely sync up perfectly and you’ll be left with a discontinuity in the audio that listeners will hear. Most online examples of this technique seem to do this at around the 10 minute mark (although I reckon you could go to about 1/2 hour). Using this method will never expose the optional metadata that is embedded in the stream because Flash cannot send the “Icy-MetaData” header to switch the server into sending these metadata packets. Even if it could, it would break the integrity of the MP3 stream meaning that the Sound API would not work.

Option 2:

Wowza Media Server offers a Shoutcast/Icecast to Netstream convertion. Pricey, but nice. You won’t suffer memory issues on the client using this, and it will relay the metadata embedded in the stream too. It can also repackage AAC Shoutcasts such that newer Flash players can play these streams too (no server recompression required).

FileReference.load is a nice new feature in Flash Player 10, but the only thing you get back from it is the FileReference.data property, which is a ByteArray. This is useless (well, not altogether, as we’ll see) in the case of MP3 audio files, because the built-in Sound API does not support loading from ByteArray.

When “making-some-noise” wasn’t quite so easy, I invested a bit of time messing around with MP3 data to see if my Flash sound hack for playing PCM audio data could be subverted to use MP3 data instead. While I got this partially working, I realised that it was a library without any application. What use was it? I tried to use it for shoutcast streams, but it is not possible to seamlessly stitch together arbitrarily small frames of MP3 data and fire them with a “silent sound”. Unfortunately the ramp-up time of the mp3 decoder means that the signal audibly drops out at the stitch points.

It turns out that my efforts were not in vain, because with the arrival of FileReference.load, I have been able to repurpose this library so that it is now possible to load a Sound object with the data provided from a FileReference instance.
(more…)

The new FileReference class in Flash Player 10 opens up a whole new side to web applications, where the barrier between the local file system and the network has (to some degree) been torn down.

Here’s an example of how to load MP3 files from the local file system. I’m sure you’ll appreciate why this is pretty cool…

(more…)

So the Flash Player team have finally relegated the SOUND_COMPLETE hack to the dustbin of failed tech… but replaced it with a far superior technology. You can see all the details over at Tinic Uro’s blog (here, here and here).

Naturally quite excited by this news, I have been doing some development work. Here’s what I needed to get started:

Flash Player 10
Handy doc explaining where to get Flex SDK nightly build and how to use it to target FP10
Standalone Flash Player 10 (Debug Version)

I’ve been refactoring a class library that I have been writing over the last year and a half or so, to see how well this new stuff works. The main obstacle in the refactoring process has been the transition from using standard Arrays to ship the audio data through the application to the use of ByteArrays instead. This seems to offer a significant improvement in performance (although in the debug player it runs like a dog… about 4x slower).

Ultimately the aim is to publish this as a library for all to use, but while I am investigating licensing options and a new site for the library, I will be posting demos here.

Here is a demo application (requires Flash Player 10, it won’t try to auto-update if you don’t have FP10) that demonstrates the following features:

Extracting PCM audio data from MP3 sounds
A feedback delay module
A gain module
A mixing module
A resampler (changes the playback rate)

Below is a schematic of how these modules are chained together.

After writing an article for FlashBrighton about audio and Flash, I spent a while messing around with audio synthesis in Flash. The project has become a modular, xml powered synth engine, but has lost quite a lot of momentum recently. So instead of a grand launch, I’m going to release in bits and bobs, going back over early wins on the project. Here’s a Flex prototype that I wrote not long after releasing the initial article over a year ago.

(more…)