FlexibleFactory

I’m really excited by the possibility of using PixelBender filters for audio processing, but at the moment, it seems to be a neglected area of development for the FP10 team, despite the odd mention here and there of its use in operations other than image processing.

I’ve posted several questions to people who might be able to answer, yet no further information has been forthcoming. I’m publishing a digest of these questions below in the hope that somebody might be able to shed a little light on how it is all expected to work…

Issue #1

There has been mention of using the ShaderJob API for non-image related computation. It’s a subject of interest to me as it could be used for processing audio data. However, it seems to be impossible to create PixelBender filters that allow the processing of one dimensional data. I posted a couple of questions on the Adobe forums about this, but have yet to receive a satisfactory answer. Here’s one of the questions that summarises the problem:

The Flash docs say the following:

To process a ByteArray containing a linear array of data (as opposed to image data) set the corresponding ShaderInput instance’s height to 1 and width to the number of 32-bit floating-point values in the ByteArray. In that case, the input in the shader must be defined with the image1 data type.

What must the output be defined as? Float, right? This would mean a simple doNothing filter could be defined as in the attached code below.

<languageVersion: 1.0;>
kernel doNothing
<   namespace : "flexiblefactory.co.uk";
    vendor : "flexiblefactory.co.uk";
    version : 0.1;
    description : "do nothing filter"; >
{
    input image1 src;
    output float dst;
    void evaluatePixel()
    {
        dst = sampleNearest(src, outCoord());
    }
}

But PixelBender gives the following error:

ERROR: (line xx): ‘dst’ : cannot have 1 or 2 channel outputs

So, either my assumptions are incorrect, or PixelBender does not support this kind of operation.

Can an Adobe head tell me what’s going wrong here? Am I barking up the wrong tree or are we waiting for a new version of PixelBender that supports 1dimensional data?

Issue #2

The default endian-ness for all audio big endian, but the default endian for the Shader API when using it to operate on ByteArray instances is little endian. The only way to convert the endian of a ByteArray full of floating point numbers is by iterating through each value, and copying to another ByteArray with differring endian. When moving high volumes of data, this conversion limits the usefulness of a Shader to only more complex calculations.

For instance: a simple Shader designed for setting the gain of an audio stream becomes pointless if you first have to iterate the contents of the ByteArray in Actionscript to change the endian, and subsequently convert the output back again for playback. This would negate any performance gains made by using the Shader.

So, all this talk of using the Shader for audio operations comes with a major drawback, right?