Different calculation results in...

Hi,

I have a PixelBender Kernel where I calculate the temperature of the color to generate a blue/green-screen effect. I noticed now that I get different values in flash and in the toolkit.

I also noticed that the order of multiply and divide does matter in flash kernel but not in pixelBender toolkit. Now I use higher values for the temperature but still have a very different range in flash compared toolkit. To green-screen I have in toolkit hte values h1=760 to h2=1210 wich look absolut clean and in flash I have to use the values h1=592 and h2=631 wich gives me more diry results (can't get a clean seperation of green and yellow).

Here are the important code lines of my pixel bender. In the toolkit I get more accurate results and have to use different ranges (h1 = from temperature , h2 = to temperature).

?parameter float h1
%26lt;minValue: 00.0;
maxValue: 1600.0;
defaultValue: 760.5;%26gt;;

?parameter float h2
%26lt;minValue: 00.0;
maxValue: 1600.0;
defaultValue: 1210.5;%26gt;;

...

r = curColor.r; g = curColor.g; b = curColor.b;

maxColor = max(max(r,g),b);
delta = maxColor - minColor;

// calculate green only

if (maxColor == g){

?// old routine giving very raw results in flash
//h = 2.0 + (b-r)/delta;
?//h *= 60.0;//h

?// bigger number to be more acurate
h = 1200.0 + (b-r)*600.0/delta;

dst.rgb = center.rgb;

dst.a = 1.0;

if (?((h%26gt;= h1) %26amp;%26amp; (h %26lt;=h2)) )

dst.rgba = float4(0);

}
}

Different calculation results in...

Is there any chance that delta is either 0.0 or very small? Division by zero is undefined, and often leads to different results between the toolkit running on GPU, the toolkit running on CPU and flash. You might need to check for abs( delta ) %26lt; epsilon.

If this doesn't help could ypu post your complete Pixel Bender kernel and I'll take a look at it.

Different calculation results in...

I tried

if (maxColor == g %26amp;%26amp; abs(delta)%26gt;0.01 ){

..

but no change in the result. Otherwise h would be 0 or undefined and the pixel should look like any other.

My Pixel Bender Kernel Filter attached ...

I think there are several potential problems with this kernel, but I think some of them are occuring because of the way you've simplified it. In the version you attached to your last message, h and s are not necessarily initialized - since they are then used in the comparison at the end we have no way of predicting how that comparison is going to work out. This is exactly the kind of thing that Flash and the toolkit will handle differently.

Also, you corrected one place where you were comparing delta to zero, however there's a comparison of maxColor to zero and of maxColor to g. Any equality / inequality comparison between floating point numbers is suspect, you cannot rely on it working at all and you canot rely on it being the same between Flash and the toolkit. Instead of this:

?if (maxColor == g %26amp;%26amp; abs(delta)%26gt;0.01 ){

you should be writing this:

?if (abs(maxColor - g ) %26lt; 0.001 %26amp;%26amp; abs(delta)%26gt;0.01 ){

Ok, my calculation looks good now by referening to curColor.g (and r and b) instead of refering to its single float copys.

But I wonder what is going wrong here. Is there a difference between floatprecision of a float3 element and a float variable? And it surprisingly works better on GPU than software (emulated?) in flash. Smells like a bug.