Bit shifting and masking, bitwise operators in AS 3. Getting Alpha channel out of the color.

Jan

2010

Bit shifting and masking, bitwise operators in AS 3. Getting Alpha channel out of the color.

In this article I'm going to show how to extract color channels out from the color value and how to create new colors using bitwise operators in Action Script. First there is some theory with examples and then practising with coding.

First some theory about the colors:

ARGB colors as 32 bit variables are specified by 4 groups of 8 bits each / or 2 hex each:

In binary:

AAAAAAAA RRRRRRRR GGGGGGGG BBBBBBBB

In hex:

AA RR GG BB

where each group defines intensity of each of the colors channels, A is alpha, R is red, G is green, B is blue.

FF is full intensity (255), 00 is no color in the channel (0).
White (full intensity on all channels) is in hex: 0xFFFFFFFF.
Black (no color in any of the Red, Green, and Blue channels) is in hex: 0xFF000000.
Full intensity on the alpha channel means no alpha (FF) and no intensity (00) means full alpha. So a transparent pixel color value is 0x00rrggbb.

Everybody use hex as that's the one, which is easiest to read (use 0x where declaring variables in hex, ie: var myColor:int = 0xAABBCCDD). Although binary representation is very long, we use it to understand how things work behind the scenes. In decimal the range for each channel is 0 - 255.

Some theory about the bit operators:

& - bitwise AND operator, used often for masking, works the way that 1 AND 1 gives 1, 0 AND 0 gives 0, 1 AND 0 or 0 AND 1 gives 0.

Below example:

	1010
AND	0111
gives	0010

Let's code it:

var a:int = 10;
trace(a.toString(2) + " (bin) | " + a.toString(10) + " (dec)");
 
var b:int = 7;
trace(b.toString(2) + " (bin) | " + b.toString(10) + " (dec)");
 
var c:int = a & b;
trace(c.toString(2) + " (bin) | " + c.toString(10) + " (dec)");

Traced results:

a: 1010 (bin) | 10 (dec)
b: 111 (bin) | 7 (dec)
c = a & b: 10 (bin) | 2 (dec)

| - bitwise OR operator, works as follows, 0 OR 0 gives 0, 0 OR 1 or 1 OR 0 gives 1, 1 OR 1 gives 1

Below example:

	1010
OR	0111
gives	1111

Let's code it:

var a:int = 10;
trace("a: " + a.toString(2) + " (bin) | " + a.toString(10) + " (dec)");
 
var b:int = 7;
trace("b: " + b.toString(2) + " (bin) | " + b.toString(10) + " (dec)");
 
var c:int = a | b;
trace("c = a | b: " + c.toString(2) + " (bin) | " + c.toString(10) + " (dec)");

Traced results:

a: 1010 (bin) | 10 (dec)
b: 111 (bin) | 7 (dec)
c = a | b: 1111 (bin) | 15 (dec)

>>n or <<n - right or left shift by n-bits

>>>n - unsigned right shifting by n-bits

Example:

1011 (11 in dec) >> 2 gives 0010 (2 in dec)

1011 (11 in dec) << 2 gives 101100 (44 in dec)

Right shifting:

var a:int = 11;
trace("a: " + a.toString(2) + " (bin) | " + a.toString(10) + " (dec)");
 
var b:int = a >> 2;
trace("b = a >> 2: " + b.toString(2) + " (bin) | " + b.toString(10) + " (dec)");

Traced results:

a: 1011 (bin) | 11 (dec)
b = a >> 2: 10 (bin) | 2 (dec)

Left shifting:

var a:int = 11;
trace("a: " + a.toString(2) + " (bin) | " + a.toString(10) + " (dec)");
 
var b:int = a << 2;
trace("b = a << 2: " + b.toString(2) + " (bin) | " + b.toString(10) + " (dec)");

Traced results:

a: 1011 (bin) | 11 (dec)
b = a << 2: 101100 (bin) | 44 (dec)

Now time for practicing - replacing alpha channel with AND and OR

So now let's try to use what we've just learned to replace the Alpha part of our color with Alpha from different color.

Let's say I have color named "my color", that I want to apply the alpha part from the "other color". I'm going to do that in three steps.
First I mask out the Alpha part from "my color". In the second step I mask out the color parts from the "other color" leaving only the alpha part. At the last step I will "merge" two variables creating new color.

"My color" is: 0xEEDDEEEE (hex), the "other color" is: 0xABCCDDEE (hex).
Let's change representaion to binary to see how the whole operation looks like.

First let's get rid of the alpha part from "my color". To mask it out, we use AND operator and 32 bit value that has the first part (first byte - 8 leading bits) only "0"s

our color: 11101110 11011101 11101110 11101110 (bin) 0xEEDDEEEE (hex)
mask: 00000000 11111111 11111111 11111111 (bin) 0x00FFFFFF (hex)
result of AND: 00000000 11011101 11101110 11101110 (bin) 0x00DDEEEE (hex)

Now similar thing with the "other color", we mask out everything except the Alpha part:

other color: 10101011 11001100 11011101 11101110 (bin) 0xABCCDDEE (hex)
mask: 11111111 00000000 00000000 00000000 (bin) 0xFF000000 (hex)
result of AND: 10101011 00000000 00000000 00000000 (bin) 0xAB000000 (hex)

Now let's combine both results using OR operator:

masked our color: 00000000 11011101 11101110 11101110 (bin) 0x00DDEEEE (hex)
masked other color: 10101011 00000000 00000000 00000000 (bin) 0xAB000000 (hex)
result of OR: 10101011 11011101 11101110 11101110 (bin) 0xABDDEEEE (hex)

OK, let's code it:

var myColor:Number = 0xEEDDEEEE;
var otherColor:Number = 0xABCCDDEE;
 
var resultColor:Number = ((myColor & 0x00FFFFFF) | (otherColor & 0xFF000000));
 
trace("myColor:     " + myColor.toString(2) + " (bin) | " + myColor.toString(16) + " (hex)");
trace("newAlphaColor:   " + otherColor.toString(2) + " (bin) | " + otherColor.toString(16) + " (hex)");
trace("resultColor:   " + resultColor.toString(2) + " (bin) | " + resultColor.toString(16) + " (hex)");

And here are traced result:

myColor: 11101110 11011101 11101110 11101110 (bin) | eeddeeee (hex)
newAlphaColor: 10101011 11001100 11011101 11101110 (bin) | abccddee (hex)
resultColor: -1010100 00100010 00010001 00010010 (bin) | -54221112 (hex)

Looking at the results, we notice that this is not what we actually expected... It looks like negative value. Why?
Let's amend last line in the code to trace results as unsigned int:

1	trace("resultColor: " + uint(resultColor).toString(2) + " (bin) \| " + uint(resultColor).toString(16) + " (hex)");

and now the result is correct:

resultColor: 10101011 11011101 11101110 11101110 (bin) | abddeeee (hex)

Action Script data types like int or Number are stored the way that the first bit (the leading bit) indicates whether value is positive or negative. To read more about negative int please click here. There are no negative values in the colors, so we actually should be using unsigned int (uint) rather than int (or Number - actually this data type should be rather used for values bigger than 32 bits due to the performance reasons).

Extracting individual channels with bit shifting

We can use bit shifting as a method of extracting individual color channels: first by moving necessary bits to the right and then masking out unnecessary values if there are any.
Let's code it:

var myColor:uint = 0xABCCDDEE;
 
var myAlpha:uint = myColor >> 24;
var myRed:uint = (myColor >> 16) & 0xFF;
var myGreen:uint = (myColor >> 8) & 0xFF;
var myBlue:uint = myColor & 0x000000FF;
 
trace("myColor:" + myColor.toString(2) + " (bin) | " + myColor.toString(16) + " (hex)");
trace("Alpha:   " + myAlpha.toString(2) + " (bin) | " + myAlpha.toString(16) + " (hex)");
trace("Red:   " + myRed.toString(2) + " (bin) | " + myRed.toString(16) + " (hex)");
trace("Green:   " + myGreen.toString(2) + " (bin) | " + myGreen.toString(16) + " (hex)");
trace("Blue:   " + myBlue.toString(2) + " (bin) | " + myBlue.toString(16) + " (hex)");

And the result is:

myColor: 10101011 11001100 11011101 11101110 (bin) | abccddee (hex)
Alpha:     11111111 11111111 11111111 10101011 (bin) | ffffffab (hex)
Red:                       11001100 (bin) | cc (hex)
Green:                                   11011101 (bin) | dd (hex)
Blue:                                                    11101110 (bin) | ee (hex)

Hmm, the red, green, blue are as expected but what has happened to our Alpha channel, we are expecting just AB and we got series of "1" in front of AB. This is again related to the way how the data types are stored in Action Script. This will happen every time when you will try to right-shift 32 bit values. It will be fine for 24 bit (RGB). First hex digit of our color is A which is 1010 in binary. The first bit is "1" what means this is a negative value. You may think, ok but we are using unsigned int... Well there is another way of right-shifting bits: >>>n shifts n bits to the right without caring about sign. It's called unsigned right shift.

So remember:
>> - pads with 1 or 0, if negative or positive respectively
>>> - pads with 0 no matter what

Let's see a difference on example below:

This was positive value, missing bits on the left were replaced with zeros.
01010101 00000000 00000000 00000000 >> 2 = 00010101

This was negative value, missing bits on the left were replaced with ones. This is again the way how Flash works with signed values. Read more here.
10101010 00000000 00000000 00000000 >> 2 = 11101010

01010101 00000000 00000000 00000000 >>> 2 = 00010101

No matter what is the leading bit, one or zero, unsigned right shifting replace missing bits with zeros.
10101010 00000000 00000000 00000000 >>> 2 = 00101010

So let's modify our code and see what we will get:

var myColor:uint = 0xABCCDDEE;
 
var myAlpha:uint = myColor >>> 24;
var myRed:uint = (myColor >> 16) & 0xFF;
var myGreen:uint = (myColor >> 8) & 0xFF;
var myBlue:uint = myColor & 0x000000FF;
 
trace("myColor:" + myColor.toString(2) + " (bin) | " + myColor.toString(16) + " (hex)");
trace("Alpha:   " + myAlpha.toString(2) + " (bin) | " + myAlpha.toString(16) + " (hex)");
trace("Red:   " + myRed.toString(2) + " (bin) | " + myRed.toString(16) + " (hex)");
trace("Green:   " + myGreen.toString(2) + " (bin) | " + myGreen.toString(16) + " (hex)");
trace("Blue:   " + myBlue.toString(2) + " (bin) | " + myBlue.toString(16) + " (hex)");

This time results are fine:

myColor: 10101011 11001100 11011101 11101110 (bin) | abccddee (hex)
Alpha:     10101011 (bin) | ab (hex)
Red:                      11001100 (bin) | cc (hex)
Green:                                  11011101 (bin) | dd (hex)
Blue:                                                   11101110 (bin) | ee (hex)

How is negative integer "stored" in memory?

You won't find much literature explaining how the data typs including negative integer are stored in memory. I had troubles understanding what's going on when shifting bits right in 32 bit variables. I got some help from Lordofduct on actionscript.org, who has shed some more light on that subject (link to that thread at the end of this article).

I'm going to quote him from the thread on the forum:

~ - NOT - this is not with only one value. A 1 is placed where a zero is, a 0 where a one is. This is called the 'binary inverse' of the value. The binary inverse of N bits in length of a value is the same number that needs to be subtracted from (2^N) - 1 to make the negative value of said value. That is why a negative int is just the NOT form of the positive value. You can try it, take a uint, NOT it, and then cast it as an int and you'll have that value negative - 1.

trace( uint( ~3 ) );//you get -4... -(3 + 1)

or as such:

trace( uint.MAX_VALUE - uint(~3) );//you get 3

I bring this up because this is how negative integers are stored. It's 1 followed by the 31 bits that represent the inverse of the absolute value. Anways so you can see the operand in action:

~1010 == 0101

The NOT of any positive value equals the -(value+1)

Example: What is NOT of 15?

0000 1111 (bin) == 15 (dec)
1111 0000 (bin) == -16 (dec)

var pos:int = 15;
var neg:int = ~pos;
trace("Positive value: " + pos + " is in binary: " + pos.toString(2));
trace("Negative value: " + neg + " is in binary: " + uint(neg).toString(2));

Gives as a result:

Positive value: 15 is in binary: 1111
Negative value: -16 is in binary: 11111111 11111111 11111111 11110000

So now you know how negative integers are stored in memory... but if you still wonder - why...well it makes sense? For further explanation visit a thread with discussion http://www.actionscript.org/forums/showthread.php3?t=190086 and read the explanation with examples from Lordofduct.

Add comment

JComments