Well you have done a small mistake in the macro you have to put it in the bracket i.e
#define max(a,b) (a>=b?a:b)
Since what macro does it it just replicates you code as soon as it gets max(x,y) by a>=b?a:b now you would expect it to solve your max(R,B) first evaluate it and then compare with G but this is not happening in fact what you compiler evaluates that expression is a>=b?a:b>g?a>=b?a:b:g and then give you a answer. I hope I made it clear.
Your Correct Code-DZvm0S - Online C++ Compiler & Debugging Tool - Ideone.com
Oh! Thank you !