Java OpenGL apply fragment shader to partially transparent texture mapped quad

Question

I'm drawing a texture mapped quad with a texture that has some transparent pixels on it. So, I load the texture and then draw it

    gl.glEnable(GL.GL_ALPHA_TEST);
    gl.glAlphaFunc(GL.GL_GREATER,0);
    gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
    gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL.GL_RGBA, width, height, 0, GL.GL_RGBA, GL.GL_UNSIGNED_BYTE, buff);
    gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR);
    gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
    gl.glTexEnvi(GL.GL_TEXTURE_ENV, GL.GL_TEXTURE_ENV_MODE, GL.GL_REPLACE);
    gl.glColor4i(1,1,1,1);
    gl.glBegin(GL.GL_QUADS);
    // draw my quad here
    gl.glEnd():

When I draw this, the transparent pixels do not show as expected. However, I want to apply a fragment shader. In this case I'll test something simple:

void main() {
  gl_FragColor = vec4(0.0);
  if ( gl_Color.a > 0.0 ) { 
    gl_FragColor = vec4(1.0,0,0,1.0); 
  }
}

In this case, ALL of the pixels should up as red, instead of just the non-transparent ones. Can someone explain how to just color the non-transparent pixels using a shader?

thanks, Jeff

Answer 1

gl_Color is the color output by the vertex shader. Unless your vertex shader did your texture fetching for you, then your vertex shader probably passes the color attribute directly to your fragment shader via gl_FrontColor.

If you're not using a vertex shader, and just using fixed-function vertex processing, then it's a certainty that the fragment shader was given only the color. Remember that fragment shaders override all per-fragment glTexEnv operations, including the fetching of the texture.

If you want to test the texture's opacity, then you need to fetch from the texture yourself. That requires using a sampler2D object and the texture2D function (assuming you're using GLSL version 1.20. If you're using later versions, you want the texture function). It's been a while since I did anything with OpenGL's built-in inputs and outputs, but the shader would look something like this:

#version 120
uniform sampler2D myTexture;

void main()
{
    gl_FragColor = vec4(0.0);
    vec4 texColor = texture2D(myTexture, gl_TexCoord[0]); //Assuming you're using texture coordinate 0.
    if(texColor.a < 1.0)
        gl_FragColor = vec4(1.0,0,0,1.0);
}

You will also have to set the program uniform's value to the texture unit you're using. You'll have to look that one up though.