Incorrect behavior when sampler uniforms are distributed among multiple programInfo.setUniforms calls about twgl.js

[greggman]

Yes, setting all the uniforms is probably best, it's only shared uniforms you don't need to set every draw call (though you probably have to set them every frame). For example if you had a shader with these matrix uniforms

 uniform mat4 world;
 uniform mat4 view;
 uniform mat4 perspective;

It's likely that view and perspective are the same for all objects that use the same shader so you could theoretically just set the view and perspective uniforms once. world on the other hand is most likely unique for each object.

Me though, except for short demos that draw lots of stuff I'd probably just set them all for each object and only later, if things were too slow, check if setting uniforms is the bottleneck and optimize that if it is.

[greggman]

I think I need more info. When you say sampler do you mean like WebGLSampler or just the fact that you have a uniform sampler in your shader?

Here's a test I could not repo

https://jsfiddle.net/greggman/4d4ts3k1/

Maybe you can adjust it to repo?

The code for setting uniforms is pretty straight forward. It only sets the things you tell it to set although it does bind textures but only for the uniforms you pass in.

[pineapplemachine]

I fiddled with the fiddle but didn't get anywhere.

Here's a simplified repro case from the code I'm working on: https://www.dropbox.com/s/g2xmi7wck1gam37/samplerrepro.zip

In shader.js line 21 you'll find that no matter which sampler is being sampled from, data from u_mapSampler is shown. (At one point it was working correctly for one of those initially set samplers and not the other? But as of the state of the code in the zip, both are overridden.)

You can see where the uniforms are initially set in repro.js on line 80.

In tilemap.js lines 96 & 97 you can uncomment the two commented-out lines and the samplers will behave as expected.

[greggman]

So here's the issue and at the moment I can't think of a simple work around. Maybe something will occur to me.

The issue is for samplers there's 2 pieces of state, one is the uniform, the other is the texture on the texture unit. twgl.setUniform sets the uniform AND the texture unit but that texture unit is reused by other code before you render. Without looking at your code imagine we did this

 set up glyphTexture (calls gl.bindTexture)
 call setUniforms

At this point the uniform is set to use texture unit 0 AND texture unit 0 has glyphTexture bound do it. So next we do this

 set up paletteTexture (calls gl.bindTexture)

At this point gl.bindTexture has been called. It's using whatever is the activeTexture unit which is still unit 0. So paletteTexture unit 0 is now bound to paletteTexture

In your code you setup paletteTexture and glyphTexture at startup but they're loaded from images. When those images finish loading they'll use whatever the current active texture unit is to upload the texture. If that happens to be used by some other uniform and you don't call setUniforms nothing will rebind the correct texture.

I'm not sure there's an easy "generic" (as in something I can do to twgl) that would fix that issue. Twgl calls both gl.bindTexture and gl.uniform for any particular uniform but it can't stop you from calling other things that change the texture bindings.

One hack might be for TWGL to always use the highest numbered texture unit and then either restore the previous active unit (which would be slow) or just leave it. In other words all the texture manipulation functions would call gl.activeTexture(highestNumberedTextureUnit) before doing anything. Very few apps use all texture units so that would mean no conflicts. If you bound some textures to units 0, 1, 2 etc they'd stay bound (or at least twgl wouldn't be doing anything there). I'm not yet sure this is something twgl should do though or if the user should do it. Or maybe it could be an option. I guess I need to think about it. The problem becomes strange things might happen. Example

 gl.activeTexture(desiredUnit);
 tex = twgl.createTexture(gl, {src: [255, 0, 0, 255]});  // make red texture and bind to desiredUnit

If twgl overrode the active texture unit automatically that code would fail.

I could let you pass in a texture unit maybe?

 tex = twgl.createTexture(gl, {
     src: someUrl,
     textureUnit: desiredUnit, 
 });

That way you could choose a unit and if you don't choose one it would use the current one

I could also make an optional default as in

 // tell TWGL to do all texture manipulation on unit 12 (if not called it uses the current unit)
 twgl.setDefaults({useTextureUnit: 12});

Thoughts?

note: as a test I changed your shader to this

    void main(){
        vec4 c1 = vec4(texture2D(u_mapSampler, v_mapPosition/8.0).xyz, 1.0);
        vec4 c2 = vec4(texture2D(u_paletteSampler, v_mapPosition/8.0).xyz, 1.0);

        gl_FragColor = vec4(texture2D(u_glyphSampler, v_mapPosition/8.0).xyz, 1.0) + (c1 + c2) * 0.000000001;
    }

just to force that all samplers exist (they aren't optimized out). In this case it worked (by luck). Because u_glyphSampler is set first and then u_paletteSampler is set second then when the textures actually finish loading u_paletteSampler unit is used to upload the texture (the last activeUnit) therefore u_glyphSamplers unit's binding is untouched.

This isn't really twgl issue, the same issue would exist in normal WebGL. the only question is would adding those options help, make this worse, or what.

[pineapplemachine]

I think having an option to set the texture unit would probably work well. To make sure I'm understanding - would that mean that specifying a different index for each texture using an option like that would be one way to fix the issue?

The default texture unit seems like a reasonable addition, too.

Thank you for investigating that!

[greggman]

No, you wouldn't specify a different texture unit for each texture. Texture units are global, a normal webgl app (game) has thousands of textures but the texture units are global so they have to be set before each draw call.

Actually given that, thinking more about it, I'm not really sure it makes sense to add a twgl feature to work around this. It's kind of just the way GL works. It's similar to the way attributes work when not using vertex array objects, you just have to set them period. The same is true of texture units.

My point being that any workaround is really only for the exception. The case where you're using like a single shader and one set of textures. In the normal case where you're drawing lots of different things you'd end up having to bind the textures all the time.

So that means maybe it doesn't make sense to try to do something for the special case when it's just that, a special case.

I'm not sure I'm making any sense or what level your GL experience is. My point is the normal case is this

 // to render
 for each object
      gl.useProgram(whateverShaderThisObjectNeeds)
      for each attribute of this object
         gl.bindBuffer
         gl.vertexAttribPointer
      for each texture this object uses
         gl.activeTexture
         gl.bindTexture
      for each uniform on this object
         gl.uniform
      gl.draw

That's the normal case.

You can get rid of the attribute loop with vertex array objects (see OES_vertex_array). In WebGL2 you can partially get rid of the uniform loop using uniform block objects except of course if uniforms are changing you still need to update them (objects move, cameras move, etc...)

But, you never get rid of the texture loop unless you're making the simplest of WebGL apps (one that draws one or two things and doesn't use different textures between the calls). And, arguably that simple of an example doesn't need any workarounds.

Because your repo only has a single shader and 3 textures you were assuming you could do some of that per object setup at init time instead of render time and if you do everything right you can, but it's not the normal case so does it really need a twgl level workaround?

For example if you did want to work around it yourself put a call to this.gl.activeTexture(this.gl.TEXTURE7); at the end of scene.start and scene.gl.activeTexture(scene.gl.TEXTURE7); the end of scene.render and your repo code would work.

But, assuming you're going to start drawing more stuff later (sprites etc,) you'll need to start setting the textures before each draw call. Using twgl, it's twgl.setUniforms that binds the textures so you'd have to call it anyway.

[pineapplemachine]

Ah, I think I see.

I'm still learning OpenGL so please forgive me my less intelligent moments

So the best thing to do is... probably just what I was doing anyway? Setting all the uniforms before each draw call?