Expected format for textures? about neural_renderer

[jcjohnson]

@jbohnslav Yes, after looking at the code again I figured it out.

For each face, the texture tensor contains a discretized set of samples (in barycentric coordinates) of linear combinations of the colors for each vertex; during rasterization these sampled colors are re-sampled to generate the actual colors of faces. The texture_size controls the number of color pre-samples per face.

More concretely, suppose texture_size = T. Then textures has shape (N, F, T, T, T, 3), with the following semantics (up to a possible permutation of vertices):

textures[n, f, T - 1, 0, 0] is the RGB color of vertex 0 for face f
textures[n, f, 0, T - 1, 0] is the RGB color of vertex 1 for face f
textures[n, f, 0, 0, T - 1] is the RGB color of vertex 2 for face f

Other elements of the textures matrix give linear combinations of the colors at vertices, so for example:

textures[n, f, 0, 0, 0] is 0
textures[n, f, T - 1, T - 1, T - 1] is the sum of the colors at the three vertices for f
textures[n, f, T / 3, T / 3, T / 3] is equal to color1 / 3 + color2 / 3 + color3 / 3 (with T % 3 == 0)
textures[n, f, T / 2, T / 2, 0] is equal to color1 / 2 + color2 / 2 (with T % 2 == 0)

[nkolot]

Hi Justin, my CUDA kernel is the same as the original CUDA code in the Chainer inplementation by the authors. I am not an expert in Computer Graphics and I don't to give you an incorrect answer, so I think it will be better to ask the original authors. Please let me know if you have any additional questions.

[jcjohnson]

Thanks! I'll ask over at the original repo.

[jbohnslav]

Did you ever figure this out? I looked on the issues page of the original implementation, and didn't find your question.

[jbohnslav]

Thanks so much!