slothfulxtx / diff-gaussian-rasterization Goto Github PK

Hello, for alpha mask isn't this the equation

$$ \text{CumulativeOpacity} = \sum_{k=1}^{N} \alpha_i \cdot T_i $$

However, the code is different https://github.com/slothfulxtx/diff-gaussian-rasterization/blob/main/cuda_rasterizer/forward.cu#L372
in that only account for $T$ why not $\alpha_i$? Maybe I am confused what is alpha mask? (1 indicates opaque and 0 indicates transparent?)

Hello. I am confused about the meaning of variable radii from this code:

from diff_gauss import GaussianRasterizationSettings, GaussianRasterizer

rendered_image, rendered_depth, rendered_alpha, radii = rasterizer(
    means3D = means3D,
    means2D = means2D,
    shs = shs,
    colors_precomp = colors_precomp,
    opacities = opacity,
    scales = scales,
    rotations = rotations,
    cov3D_precomp = cov3D_precomp
)

Could you please explain me what is the meaning of radii for each Gaussian and what its value represent in context of one Gaussian? I see that output of this variable is:

tensor([15,  7, 15,  ...,  4,  5,  3], device='cuda:0', dtype=torch.int32)

So for each Gaussian in the scene, there will be one radii value. What does this value represent?

P.S. It seems that radii is plural of radius, which I didn't know :D.

But still I am confused about what value represent? Radius of Gaussian in which unit and in which space (2D/3D)? What is this value if Gaussian is non-isotropic? Maybe none of this is true, so I am confused about this variable meaning.

Thanks for your great job!
I want to know why added the accum_rea = last_alpha + (1.f - last_alpha) * accum_rea; dL_dalpha += (1 - accum_rea) * dL_dpixel_alpha; in renderCUDA backward.cu You know that in the gaussian-splatting's code ,this isn't be added. and how can I get the formula about this part.

Hello, thanks for the wonderful work!

I have a question regarding the range of the parameter colors_precomp. I have a color_raw which ranges from 0 to 300. When I clamp it using:

color = torch.clamp(color_raw, 0.0, 256.)
out = rasterizer(
          means3D = means3D,
          means2D = means2D,
          shs = None,
          colors_precomp = color,
          opacities = ones,
          scales = scales,
          rotations = rotations,
          cov3D_precomp = None)[0]

I get a more white image, compared with:

color = torch.clamp(color_raw, 0.0, 1.)

I'm wondering would this function clamp the colors_precomp between 0-1? If yes, why does it get a more white image when clamping this term using a wider range?

I hope you could give me a hint about it. Thank you very much!

Hello,
Is it possible to render just single pixel which returns the color of each single ray from different colmap camera positions using depth/diff-gaussian-rasterization?

I am interested in getting final color at particular location for different view directions, by providing: 3DGS pointcloud, colmap camera poses, "location of target pixel for tracking in first fully rasterized image using first colmap camera pose from colmap pose sequence" or "3D location in scene (how to find it?)".
Can you please guide me to achieve this?

1. does this make trained scene representation/ rendering better than original 3DGS rasterizer?
2. is there a way to use this to make loss term used in 3DGS more efficient and better?

hi, thanks for your work, Did your transform the depth map to point3ds for visualization? I test the depth map on bicycle (mip360-datasets), the depths seem are not consistent on multi-view and in some views the bicyle seems distorted.
Looking forward to your reply, thanks! @slothfulxtx

Hello!

Thank you for allowing the amazing implementation to public!

I was studying over your implementations and had some questions on how to compute loss gradient w.r.t 3D means due to gradients of depth from rendering procedure, which is implemented in preprocessCUDA() of backward... The code snippet is

glm::vec3 dL_dmean2;
float mul3 = view[2] * m.x + view[6] * m.y + view[10] * m.z + view[14];
dL_dmean2.x = (view[2] - view[3] * mul3) * dL_ddepth[idx];
dL_dmean2.y = (view[6] - view[7] * mul3) * dL_ddepth[idx];
dL_dmean2.z = (view[10] - view[11] * mul3) * dL_ddepth[idx];

1. Can you please explain how this part is derivated?

2. Why use world2view matrix (view), not projective transform matrix (proj) as was used to calculate loss gradient w.r.t. 3D means due to gradients of 2D means? It was a bit confusing for me because the points go through projective transformation to get rasterized, meaning that the backward process to depth info should inevitably encounter projective transformation??

Again, thank you so much for your implementation!