jalberse / raytracinginoneweekendinrust

Ray Tracing In One Weekend, in Rust

Ray Tracing in One Weekend, In Rust

This repository is a CPU-bound ray tracing library based on Peter Shirley's books on the topic.

It adds some additional features such as parallel and tiled rendering. I also use it for some ray tracing experiments, such as implementing hash-based ray path prediction for skipping acceleration structure traversals.

It further makes changes from the books to achieve more idiomatic Rust code where possible.

Features

• Materials
  • Lambertians
  • Dialectrics
  • Metals
  • Extensible Material trait representation.
  • Isotropic
• Textures
  • Image textures
  • Procedural textures
  • Extensible Texture trait representation.
• Geometry
  • Constant density convex mediums
  • Implicit surfaces (e.g. spheres, rectangles)
  • Motion Blur
• Performance
  • BVH (Bounding Volume Hierarchy) implementation for fast ray collisions.
  • Multi-threaded, tiled rendering
  • Hash-based ray path prediction (experimental)
• Camera
  • Depth of Field
  • Shutter Speed

Usage

For more details on how to use this crate, run cargo doc --open in the cloned repository.

The binary provides a command line interface to rendering sample scenes. To install, while in the cloned repository, use cargo install --path .. Then use shimmer --help for more informtion. Or, skip installation and run cargo run -- --help.

Sample Renders

A variety of materials.

Constant Density Mediums

Motion blur

Acknowledgements

This repository is based on Peter Shirley's book Ray Tracing in One Weekend. I will readily recommend this book to anyone interested in computer graphics, along with his other books Ray Tracing: The Next Week and Ray Tracing: The Rest of Your Life.

A note on the "Shimmer" moniker and further development

Some names in this repository reference "shimmer". That was what I originally named this crate, but now this repository holds that moniker, and is where I've directed my interests in rendering.

raytracinginoneweekendinrust's Issues

Add alternative color management schemes

I believe we would have a ColorManagementScheme enum where each variation is a different scheme. We would make the current scheme into an sRGB variation. We could further have ACES, etc.

Take advantage of SIMD processing

Taking advantage of SIMD processing would be a great feature for the ray tracer. However, it's related to performance, and not really worth looking into until we're more feature complete (speed doesn't matter if you can't render interesting things anyways). We can focus on simpler speed-ups in #37 and #38, and focus on feature development, before we approach this.

See slides regarding SIMD processing:
https://cseweb.ucsd.edu/classes/sp17/cse168-a/CSE168_16_HighPerformance.pdf

Add OpenEXR output

We'd like to output exr files. The exr crate should be good for that.

Fix aspect ratio mismatch for demo

Camera aspect ratio and renderer aspect ratio are off (3:2 vs 16:9).

This is really just an issue for the demo render. I think it caused the image to render slightly elongated.

Let's also re-render the image with a matching aspect ratio and update it on the README (and my website).

Replace ImageColors and PixelCoordinates with Grid crate usage

These have been basically implemented like this just to have a flat matrix. They have nice names as structs, but I think it's probably better to just use the Grid crate to get the same functionality, and the variable names can carry the meaning of the data.

https://lib.rs/crates/grid

Add perlin textures

https://raytracing.github.io/books/RayTracingTheNextWeek.html#perlinnoise

Add Bounding Volume Hierarchies

We'll follow https://raytracing.github.io/books/RayTracingTheNextWeek.html#boundingvolumehierarchies

I'd like to get this done early, since it should speed up iteration during development.

Create Color newtype around DVec3

This should help readability and maintainability.

Add sample render with Marble with some reflectivity

Maybe we want to have a Metal material with the Marble texture

The lambertian marble doesn't look fantastic

And we'd add to README as well

Add scene files

We'll want something like this: https://www.cs.utexas.edu/users/fussell/courses/cs384g-fall2011/projects/raytracing/fileformat.html

This allows us to define a scene in a file, which we can then pass to the renderer.

We should replace the main.rs scenes by just providing sample scene files.

Improve error handling

Anywhere I use unwrap(), I should propagate out at appropriate. thiserr might be useful.

Generally, make the error handling in line with a proper library.

Add GUI

I'd like to add a basic GUI.

Use egui.

We can start with a simple UI to select rendering options, without showing the image being rendered, it can just continue to dump it to a file.

We would ensure we keep the command line options.

Add progress bar

With #44 , our hack of a progress bar no longer works (since our solution would not work in parallel with Rayon).

We should use indicatif or linya to add progress bars for our parallel processes.

QBVH

It is a common optimization to not use a binary tree for BVH, but instead to use a quad tree in order to take advantage of SIMD instructions, with the ray testing all 4 boxes at once. This is a win because it performs multiple collision checks in parallel and cuts the tree depth in half.

https://cseweb.ucsd.edu/classes/sp17/cse168-a/CSE168_16_HighPerformance.pdf is where I first encountered this idea.

For a more in depth look, maybe start here: https://www.uni-ulm.de/fileadmin/website_uni_ulm/iui.inst.100/institut/Papers/QBVH.pdf

This is a bit more of a plain english discussion: http://www.joshbarczak.com/blog/?p=787

I don't want to reinvent the wheel here, though. parry3d-f64 provides a Qbvh implementation that is optimized for SIMD! https://docs.rs/parry3d-f64/latest/parry3d_f64/partitioning/type.Qbvh.html

The parry ecosystem appears quite mature, and quite popular.

If using parry's Qbvh turns out well, I might be interested in using other aspects of their ecosystem as well, such as their ray casting. We would need to switch from our homebrew Aabb to their Aabb to use Qbvh, but that should be fine. But that's for later. Their Qbvh implementation seems relatively self contained, so we can just use that without buying in to switching everything.

Use enum_dispatch

The enum_dispatch crate will take our traits (Hittables, materials, textures, etc) and make them concrete enums and eliminate the dynamic dispatch calls (an equivalent is possible by just implementing the traits as an enum myself and matching on the variants for the trait method, but this is much less boilerplate).

The principle advantage here is speed from eliminating dynamic dispatch. The crate's benchmarks are quite impressive.

A disadvantage here would be that users of the library wouldn't be able to use the trait extensibly to create their own types. Or they could, but since the rest of the modules assume enum dispatch rather than boxed dyn traits, the user-created implementations wouldn't be compatible with the rest of the shimmer code base. There might be a way we could design it to get the best of both worlds, but I'm not sure.

I'd like to at least try the switch to enum_dispatch and measure the speed-up. We can determine if we like the trade-off once I've done that.

It kind of depends on what the goals of the project are. Is it a ray tracing library that other rust projects and plug into and extend, or is it a ray tracing engine that should be plug and play? I'm leaning towards the latter, because this is going to be open source anyways so if they want different materials etc they can fork it or submit a PR.

Follow "The Next Week"

https://github.com/RayTracing/raytracing.github.io/blob/master/books/RayTracingTheNextWeek.html

Let's do it!

Add a license

Let's get our ducks in a row

Follow "The Rest of Your Life"

https://github.com/RayTracing/raytracing.github.io/blob/master/books/RayTracingTheRestOfYourLife.html

Add instances

https://raytracing.github.io/books/RayTracingTheNextWeek.html#instances

Add multi threading

This is what Rust is good for! Let's make use of it. Fearless concurrency!

We would really want to parallelize at the main loop in the Renderer.

We'd want to look into how to determine the most efficient number of threads. Can that be adjusted based on the machine? Or should we simply accept a user input with some reasonable default? Experiments would be nice to find a reasonable default.

While we're at it, we might want to look into how other CPU bound ray tracing render engines deal with performance, or rather the problem of reducing iteration time of the artist. Progressive rendering, batching into grids, etc. I'm only vaguely aware of these techniques. We won't implement them in this ticket, obviously.

Ray batching/reordering for cache efficiency

See the final slides in https://cseweb.ucsd.edu/classes/sp17/cse168-a/CSE168_16_HighPerformance.pdf

If we really want to head down a path of improving performance, this might be a direction to go. It might displace the tiled approach that should be implemented before this in #38 .

Switch multi-threading implementation to use Rayon

This is only an idea, but it's low-priority since the current implementation works fine.

Our multi-threading works via a pretty standard implementation with passing messages, but it's a bit verbose. If we could take advantage of the ergonomics of Rayon, that would be great.

https://github.com/rayon-rs/rayon

Improve Renderer abstractions

I don't like Tile::tile() require passing an image width and height. It would make more sense to have some Image::tile() -> Vec function.

Maybe the ImageColor struct can become that Image class. The process of rendering then becomes properly modifying the Image struct's data to populate the colors. Or something like that, I don't love how that implies we'd make Image mut (though ImageColor already is, so, whatever).

Just some observations about how that could be improved.

Just think about the structure of this module and how we could make it more idiomatic. I just had these thoughts and wanted to write them down

Add solid textures

https://raytracing.github.io/books/RayTracingTheNextWeek.html#solidtextures

Add USD Support - Scene Descriptions

Add support for using (at least a subset of) the Universal Scene Description file format as the input for the ray tracer.

For main.rs, add command line parsing for scene selection, render options

Rather than modifying e.g. image resolution, aspect ratio, and which scene we're using by modifying code, we should use clap to specify all of that in the command line.

Largely, this will be helpful to reduce edits to main.rs in git history. We can instead just add everything we need like new scenes.

Add new test render with new features

https://raytracing.github.io/books/RayTracingTheNextWeek.html#ascenetestingallnewfeatures

Switch to workspace

I'd like to split the crate into workspaces. One contained crate would be just the library, the other would be what is now main.rs (and would eventually become a GUI application + CLI).

The reasoning being the library should really be able to stand alone as a crate.

Create unified surface material

I'd like to work towards a material that's in line with PxrSurface in Renderman, or a subset.

I'm wondering if we can avoid any need for a dyn material if we represent materials in a unified structure, and we simply construct the material with the various components (diffuse, subsurface, emissive, etc) as needed, letting a unified function handle light interaction considering all those components. An advantage of this approach is e.g. mixing diffuse and subsurface properties by mixing the gain on each, which would be impossible if we tried to represent those as simply different material types.

Add re-exports for modules, make module organization more rusty

A lot of our modules are quite nested in an unhelpful way, e.g. shimmer::bvh::Bvh. Should we just re-export Bvh to get shimmer::Bvh?

Generally look for places that might benefit and do them

Use Enums for static dispatch

The code has many smart pointers to enable different kinds of materials, textures, etc.

This comes with a runtime cost due to the level of indirection. We can instead declare each type that would implement the trait as a variant of an enum (eg a Material enum, a Texture enum, a Hittable enum, etc). We would pass around the enum and match on each variant, thereby performing static dispatching. Alternatively, we could use the enum_dispatch crate to avoid boilerplate code.

This was actually my original design, but I went with dyn Trait because I was unsure of the priorities of the project, and dynamic traits seemed for flexible for extensibility. But right now, I think performance is more important that flexibility for consumers of the library in almost every case, especially if we can create "unified" surface materials and so on. Someone could fork if they really wanted to.

So, maybe let's change traits over to enums.

I really want benchmarks of before and after on this change. I'm very curious what the change will be. However, I think we should ensure that's done on a more complicated scene than the very basic ones we're using right now. Maybe we want until we have obj loading or some more complex thing to test on.

If the cost of dynamic dispatch really is very low, then this change might not be worth it.

Consider switching from Enum representation of Materials to a Trait representation

Loosely, we would rename the Scatterable trait to Material, and get rid of the Material enum. Anything referencing the Material enum would now have a trait bound on Material instead.

The Material scatter() implementation just calls each variant's Scatter - we don't get benefits from having these as variants rather than just Boxing them (other than letting us have a Material vec on the stack without boxing anything, but I don't care about that).

This would allow users of the library to implement their own Materials, and would I think make implementing new materials have less boilerplate.

Try implementing it and see how it is.

Add progressive image rendering

This depends on #59

I'd like a window which shows the render as it progresses. Something like the ImageTool in Maya.

We currently completely render each tile before moving onto the next one. This is an issue for showing progress, since you generally want to see the image overall. So render tiles might only complete some part of the samples before we move on, and we come back to those tiles later. This might be slightly less efficient for cache hits but it's quite necessary for the user experience. So, we actually want two modes: one for interactive renders that spreads computation across tiles, and one that is the current approach which is more useful for offline renders.

Add motion blur

https://raytracing.github.io/books/RayTracingTheNextWeek.html#motionblur

Profile performance

It would be a good idea to generate a flamegraph for a few renders to get an idea of performance bottlenecks, to see what can be improved.

We can leave data here and resolve the issue once it's collected for future reference.

BVH - Flatten Structure

Right now, our BVH uses Rc to store pointers to child nodes.

I don't think this is very idiomatic rust. We could likely flatten the structure, somewhat like the bvh crate does: https://docs.rs/bvh/0.7.1/bvh/bvh/struct.BVH.html

Consider: https://rust-school.io/writing/overcoming-lifetimes/

We could also just begin making use of a BVH crate, which will likely be better optimized than rolling my own.

Use a linear Color type instead of DVec3 for color calculations

Color calculations for ray tracing must be done in linear color space, and so far I have just been using DVec3 for that. But, I think it would be nice to specify that the type is a color, so that we have more of the business logic encoded in the type system, avoiding future potential errors.

Using Palette::Rgb with T: f64 might be nice. It would play well with later conversion to Srgb for saving the image off. Or, maybe a simple NewType pattern would suffice.

Create README.md

With some nice renders!

Add volumes

https://raytracing.github.io/books/RayTracingTheNextWeek.html#volumes

Improve documentation

Go through and improve documentation.

Refactor modules to be more Rust-y

I think we want to refactor the modules to be more Rust-y. For example, I'm splitting each struct into its own file. I think that's not typical of most Rust crates that I've seen.

One thing I'd like to do is have an Implicit submodule for a Geometry module that includes e.g. spheres, cubes, rectangles, etc - items that we define implicitly rather than load in via models.

Possibly - Reduce Arc usage for materials

I don't love the need to pass around Arc's for materials and textures (which really stemmed from the fact that HitRecords need to get a copy of them from the hit objects). Possibly look into more idiomatic ways to work with this.

This is pretty low in my priorities though, since it works fine enough and I doubt it has a notable impact on performance.

Read Embree paper

https://cseweb.ucsd.edu/~ravir/274/15/papers/a143-wald.pdf

Embree is a great open-source CPU-bound ray tracer. Let's read its paper and learn from it, creating new tickets for improvements we can make.

Convert to Library crate

Right now, we have a binary crate. This would be more useful as a library crate, and I can have a separate repository that consumes it for testing renders, building scenes, etc via an executable.

It may be good to do this early in development, as it makes reasoning about the library's interfaces easier, since we will actually use those interfaces.

Use match for bounding box union()

This would be cleaner code than nested Ifs.

Improve HittableList representation

Right now, HittableList can only handle one type (following the pattern shown in 17-6): https://doc.rust-lang.org/book/ch17-02-trait-objects.html

We'd rather use a dyn T type for the Vec, similar to listing 17-4.

We'd rather use a dyn trait to allow for us to have a HittableList that includes Spheres, Volumes, Tris, etc.

Some functions will need to change (such as from_vec()) to avoid running into associated type bounds which is not stable in Rust.

Add image texture mapping

https://raytracing.github.io/books/RayTracingTheNextWeek.html#imagetexturemapping

Add rectangles and lights

https://raytracing.github.io/books/RayTracingTheNextWeek.html#rectanglesandlights

Read Coherent Ray Tracing paper, add issues

Read this paper: https://www.sci.utah.edu/~wald/Publications/2001/CRT/CRT.pdf

Examine where we can improve our code. This is an old paper but quite fundamental.

Add tiled rendering

Right now, the Renderer works via scanlines, working down the image row by row.

This is inefficient for the cache. A tile-based approach would be better, as within each tile we are more likely to hit the same objects compared to within each row. This improves our cache hit ratio.

So, instead of working via rows, work via tiles. Maybe each image is divided into 32 x 32 tiles.