As a user I want to write a simple shaders for graphics features that can do the following:

• Make a vertex color brighter/darker
• Displace vertex positions based on normals

Add test scripts to test the above cases

Windows and specifically when using mingw TLS is messy. We got some global destruction STL issues.

This fails consistently when running:
gdb --args cbl.exe .\src\tests\subchains.clj

See this image:

Location: https://github.com/fragcolor-xyz/chainblocks/blob/4ccf2302e53806ba17dbb8988d8be0dc64eed031/src/core/runtime.cpp#L2433
But this might also be an issue (also the 2 can be refactored): https://github.com/fragcolor-xyz/chainblocks/blob/4ccf2302e53806ba17dbb8988d8be0dc64eed031/src/core/runtime.cpp#L2358

I've tried refactoring a few times with no success with easy/trivial solutions. It's not easy to get rid of those collections given the way hashing works (stateless).

As a creator I want to utilize lighting from environment map images.

The only input required should be the environment map image and it should provide lighting to a all other drawn objects.

The following settings should also be exposed on this light:

• Light Multiplier

Currently GLTF.Load returns a single opaque object representing the entire model.
Individual materials and transform nodes should be accessible/modifiable by name.

As a creator I want to be able to play back the animations that move transform nodes embedded inside my glTF models.

See #218 for skinning support

Following this
https://github.com/KhronosGroup/glTF-Sample-Viewer/tree/master/source/Renderer/shaders
Implement PBR shaders, getting also vertex skinning ready to be used by Animation system.

Moreover we don't need a material system yet cos chains and blocks and scripts ARE material definitions if wanted.

Implement feature blocks.
Feature blocks are a grouping for render state and shader code.

Required components for this issue:

• Feature block
• Render state
• Shader entry points
• Shader parameters

Requires #168 for shader support
Check FIP-1 for specs

As a creator I want to utilize standard lighting types that affect all drawn objects

The following light types should be supported:

• Spot
• Directional
• Point

With their respective attributes:

• Location/Direction/Radii
• Color
• Intensity

Should be able to run on simulator on a MacOS CI machine

The app bundle
(does not include running tests)

Need some utilities to manipulate strings (skip, substr, remove, concat...).

Originally posted by @Kryptos-FR in #75 (comment)

Pitch
As a developer new to Shards, I want to access the Shards architecture guide so that I may be able to use Chainblocks better in my game development projects as well as contribute code and docs to this project.

Acceptance criteria
The architecture guide must

• Explain all the internal components of the Shards engine
• Explain the tradeoffs made in designing Shards as a fast, flexible and easy-to-use scripting language

Additional Context

• This architecture guide must be hosted on https://docs.fragcolor.xyz/architecture-guides/...
• WIP/reference doc at https://md.fragcolor.xyz/1R4aM7sKSq-LkVlPJ5FqGA?both

• Should contribute the GFX.Context _lightCount, which is used by shaders.
• Should contribute to GLTF _numLights as well
• Should take a Float3 as world position
• Should have classic game light parameters (Spot, Point, etc) and related parameters

Simple forward for now - figure out later future render complexities like implementing clustered rendering etc ( We need to discuss and spec before doing anything towards that direction )

Although widely supported, #pragma once is apparently non-standard and doesn't handle some edge cases.
Code to replace is mainly inside graphics code.

Catalina is now available, turn on testing on Mac as well!

https://github.com/sinkingsugar/chainblocks/blob/49395662b06f5303eb7cc6aba07c9d0b29739391/src/core/runtime.cpp#L2098-L2098

Currently sporadically fails (Linux) due to github actions VMs internals.
This might be a solution https://cirun.io

Android package
(does not include running tests)

No real Android/iOS workflow exists currently. iOS is built as part of CI but only the library & no testing.

Tasks:

• Setup android toolchain
• Setup android app target
• Setup iOS app target

When defining an external variable, the actual type must be set beforehand. There are scenarios where it is not practical and instead it should be possible to set it as Any and have the chain execution sets the final type once data gets written into it.

https://github.com/sinkingsugar/chainblocks/blob/c3aff2a0a5f65d48b5bb16ebea8f6933dd1eddca/src/core/runtime.cpp#L741-L741

Try and benchmark: remove this switch and state
Replace with preallocated exceptions like StopChainException
removing this should make every block activate faster, 1 check less.

Requires major refactor for return mechanics, which would benefit a lot too if the unwinding is cheaper.

Add technical document the workings of the shader generator code inside src/gfx/shader
The document should explain the general structure and flow of the shader generator.
Limit <1500 characters

As a user I want to be able to load and draw static glTF models directly in my scripts
Should be able to load from:

• A file
• Byte stream

Note: backend code exists only blocks are missing

As mentioned by @Kryptos-FR : #162 (comment)

https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html#cargotoml-vs-cargolock
The above describes how the cargo lock files should be used.

Since we mostly produce binaries (dll/staticlib/exe) we should version the rust lock files to make builds more reproducible

Validate structure of blocks used in shaders

Looks like (ParseInt) is limited to 32-bit integers while our int type is supposed to be 64-bit.

"4294967295" (ParseInt) (Log) ;; ok
"4294967296" (ParseInt) (Log) ;; Block activation error, failed block: ParseInt, error: stoul

I guess we need to use std::stoull instead of std::stoul.

Pitch
As a programmer new to Chainblocks I want the functions referenced in Snake tutorial to be documented so that I may be able to follow their use in the tutorial.

Acceptance criteria

• All Chainblocks functions referenced in this tutorial should be documented under functions
• This documentation should contain a description of what this function does, its input and its output
• This documentation should contain working example code to demonstrate the usage of this function

Additional Context
Functions to be documented:

• defchain - committed
• def - committed
• int2 - committed
• defn - committed
• defblocks - committed

This makes my dearest Valgrind cry in all languages. Fix it.

As a creator I want to enable the casting of shadows for all of the classic light types onto other draw objects

Light types required to cast shadows:

• Point
• Directional
• Spot

Add blocks for creating textures from images

Combine the following test executables/scripts:

• test_runtime
• test_extra
• test_gfx
• all scripts inside src/test that are currently run from CI

This way we can more easily test on platforms that have their own filesystems or where we're unable to run an executable multiple times with different arguments.

Eventually we can add a small remote protocol inside this application so that we can retrieve test results and logs from Web/Android/iOS/Console

Pitch
As a programmer new to Chainblocks I want the blocks referenced in Snake tutorial to be documented so that I may be able to follow their use in the tutorial.

Acceptance criteria

• All Chainblocks blocks referenced in this tutorial should be documented under blocks
• This documentation should contain a description of the block's definition, parameters, input, output, & any other details as applicable
• This documentation should contain working example code to illustrate the usage of the block

Additional context
Blocks to be documented (open PR #180):

• GUI.Window - commited
• GUI.Text - committed
• Math.Add - committed
• Math.Multiply - committed
• -> - deferred : alias for function defblocks hence to be documented under language-dsl aliases
• Msg - committed
• Log - committed
• Match - committed
• Sub - committed
• ForEach - committed
• Once - committed
• Take - committed
• RTake - committed
• Slice - committed

Description

Consider the following EDN input:

; input.edn
{:keyword "value1", "key string" "value2"}

Read this input in a simple chainblock program:

(defnode main)
(defchain keyword_test
  "keyword_test"
  (read-string (slurp "input.edn")))

(schedule main keyword_test)
(run main)

Expected

The result of (read-string) is a CBTable with two entries:

"keyword" -> "value1"
"key string" -> "value2"

Actual

The result of (read-string) is a CBTable with two entries, but the keyword is missing the last character:

"keywor" -> "value1"
"key string" -> "value2"

it's nearly there.
https://web.dev/gpu/

Pitch
As a developer new to Shards, I want to access the swappable graphics-rendering pipeline architecture guide for Shards so that I may be able to customize the graphics rendering pipeline of my game development projects.

Acceptance criteria
The architecture guide must

• Explain how the rendering pipeline customization feature works under the hood
• Explain the implementation-level details

Additional Context

• This architecture guide must be hosted on https://docs.fragcolor.xyz/architecture-guides/...
• WIP/reference doc at https://github.com/fragcolor-xyz/FIPs/blob/FIP-1/FIPs/FIP-1.md

Implement functionality as demonstrated by
https://github.com/fragcolor-xyz/FIPs/blob/FIP-1/FIPs/FIP-1.md#post-effects
and
https://github.com/fragcolor-xyz/FIPs/blob/FIP-1/FIPs/FIP-1.md#velocity-pass

TODO:

• Add EffectPass render step
• Add definition of frame buffers to render steps
• Add input/output color targets to effect passes
• Add output color targets to drawable passes

Sprite & Animation Systems

(Sprite.Sheet :Texture :Atlas)

{"name" {"width" 1024}}
(Sprite.Sheet :Image .image :Atlas {"name" {"width" 1024}})
(Sprite.Sheet :Image .image :Atlas (read-string (slurp "atlas.edn")))

(Sprite.Draw :Range :Repeat :Speed :PlayFromStart true)

Keeps a step of time

(defloop RenderSprite
  (Setup (Sprite.Sheet :Atlas) = .sheet)
  ; ... define state
  .state
  (Match ["idle" (Sprite.Draw :Sheet .sheet :Range (Int2 0 0))
          "walk" (Sprite.Draw :Sheet .sheet :Range (Int2 0 10))
          "run" (Sprite.Draw :Sheet .sheet :Range (Int2 20 25))]) = .texture)

Animation

Discord

Animation tracks should be generic, they should also be a fundamental component of our system. They should represent the concept of a "sequencer" (and in fact I'd take most of the inspirations from computer music sequencers).

This is extremely easy in chainblocks, because the runtime itself is a sequencer, this was a core design choice.

What needs to be done now is create an Animation Type (as CBType::Object or even dedicated CBType::Automation) that holds generic time based and optionally interpolated data.

Inspirations

https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#animations

Notice that GLTF supports only node and weight animations for now. Our system instead should support animating anything, this again is made possible because we can animate anything that is a CBVar and so any CBType.

The implementation itself can be inspired by (Time.Pop) essentially it will be something similar where according to time (and interpolation parameters) we will either wait or interpolate values and output the animation state
https://github.com/fragcolor-xyz/chainblocks/blob/c0d94649df7709acb8340a18eff642a55c754e0a/src/core/blocks/time.cpp#L74

Data

Example

; Skeleton
{"root" {"rotation" {"t" [0.0] "v" [(Float4 0 0 0 1)] "i" Interpolation.Linear}
         "position" {"t" [0.0] "v" [(Float3 10 10 0)] "i" Interpolation.Linear}}
 "pelvis" {"rotation" {"t" [0.0] "v" [(Float4 0 0 0 1)] "i" Interpolation.Linear}
           "position" {"t" [0.0] "v" [(Float3 10 10 0)] "i" Interpolation.Linear}}}
; Audio tracks
{"track1" {"volume" {"t" [0.0 3.0] "v" [99.0 80.0] "i" [Interpolation.Step Interpolation.Linear]}
           "pan" {"t" [0.0] "v" [0.0] "i" Interpolation.Linear}}
 "track2" {"volume" {"t" [0.0] "v" [100.0] "i" Interpolation.Linear}
           "pan" {"t" [0.0] "v" [-25.0] "i" Interpolation.Linear}}}

"t" - linear time in seconds

"v" - value in any kind of CBVar

"i" - interpolation

• is first read and the interpolation mode is set
• can also be a single value in this case applies to the whole track's frames
• works only for certain types of CBVar (numeric, etc)
• can also be omitted, in such case no interpolation will happen (STEP)

Blocks

Animation tracks related blocks

(Track.Play :Source <data table variable>)

Outputs the current interpolated frame.

(Track.Record :Destination <data table variable>)

Takes as input anything and records it automatically as a timeline key-frame

Blocks usage

(defloop animation-controller
         (Setup 
          = .model ; the input of this chain is the gltf model
          ; extracts a track from a model
          .model (GLTF.Track "walk") = .animation-table-data)
         (Track.Play :Source .animation-table-data) = .frame
         ; we could manipulate the animation output here!
         ; we could even pause! this chain runs on its own time schedule!
         ; consumers just read *current* output
         .frame)

(Setup "x.glb" (GLTF.Load) = .x)
(GLTF.Draw .x :Controller animation-controller)

(defloop animation-controller
         ; ...
         (Track.Play :Source .animation-table-data) = .frame
         ; we could manipulate the animation output here!
         ; we could even pause! this chain runs on its own time schedule!
         ; consumers just read *current* output
         .frame)

(Sprite.Draw .x :Controller animation-controller)

Todo

• Interpolation of CBVar types
  • with limits, some types will only support step which is no interpolation basically

  [name=Giovanni Natural place for this is C++ given the high performance and efficiency requirements]

Notes

• Root motion

Libraries that could help

• EDN-rs
• EDN-derive
• Spine-rust Made by Nicolas, only private repo on GitHub as of 2021-07-12

As a creator I want to control exposure and auto-exposure parameters for the camera, using parameters commonly used for these.

References:

• https://google.github.io/filament/Filament.html#imagingpipeline

Actually the issue might just be custom objects serialize are missing form json routines.

There are plugins to add syntax coloring. It just that the documentation of such plugins is very hard to understand: they expect you to know how mkdocs works internally and provide no easy step to integrate it.

Originally posted by @Kryptos-FR in #75 (comment)

We want to limit the allowed set of blocks in specific scenarios.
Example cases:

• Shader code
• Audio effect code

Add GFX.Window fullscreen support on desktop platforms

Most likely requires a physical device hooked up to a CI machine

https://github.com/sinkingsugar/chainblocks/blob/c3aff2a0a5f65d48b5bb16ebea8f6933dd1eddca/src/core/runtime.cpp#L741-L741

Often it is helpful to test both the DEBUG and RELEASE versions of cbl side-by-side.

As of now, they have the same name. We could add a "d" suffix for the DEBUG version.

Translate blocks required for #201 into shader nodes as currently specified in src/gfx/shader/blocks.hpp
Disregarding validity of used blocks.
Depends on #167

Blocks:

• Basic math (vectors/numbers) (mul/div/add/sub)
• Input Attribute
• Output Attribute

Implement builtin features for rendering based on the glTF reference.
glTF reference PBR shaders: https://github.com/KhronosGroup/glTF-Sample-Viewer/tree/master/source/Renderer/shaders

Tasks:

• #208
• #209
• #211
• #212
• #213
• #210