ambientrun / ambient Goto Github PK

The camera for the runtime is created on the client; this means that there is no way to modify the camera from the server. In the future, this should be controlled by #6, but in the meantime we should move the camera entity to the server and have a clientside system that activates it for you.

Right now, we call them "script modules", but that feels like it's underselling them. Random ideas:

• Just "modules"
• "Logic modules"
• "WASM modules"

The runtime should have an operating mode where it will run the client and server as normal, and exit with a 0 or error status code depending on one of these:

1. Error: The client fails to connect to the server
2. Error: A timeout after initial startup expires (e.g. 5 seconds after the client expires)
3. Success: The guest code calls a runtime function to notify it that it is done. Upon doing so, the runtime will take a screenshot of the current state. If the screenshot does not match a stored screenshot, it fails. Otherwise, it succeeds.

The examples would have something like

runtime::ci_test(|success, fail| async {
  sleep(1).await;
  if entity::exists(id) { 
    success();
  } else {
    fail("The entity did not exist.".to_string());
  }
});

at the end of their execution. This is not automatic as the guests should be able to specify their own success conditions (e.g. "was this entity created?") and to lock their state for the screenshot (e.g. for the asset loading example, it should lock its cube in place for the screenshot)

In order to publish to crates.io we need to get rid of the git dependencies:

• Profiling https://github.com/AmbientOrg/Ambient/blob/main/Cargo.toml#L19
• cap-std etc. https://github.com/AmbientOrg/Ambient/blob/main/Cargo.toml#L139
• erased-serde https://github.com/AmbientOrg/Ambient/blob/main/crates/ecs/Cargo.toml#L14
• russimp https://github.com/AmbientOrg/Ambient/blob/main/crates/model_import/Cargo.toml#L43
• wit-bindgen-host-wasmtime-rust https://github.com/AmbientOrg/Ambient/blob/main/crates/scripting/Cargo.toml#L44
• physx-sys https://github.com/AmbientOrg/Ambient/blob/main/libs/physxx/Cargo.toml#L10

https://github.com/gfx-rs/wgpu/releases/tag/v0.15.0

Parameters are currently supplied by the ECS to the script, which means scripts can't know what parameters they'll have, and you need to manually populate the parameters in the ECS to match a script.

Ideally, there would be a macro - like our components macro- that will generate a build artifact that the Dims host can pick up on, and populate the ECS from. Just need to make sure it doesn't damage the build time too much.

As reported by @SK83RJOSH: the host code has the ability to create spheres, but this is not exposed to the API due to the use of a non-primitive component. To resolve this, we should consider splitting that into multiple components, as is the case with character_controller_[height|radius].

Rust compilation is currently handled by a system in elements_scripting. It should be moved to the asset build system so that it can benefit from the asset pipeline and so that it becomes one option for scripting, instead of the only option.

There are a few potential pain points:

• The platform currently depends on hot-reloading. We have agreed to break this for the moment.
• There are auto-generated files for the parameters and for the components. The former is not relevant to the runtime, so it can be removed for now (or #3 can be done). The latter may depend on #13. For the purposes of this issue, hacking in the components is sufficient (including forcing the user to include their own components.rs).

At present, the runtime does not build on stable Rust. This is due to the use of four features:

#![feature(map_many_mut)]
#![feature(cursor_remaining)]
#![feature(coerce_unsized)]
#![feature(unsize)]

The first two features are primarily used for convenience and can be replaced with slightly-less-ergonomic equivalents. The latter two features are used to enable easy construction of Cb callback wrappers - it's likely that the downstream code will need to be changed to not rely on this, or another way is found to make Cb construction simpler.

All of the WASI functionality is currently exposed to scripts, including FS access. We should lock this down and figure out what they actually need.

The access will be dependent on what they do - an asset loader is likely to require more system access than a gameplay script.

As reported by @SK83RJOSH: when collision bugs occur, developers would benefit from a way of visualising the colliders for a given object. This exists in the platform, but needs to be brought over.

We might be able to support cargo run in a Ambient project folder; we need to add this to Cargo.toml (for example in the examples):

[target.wasm32-wasi]
runner = "ambient"

To make this work we would also need to make the asset build part of the cargo build. Needs to be experimented with a bit though.

Provide some kind of default lighting; right now everything looks flat because there's no "sun" entity set up.

At present, PlayerJoin/Leave events are being called for every player when a module loads/unloads. We should remove this and have explicit events for loading/unloading.

[platform-name]://192.168.0.3 should connect you to that server. Perhaps with a dialog asking you what your username should be?

EntityIds are now uuids (u128), so we can get rid of the EntityUid s and just use EntityIds in their place.

I.e. this file should be removed: https://github.com/KiwiOrg/Kiwi/blob/main/crates/ecs/src/entity_uid.rs
And anywhere where we are currently using EntityUid we should switch to using EntityId.

• Terrain
• Sky
• Editor

At the time of creating this issue, we are using a relatively old (Sept 2022) version of wasmtime. We have been unable to update due to the ongoing work with wit-bindgen.

However, it looks like this work is soon coming to an end: bytecodealliance/wasm-tools#867

We should hopefully be able to update soon.

https://github.com/crate-ci/cargo-release

Should already work mostly out of the box, but we should ensure version numbers are updated everywhere, including:

• primary workspace version
• Rust guest version
• all examples that depend on the Rust guest version

At present, info logs from our crates and application are not visible unless you specify them explicitly or opt into the firehose with RUST_LOG=info. We should set up a reasonable logging level for all of our crates and ignore the more spammy ones (wgpu etc) by default.

A tilt.toml describes projects. It should include:

• name: a ASCII snake-case identifier (should this be id instead?)
• display_name: a display name to use in UIs and such (should this be name instead?)
• organization: the name of the organization this belongs to (optional, may be relocated elsewhere)
• dependencies: other projects this project depends on; at first, Git dependencies only
• components: a list of components defined by this project

At present, the auto-generated components.rs is not populated as Elements does not have any decorated components. To alleviate this, we should export all Elements-relevant components to a local TOML, and import that by default.

As reported by @SK83RJOSH: there is currently no way to get or set linear velocity for physics objects. This could be exposed as functions, but it would be nice to do it through components (and to make sure the rest of the API is consistent with this choice.)

Only concern I have is that that might introduce weird frame delay issues or conflicts resulting from the component being both driven and driving. Need to discuss.

type T = {Vec,Option}<{EntityUID, String, ObjectRef}>;
entity::set_component(entity_id, Component<T>, T::a_value);

Doesn't work, as the prototype is

pub fn set_component<T: SupportedComponentTypeSet>(
    entity: EntityId,
    component: Component<T>,
    value: T,
);

and SupportedComponentTypeSet is only implemented for the reference to containers of owning types:
impl<'a> SupportedComponentTypeSet for &'a Vec<EntityUID>

I need to figure out a way to:

• ensure that component and value refer to the same type
• value's type can be coerced to the closest supported SupportedComponentTypeSet

so that
set_component(id, nodes(), vec![id1, id2]);

type-checks (it doesn't currently because Vec<EntityId> does not implement SupportedComponentTypeSet) and the implementation coerces value to a &SupportedComponentTypeSet.

Maybe an intermediate trait?

Blocked on #1 (unless I add in another layer of conversion)

Scripting currently has no way to know what the player's inputs are. For now, we'll send their keyboard and mouse state when the window is focused. We can investigate more complicated schemes later.

There are currently three scripting interfaces (base, runtime, and platform (not in this repository)). These should be unified so that there's only one interface.

If an error occurs in a polled future (e.g. as part of an async execution) in the guest, there's very little information about where it came from. The event subscription should store information about the subscription itself so that users have more information about where runtime errors are happening.

Tilt, at present, always installs a rustc. However, a lot of our initial users are likely to have a Rust toolchain installed. If the runtime/asset pipeline (#15) detects a system rustc that's above the target version (as set by https://github.com/TiltOrg/Dims/blob/2b79b7539676a80d978d017d5b10542e1d273c47/elements/crates/scripting/host/src/server/mod.rs#L45), it should use that instead of installing a new version.

After #15 is done, all major developer interaction will be through the asset pipeline. Hot reloading will take a fair bit of work to get right, so in the meantime, we should add a way to rebuild and reload all assets and the scene. This would remove the existing world, rebuild all assets (including models, sounds, scripting, etc), and then recreate everything.

Suggested hotkey is Ctrl+R.

As reported by @SK83RJOSH: The player_camera component is undocumented, and is thus not exported to the TOML. Easy fix.

I imported a number of meshes from Kenney NL and close to half of them failed to cook:

[2023-02-12T17:38:01Z ERROR kiwi_model_import::model_crate] Failed to cook triangle mesh: Success
C:\Users\Josh\.cargo\git\checkouts\physx-rs-0df26257bd263938\92a09a9\physx-sys\PhysX/physx\source/physxcooking/src/mesh\TriangleMeshBuilder.cpp (1043) : internal error : cleaning the mesh failed

examples.zip

We can use the text component etc. to draw UI, even without a UI "framework" (like porting over the elements library)

cargo-dist is Axo's solution to easy cross-platform builds with CI. I've set it up with two repositories and have found it to be relatively easy to use.

In general, all assets (scripts, models, sounds etc.) have three "phases" in Ambient;

• Building: We take a source asset (.fbx, .wav, a rust crate, etc.) and turn it into a format consumable by Ambient
• Serving/loading: That asset is loaded into memory by Ambient
• Hot reloading: The source asset has changed, so we re-run the two first steps and update the game with the new asset

Building
The core idea here is that conceptually we always run a full build. However, the build consist of "build nodes", for instance we might have a PipeImage node which takes a source image, transforms it (resize for instance) and then outputs the result in the target directory. The build node can check if the output file(s) is newer than the input file(s), and if that's the case it can skip that build node.

Builds can be triggered in a number of ways, such as running ambient build or by the hot-reload watcher detecting that any of the source input files have changed

Serving/Loading
Once the assets have been built, they can be served. The important thing here though is that we have to assume that they are served over the network. We also have to assume that they might be served from a static http content server. More over, the assets are normally also cached on the client side (see BytesFromUrl).

Hot reloading
Hot reloading consists of two parts; on the server side we need to detect when anything changes in the source files, and then we should run a full build. And then once the build has finished, we need to figure out which output assets have changed, and then notify all clients (including the running server) which assets have been invalidated. They then need to re-load those assets, potentially with some cache breaking (like for instance adding ?cache_break=random_id to the url).

Need to get rid of git dependency:

• wit-bindgen-guest-rust https://github.com/KiwiOrg/Kiwi/blob/main/guest/rust/api/Cargo.toml#L16

At present, the component TOML, used in the scripting interface and to decorate the system-defined components, is generated by a private tool. Ideally, we would use component attributes to include this information next to the core components themselves (so that they do not require manual decoration) and so that the TOML can be generated from the ECS directly.

At present, we shell out to ffmpeg to transcode audio to ogg. To reduce the dependency on external utilities (especially on Windows), we should consider using the symphonia crate to decode all supported audio formats, and then encode it with one of the ogg encoding crates.

As of 78801f9, we can now load TOMLs that describe components to add to the ECS. Projects should also be able to define their own components using their own components.toml file at the root.

Future work: how do dependencies work? Can assets have their own components.toml? What happens if you define a component that conflicts with an existing definition?

Should offer a few helper functions to make it easy for people to create and manipulate cameras. Choosing type of camera projection, field of view, aspect ratio, etc

Our tangent calculations might be wrong. Bevy is using this: https://crates.io/crates/mikktspace which they say is industry standard.

From the last bevy blog post: https://bevyengine.org/news/bevy-0-8/#automatic-mesh-tangent-generation

@ten3roberts Might affect PBR stuff

Running kiwi run from within any example's directory that uses assets leads to errors in the log despite the corresponding asset's main.json existing in target/assets (note that many permutations of the path were tried, and failed):

[2023-02-12T17:35:04Z WARN  kiwi_std::uncategorized::download_asset] Request for http://192.168.1.66:8999/assets/castle.glb/objects/main.json failed: 404
[2023-02-12T17:35:04Z ERROR kiwi_object] Failed to load object: Downloading http://192.168.1.66:8999/assets/castle.glb/objects/main.json failed, bad status code: 404

Full dependency support is a lot of work. For now, we should support relative dependencies and see where that takes us:

[project]
id = "skinmesh"
name = "Skinmesh"
version = "0.0.1"

[dependencies]
peasant_man = { path = "../peasant_man" }

This should make peasant_man's assets and components accessible to skinmesh. Additionally, if peasant_man introduces any scripts, they should be automatically added to the project, with an opt-out available (this may change if we develop support for spawning scripts from other scripts).

This does not support updating meshes or more complicated functionality. This is purely to enable basic procedural mesh generation.

A basic API for creating assets at runtime, just to enable simple creation, and only on the client.

mod experimental {
  mod texture {
   fn create_2d(data: &[u8], format: texture::Format) -> Result<String, TextureCreateError> {}
  }

  mod material {
    fn create(diffuse_url: &str, /* material parameters */) -> Result<String, MaterialCreateError> {}
  }

  mod mesh {
    fn create(vertices: &[Vertex], indices: &[u32]) -> Result<String, MeshCreateError> {}
  }
}

use experimental::*;

let heightmap = make_heightmap();

let (vertices, indices) = convert_heightmap_to_mesh(&heightmap);
let mesh_url = mesh::create(&vertices, &indices)?;

let diffuse_url = convert_heightmap_to_diffuse(&heightmap);
let diffuse_tex_url = texture::create_2d(&diffuse_url, texture::Format::Rgba8Unorm)?;
let material_url = material::create(&diffuse_tex_url, /* ... */)?;

let id = make_transformable()
  .with(components::mesh_from_url(), mesh_url)
  .with(components::material_from_url(), material_url)
  .spawn();

At present, the scripting uses a hand-rolled async runtime in each script that's sure to have problems. Ideally, we'd be using a battle-tested runtime.

I believe it's possible to use Tokio as the runtime with the recent work to make it WASM-compatible. This should be done after the event dispatching is cleaned up.

The element crate is our own "React" implementation, which is able to render a tree of UI based on state. It currently only runs in the native context, but we could potentially make it support the wasm target too (and make it depend on our wasm api). If we do that, we could split up the ui crate into ui and ui_components, where ui would just be the native stuff, and ui_components would be supported on both native and wasm, and would contain things like Button etc.

WASM that runs on the client, so that you can draw / render things, have clientside-only effects, etc.

This is related to Rust Analyzer introducing bad paths in some way. Tried using dunce::canonicalize + some other dirty tricks (CARGO_WORKSPACE_DIR + .cargo/config.toml) but the issue persists.

Example of bad paths:
/* This file was copied from "C:/\\/Users/Josh/Desktop/Kiwi/crates/wasm/wit/guest.wit". Do not edit it directly. */

There's currently a gpu_components macro here: https://github.com/AmbientOrg/Ambient/blob/main/crates/core/src/gpu_ecs/component.rs#L224

This should be removed, and instead we should introduce a Gpu component attribute (similar to how we have Networked, Debuggable etc.), which should indicate that the component should be synced to the gpu and accessible from shaders.

The Gpu attribute should also be able to specify how things are synced; CpuToGpu (default), GpuToCpu or None. Then the gpu_systems can be replaced by something which does this for all components (for instance this one: https://github.com/AmbientOrg/Ambient/blob/22c337dda5af5414cce47314c556f36b8638bc19/crates/renderer/src/lod.rs#L83).

For anything using MappedComponentToGpuSystem, we should probably just have two components. For instance in skinning, we could have skin and gpu_skin and have a normal system which writes from skin to the gpu_skin, which is then synced to the gpu.