konstantintomashevich / emergence

Flexible set of libraries for gameplay development. Currently under development.

Emergence

Emergence is a flexible set of libraries for gameplay development that aims to become modular game engine in the future. Currently under development.

Architecture

Emergence architecture is built upon concept of units: relatively small modular libraries that provide singular feature to the user and depend on one another in order to do it. There is 3 kinds of units:

• Interface units provide only headers.
• Concrete units provide headers and one concrete implementation for them.
• Abstract units provide headers with abstract API and several implementations for them as concrete units.

Every unit is either an object or interface library, therefore it is possible to explicitly control what goes into which output artefact (shared library or executable).

This architecture aims to:

• Make link time polymorphism easy to use through abstract units.
• Make modularization straightforward, easy to define and implement.
• Make it easy to separate modules into several shared libraries.

Features

Core

• Assertions with interactive mode in Assert unit.
• Interface for CPU profiling setup in CPUProfiler unit.
• Interface for byte sequence hashing in Hashing unit.
• Intermediate layer for input registration in InputStorage unit.
• Lightweight job dispatch implementation in JobDispatcher unit.
• Logging with multiple sinks interface in Log unit.
• Math structures for game development in Math unit.
• Memory management structures in Memory unit.
• Memory usage profiling implementation in MemoryProfiler unit.
• High level memory usage recording logic in MemoryRecording unit.
• Low level render abstraction layer in RenderBackend unit.
• Reflection-driven serialization in Serialization unit.
• Fast field-only reflection in StandardLayoutMapping unit.
• Arbitrary task graph execution implementation in TaskExecutor unit with task graph format in TaskCollection unit.
• Testing backend abstraction layer in Testing unit.
• File system virtualization with packaging support in VirtualFileSystem unit.

Resource management

• Simplistic routine for resource cooking implementation in ResourceCooking unit.
• Prefab-like resources with inheritance implementation in ResourceObject unit.
• Resource scanning and loading layer in ResourceProvider unit.

Data management and processing

• Task graph construction with race condition validation in Flow unit.
• Indexed data storage implementation in RecordCollection unit with visualization in RecordCollectionVisualization unit.
• Multi-type data storage with prepared queries support in Warehouse unit.

Celerity

Celerity is an ECS-like game world architecture solution. It provides interface for building modular worlds, that consist of multiple logical parts. For example, tools like unified editor might be modelled as a world with multiple independent sub worlds. It has lots of extensions:

• Assembly pattern implementation in CelerityAssemblyLogic and CelerityAssemblyModel units.
• Asset management routine implementation in CelerityAssetLogic and CelerityAssetModel units.
• Input abstraction layer implementation in CelerityInputLogic and CelerityInputModel units.
• Localization implementation in CelerityLocaleLogic and CelerityLocaleModel units.
• Physics 2d simulation in CelerityPhysics2dLogic and CelerityPhysics2dModel units with optional debug draw in CelerityPhysics2dLogicDebugDraw and CelerityPhysics2dModelDebugDraw units.
• Physics 3d simulation in CelerityPhysics3dLogic and CelerityPhysics3dModel units.
• Simplistic 2d rendering implementation in CelerityRender2dLogic and CelerityRender2dModel units.
• Foundation toolkit for rendering implementation in CelerityRenderFoundationLogic and CelerityRenderFoundationModel units.
• Configuration resources support in CelerityResourceConfigLogic and CelerityResourceConfigModel units.
• Prefab-like resources support in CelerityResourceObjectLogic and CelerityResourceObjectModel units.
• Transform hierarchy with logical/visual split implementation in CelerityTransformLogic and CelerityTransformModel units.
• Simplistic UI implementation in CelerityUILogic and CelerityUIModel units.

Other

• Utilities for command like tools in CommandLine unit.
• Common containers in Container unit.
• Intermediate graph format in VisualGraph unit with export to common formats in ExportGraph unit.
• Lightweight implementation of object handles in Handling unit.
• Utilities for multithreading in Threading unit.
• Utilities for working with time in Time unit.
• Utilities for working with templates and unions in Traits unit.

Executables

• MemoryRecordingClient: tool for viewing MemoryRecording serialized tracks.
• Platformer2dDemo: simple platformer game in early stage of development, used to test new Emergence features.

emergence's Issues

Create 2d platformer with multiple levels to test simple game development pipeline.

We need to create a simple 2d platformer in order to ensure that:

• We can work with multiple levels without problems.
• We can easily implement loading.
• Simplistic UI features is enough for platformer main menu and other menus.
• Current Celerity features and extensions are suitable enough for game logic.

Also, it is needed to draft architecture for planned game creation framework.

CPU Profiler Integration

We need a simple CPU profiling tool integration in order to be able to understand CPU usage.

Celerity: Simplistic UI extension on top of ImGUI and RenderBackend.

We need a UI extension for Celerity that is complete enough to make games with medium UI complexity or editors with limited UI complexity. ImGUI looks like a perfect backend for that task, but we need to hide it under service API.

Initial list of required features:

• Buttons.
• Labels.
• Images.
• Element containers (panels).
• Linear layout (vertical and horizontal).
• Free anchored layout.

Better standard way to place cascade removers in graph

Recently I had to start adding comments like this:

// In fixed update, removal is usually done at the end of the frame after mortality feature,
// which is usually dependent on input processing, therefore we need to make these removers
// dependencies of hierarchy cleanup.

Which is awful, because it's just binding engine-level graph layout to gameplay-specific feature -- mortality.

One possible solution to avoid such complication is to create separate common slot for transform-related cascade removal and place all cascade removers there. The same thing can be done for all cascade removers including asset ones.

Implement "world inheritance" through registry sharing

We need to work with assets and configs in "top level" world in order to save them while transitioning from one scene to another. To implement that we need some kind of "world inheritance" through partial sharing of world registries. For example, if parent world has storage for the type we need, we create query on parent world level instead of child world level.

But thinking of game as "multiple worlds" might be incorrect. We can think of one world with data views tree: this will provide the same multiple registries feature and the same registry sharing, but without exposing the problem of calling update multiple times.

Celerity::Render2d: Implement viewports with optional viewport-to-viewport blending.

Currently Celerity::Render2d works with predefined full-window viewport. We need to change that by adding viewports feature:

• Any number of Viewport instances can be added.
• Rendering is driven by Viewport's.
• Viewports are sorted by sort index.
• Viewport coordinates are configured in pixels.
• Viewports can overlap each other and can be blended: blending is useful for rendering something like UI in separate viewport.

Virtual File System

We need to use VFS layer across the framework instead of plain file system access. Having VFS provides us with ability to:

• Mount multiple roots, therefore asset discovery through VFS would be more versatile.
• Hide actual file access under the hood, therefore we would be able to easily implement read-only packages.
• Easily configure game assets location without hardcode.

Resource cooking platform and game packaging for Windows.

We need a basic resource cooking platform that:

• Takes all resources from mount list.
• Converts yamls to bins.
• Creates resource provider index.
• Packages everything into read-only packages with no directories nesting (to save memory).
• Is easy to extend and add additional steps (like filtering assets and checking whether they are referenced), including game-specific ones.

Also, we need a game packaging script (at least for Windows) that packages game executables and cooked resources into one directory with appropriate mount lists.

Celerity: Extract RenderCommon extension from Render2d extension

Some stuff from Render2d, like textures, materials and material instances, is common for different render extensions. For example, UI may need to use the same texture mechanism as Render2d. We need to extract that common stuff into RenderCommon extension so it is usable for any render-related extension.

Implement RenderBackend service and rework Celerity::Render2d as library that uses RenderBackend.

We cannot hide render backend, like BGFX, under the Celerity::Render2d extension, because other extensions may need to access render backend too. For example, UI needs to render its stuff directly without touching Render2d. In some cases we might want to have UI, Render2d and Render3d working all at once in one window. That means that all render-dependant extensions should be built upon one shared rendering service that will hide real render backend under the hood.

Improve Celerity::RenderFoundation.

Current version of Celerity::RenderFoundation was made as an initial prototype and needs to be updated: we need a proper support for framebuffers, render to textures and other core rendering features.

Serialization rework

Goals:

• Make resource loading easy and straighforward.
• Make resource discovery easy and straighforward.
• Get rid of file separation and make it possible to serialize complex objects.
• Provide simple interface for async file loading.

Proposed changes:

• Add VECTOR and PATCH mapping archetypes in order to cover all use cases with StandardLayoutMapping: any object should be mappable even if field projection is not possible.
• Rework serialization to object loading from StandardLayoutMapping: no more bundles or other special file formats. Everything should be covered by mapping after VECTOR and PATCH addition.
• Split JobDispatcher jobs into foreground and background, then add ability to limit thread usage by background threads. Otherwise resource loading would be able to capture all threads and pause the application.
• Create separate ResourceRegistry library for resource discovery and easier resource loading. The main principles are:
  • All possible resource types should be registered.
  • Every resource object consists of {ObjectTypeName}.{UniqueId}.{bin/yaml}. Is {ObjectTypeName} part of {UniqueId}? To be decided.
  • Other resources files can be registered as external binaries.
  • Resource discovery can be run automatically or loaded from binary discovery cache.
  • There is an API for easy loading by type+id pair and easy iteration of all objects of requested type.

Implement Input extension for Celerity

Celerity needs an input processing extension, that can be used both for device input (for now only keyboard) and for external input (for example from UI extension). SpaceShooterDemo already has an input system, we need to rework it in a more-Celerity-way and add proper support of external actions dispatch to it.

Experiment: Consider Model-Logic split in Celerity and its extensions

Data structures definitions are required everywhere, but not logic, connected to them. For example, cooking tool does not need to know about physics implementation, but still needs to convert physics configs and prefabs with physics components.

I'm not sure whether it is a real problem and whether it can be solved in a pretty way or not, but it might be worth to consider different solutions.