rojo-rbx / rojo

Rojo enables Roblox developers to use professional-grade software engineering tools

 

---

Rojo is a tool designed to enable Roblox developers to use professional-grade software engineering tools.

With Rojo, it's possible to use industry-leading tools like Visual Studio Code and Git.

Rojo is designed for power users who want to use the best tools available for building games, libraries, and plugins.

Features

Rojo enables:

• Working on scripts and models from the filesystem, in your favorite editor
• Versioning your game, library, or plugin using Git or another VCS
• Streaming rbxmx and rbxm models into your game in real time
• Packaging and deploying your project to Roblox.com from the command line

In the future, Rojo will be able to:

• Sync instances from Roblox Studio to the filesystem
• Automatically convert your existing game to work with Rojo
• Import custom instances like MoonScript code

Documentation

Documentation is hosted in the rojo.space repository.

Contributing

Check out our contribution guide for detailed instructions for helping work on Rojo!

Pull requests are welcome!

Rojo supports Rust 1.70.0 and newer. The minimum supported version of Rust is based on the latest versions of the dependencies that Rojo has.

License

Rojo is available under the terms of the Mozilla Public License, Version 2.0. See LICENSE.txt for details.

rojo's Issues

Sync rbxmx/rbxlx files into Studio from filesystem

Not sure how difficult this one will actually be without doing more investigation.

Like #6, this only covers reading from the filesystem and loading into Roblox. The other side is covered by Roblox's save-to-model feature until Rojo implements it.

There may be some server work involved here -- notably, parsing XML is not something that sounds like fun to do in Lua, but with the server trending towards being just a file API, it might be best placed there.

Bi-directional script syncing

If you edit a script in Studio, it should sync back onto the filesystem.

A lot of care needs to be taken to prevent writes from rippling back to Roblox Studio, which could be tricky with the filesystem watcher.

Partition-level ignore field

I've updated this issue to line up with the 0.5.x project format, since this has been a long-requested feature!

It'd be nice to be able to ignore instances by name on an instance-by-instance basis when live syncing. It would act like a more careful version of $ignoreUnknownInstances.

For example, if I want to ignore my StarterCharacter inside of StarterPlayer because I don't want Rojo to manage it, I could use:

{
  "name": "ignore-example",
  "tree": {
    "$className": "DataModel",

    "StarterPlayer": {
      "$className": "StarterPlayer",
      "$path": "StarterPlayer",
      "$ignoreInstances": [
        "StarterCharacter"
      ]
    }
  }
}

Notes:

• Should we call this $ignoreInstances, $ignoreChildren, or something else?
• Is there a way we can clarify that this setting only affects live-sync?
  • Should we group this and $ignoreUnknownInstances into a new $liveSync settings dictionary?

Setup automated build/tests

Rojo lacks any tests or continuous integration, making it hard to validate errors in pull requests/branches.

Protocol v2 Redesign

This is a tracking issue for #56.

Improve server error messages

Some of the error messages Rojo has are pretty bad; any place where Rojo can panic, those error messages should be customized in each case to make things more obvious.

For example, if a partition target is missing on the filesystem, you get:

$ rojo serve
Using project from ...
Server listening on port 8000
thread '<unnamed>' panicked at 'Unable to watch path!: Generic("Input watch path is neither a file nor a directory.")', src\libcore\result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Plugin version tracking

Rojo should set a user setting describing the last version that was used on this Studio instance.

When Rojo is installed for the first time, we should display a welcome message, and when Rojo is upgraded, we should make sure the user knows if they need to update their server!

Switch protocol to long-polling instead of spamming requests

This would drastically reduce Rojo's syncing latency and reduce the number of HTTP requests that are burned on syncing.

Server write/delete API

I'd really like for this API to be atomic with regards to folder writes.

Add version number to project file?

Before making any breaking changes, it seems like it would be a good idea to add a (required?) version parameter to the file to introduce new mechanics without breaking existing files.

Document object deletion behavior

Other file syncing plugins for Roblox Studio avoid deleting existing objects.

This creates problems when objects are deleted, since Roblox Studio's copy of the code won't be updated. When bi-directional syncing is implemented, this gets especially problematic, making it difficult to delete files.

Rojo is a bit different in that anything under a sync partition that isn't in the source copy will be mercilessly deleted. Because the recommended strategy is to work entirely on the filesystem, that shouldn't be a problem.

In the future, when bidirectional syncing is implemented, the recommended way to resolve conflicts will be to commit whatever you have into your VCS, then sync from Roblox onto the filesystem and resolve conflicts there.

Implement 'rojo build'

One of Rojo's goals is to replace rbxpacker.

This is in the form of a command called rojo pack, which needs to be fleshed out to work with things like multiple partitions.

Rename config file to something more generic?

Right now, the config file (rojo.json) is super specific, but I think that the config format could be more generally useful for other Roblox projects.

Should Rojo's project format be named something more generic, like rbx-project.json?

Add color to output

I'm not sure if the Rust ecosystem has a good colorizing library that supports all platforms well (especially weird configurations like Windows with Git Bash via cmd.exe).

If there is one, we should add it to the Rojo binary and issue warnings, errors, and other information messages in cool colors to catch peoples' attention.

Issue a warning on duplicate sibling names

Roblox lets you have multiple siblings with the same Name.

This is a problem on the filesystem, and Rojo should disallow these cases. Even in cases where you have foo.lua and foo.server.lua in the same directory, this poses a problem for reconciliation.

I gave this a try with two different techniques:

• Warn after any writes to Roblox by crawling the whole tree
• Warn by checking VFS representation returned by the API

I'm putting this off until later, since neither attempt covered all edge cases. Anyone else is welcome to try!

Duplicated scripts synced into tree

I noticed this working on hack week with another engineer, where we reopened our place file and started/stopped polling a lot. We'd end up with one or two different scripts duplicated multiple times. This manifested as Rojo looking like it was broken, since the changes we were making didn't look like they were applying.

If you hit this bug, it's safe to revert to 0.3.x until I get around to fixing 0.4.0 with a patch release.

Disabled script suffix

Can we have an option to automatically add a suffix to disabled scripts?

Like, if I have a server script called Foobar that's disabled, I want it to write as Foobar.DISABLED.server.lua

Malformed JSON results in a "Using default project" instead of useful error message

This is confusing and misleading to new users

Support using game as a partition target

Other syncing systems, like studio-bridge or RbxRefresh, sync from the top of the Roblox hierarchy.

As part of fixing service support when syncing into game, Rojo should also support this use case. Notably, this means not wiping away unknown objects when the current item is game.

High CPU usage in a small project

Holy cow this is some high CPU usage:

Rojo needs to not do that. My hunch is that the file watcher either has a bug or is falling back to polling because of some missing API support.

Warn if no partitions are specified during `rojo serve`

When people first pick up Rojo, they often forget to add partitions.

rojo serve should issue a warning if there aren't any specified!

Rojo Roadmap Meta-Issue

Aim high!

Bold things are high-impact changes. Hoping to do things in roughly this order:

• Fix pile of bugs reported by Tiffany
• Filter plugin implementation (#15)
• Sync JSON models into Studio (#6)
• 0.4.0 Release (finally)
• Stateful IDs redesign (#44)
• 0.5.0 Release
• Implement write/delete support in the server (#2)
• Script syncing from Studio (#5)
• Build a nifty UI in Roact (#3)
• Plugin configuration (#4)
• Sync rbxmx/rbxlx models into Studio (#7)
• Sync models onto filesystem in either JSON or rbxmx/rbxlx

Potential for syncing wrong project exists (especially when switching projects)

Serving from the wrong repository makes rojo have the potential to serve bad content to a place. This in turns results in artifacts left in the place, which is bad.

Tests on CI

I pulled in TestEZ as a dependency to start writing automated tests for the plugin.

Some steps:

• Add features to Lemur to cover Rojo
• Write test harness for plugin
• Travis-CI Setup

Configuration

I want to store configuration both in the rojo.json project and in a folder named Rojo as a direct descendant of the data model.

In the future, we'll want an API for writing configuration values to the server. For now, the configuration will just be:

• Host/port to look for a server on
• Project name, to prevent mixing up two projects' codebases

Dedicated docs website

I think that Rojo is complicated enough to warrant a dedicated docs website instead of having everything in the README.

Sync JSON files into Studio from filesystem

This is something that Studio Bridge does that's really cool.

This feature should be implementable completely in the plugin, as parsing JSON and assigning some values isn't too crazy.

Some care will need to be taken to designing a format on disk that represents object hierarchies well.

Syncing back onto the filesystem isn't covered by this ticket, as that'll require some ReflectionMetadata trickery without extra Roblox APIs.

Show changelog in plugin

Whenever the plugin or server update, it would useful to pop up a little window describing what changed!

Should not have to specify type of property for basic value types in json

I know serde json parses using this type system, but it'd be really nice if properties didn't have to be specified as boolean, number, string, et cetera.

Plugin Icons

I'd like to have icons for all of the buttons in the plugin to make it a lot nicer.

Plugin: Syncing with path pointing to service errors

Plugin issue:

Handle team create

Need to set CurrentEditor for stuff being synced, so make team create obvious

Booleans and numbers are not supported in json models

{
  "name": "InsertDefaultModules",
  "className": "BoolValue",
  "children": [ ],
  "properties": {
  	"Value": {
      "type": "boolean",
      "value": true
    }
  }
}

Expected behavior: It works

Actual behavior: errors

Project Setup Wizard

I think this should be pretty straightforward with the Rust ecosystem, but it might be tedious.

A rough draft of questions that should be asked:

• Project name? (rojo-project)
• Partition list (default to src mapping to ReplicatedStorage?)
  • Each partition needs to prompt for name, path, and Roblox target
• Default serve port? (I kind of want to remove this option from the config object by default)

Plugin should allow user to choose port to listen on

Sometimes I like to work on two projects at the same time. While I can change the port number to serve on in rojo.json, I don't believe there's an option to configure the port the plugin should look for.

If this were to be implemented, it would also be good to give a better error message when two Rojo instances serve on the same port, and you attempt to sync. The current one when I have a partition for game.ServerScriptService is:

19:19:24.734 - The Parent property of ServerScriptService is locked
19:19:24.734 - Stack Begin
19:19:24.735 - Script 'Plugin_1211549683.Rojo.Reconciler', Line 169 - field reconcile
19:19:24.735 - Script 'Plugin_1211549683.Rojo.Reconciler', Line 266 - field reconcileRoute
19:19:24.736 - Script 'Plugin_1211549683.Rojo.Plugin', Line 120 - method _pull
19:19:24.736 - Script 'Plugin_1211549683.Rojo.Plugin', Line 172
19:19:24.737 - Stack End

EDIT: Didn't pay attention to Issue #3 when I searched. My bad.

Invalid Windows Filenames

Forgive me if this has already been accounted for, I'm not at home to test...

In my game, Battle Hats, I have a folder for every hat in the game. The hat that destroys every solution under the sun that's not my awful homebrew one is "Noob Attack: Periastron", because you can't use colons in Windows. In my homebrew solution, I replace the colon with a smaller unicode colon. Is there anything like this, even just by removing the colon, in rojo?

Protocol v1 Design Issue

There's an issue with the new protocol (v1) that I'm trying to resolve but having trouble with.

Given the following filesystem tree:

src
|-- foo.lua

Rojo should produce the following Roblox hierarchy:

src (Folder)
|-- foo (ModuleScript)

If foo.lua is renamed to foo.not-lua, the file watcher will tell the plugin that these paths changed:

[["src", "foo.lua"], ["src", "foo.not-lua"]]

The plugin issues a read for these paths, and gets back this result from the server:

[null, ["details about foo.not-lua"]]

The plugin tries to reconcile the ["src", "foo.lua"] to be nil, but won't be able to find anything to delete, since the object is named foo, not foo.lua -- we'd have to map the file name to a Name-ClassName pair!

The plugin ends up creating this tree:

src (Folder)
|-- foo (ModuleScript)
`-- foo.not-lua (StringValue)

This differs from the tree we would get if we did a fresh sync, which is not good!

The protocol needs to be modified to communicate paths in a different way, but I'm not entirely sure how to progress.

Soution 1: Use Name-ClassName pairs for all route entries

Instead of getting a change as ["src", "foo.lua"], we'd get a change as [["src", "Folder"], ["foo", "ModuleScript"]] -- this requires plugins to do a little bit more work when transforming files, but not too much more.

This runs into a problem when it comes to the current JsonModelPlugin implementation -- we don't know the ClassName of an object until we read the file. If a file is deleted, we have no way to recover the identity of the object with this idea! Normally, we'd be able to change the plugin's convention to include the ClassName in the file name, but that won't be possible when rbxlx and rbxmx support is added, which have arbitrary parent containers.

Solution 2: Keep an index that maps routes to instances

This solution would introduce a map from VFS routes to Roblox instances in the plugin while it's syncing.

When the plugin picks up a change from the server, it figures out what object is affected, and can use this to destroy that object if a null result is received. Any time an object is modified, the index would be updated.

Since partial sync operations only happen when a portion of the tree is read, the map doesn't need to stick around for very long.

The major issue I see occurs when the user mutates objects that are in the index (which will happen with bi-directional syncing) it might be difficult to update or discard the correct portions of the index.

Solution 3: Keep the entire tree in-memory on the server

This acts like an extension to solution 2, but implemented on the server. Instead of reading from the disk on demand, Rojo can limit reads to when files change, keeping the results in memory. The total number of reads should be the same, so performance should be fine.

This solution has the same advantages as solution 2, but with the advantage that the user can't mutate objects in the index directly, simplifying a lot of operations.

Targeting a service with a partition causes syncing errors

If you set up a partition that targets ReplicatedStorage, with the hope that the partition's contents would be copied directly in, the plugin throws an error trying to set the Parent property of the service.

It shouldn't do that, it should be content with services as folders!

Syncing into non-default services creates folders instead

Right now, if a sync targets a service that isn't instantiated by default like Teams or TestService, that service will just be created as a folder.

The reconciler should attempt to instantiate services inside of the data model before trying to create folders.

The easy workaround for now is to insert the services before trying to sync with Rojo. I think that's reasonable.

First version of plugin UI

It should handle connecting to the server given a port, exposing a little bit of configuration.

We can reduce the number of toolbar buttons with this, which will be nice.

Support assets via rbxasset loopholes

Way back when, it used to be possible to specify files on-disk for textures, sounds, etc.

It'd be cool to support using hacks like that to manage assets from within Rojo!

Rewrite reconciler again

The reconciliation algorithm from 0.3.x didn't map well onto the new protocol in 0.4.0, so I replaced the entire reconciler with something really naive:

--[[
	An incredibly dumb algorithm to reconcile children: delete all of them and
	re-create them!
]]
function Reconciler:_reconcileChildren(rbx, item)
	-- Make sure we clean up any straggling route references.
	self._routeMap:removeRbxDescendants(rbx)
	rbx:ClearAllChildren()

	for _, child in ipairs(item.Children) do
		self:_reify(child).Parent = rbx
	end
end

I knew that the code would be sacrificing performance for simplicity in the code, but I guess I just didn't anticipate just how poorly the algorithm scales with large codebases.

Additionally, in hot reloading scenarios, the updated reconciler regresses their behavior pretty badly.

Argument 1 missing or nil

I'm trying to use rojo. When I start the server and press Sync In, I get this error.

Syncing from server...
21:24:25.214 - Argument 1 missing or nil
21:24:25.214 - Argument 1 missing or nil
21:24:25.215 - Stack Begin
21:24:25.216 - Script 'Plugin_1211549683.Rojo.Reconciler', Line 265 - field reconcileRoute
21:24:25.216 - Script 'Plugin_1211549683.Rojo.Plugin', Line 120 - method _pull
21:24:25.216 - Script 'Plugin_1211549683.Rojo.Plugin', Line 172
21:24:25.217 - Stack End

This is my rojo.json

{
  "name": "battle-hats",
  "servePort": 8000,
  "partitions": {
    "game": {
      "path": "",
      "target": ""
    }
  }
}

Overlapping partitions should trigger warnings

I discovered this issue working on the Rojo plugin itself.

The plugin currently has a partition targeting both ReplicatedStorage.Rojo and ReplicatedStorage.Rojo.modules.[library], which means the behavior is dependent on which partition is loaded first.

In this case, on my system. it means that only the plugin source and Rodux are synced into the tree! I'm surprised it took me so long to figure out that the configuration was broken -- it should warn in cases like this!

Support Different Types of Values without using model json

From what I can tell the only way to create value objects that aren't StringValues is to create a model.json file (Although the version I'm currently using, 0.3.1, doesn't support model.json). Could you support specifying a value type in the file (probably in json format) or the filename?

One way you could indicate a different value type is to append a .type at the end of the file name. So if you wanted an IntValue, you could have a filename of TestValue.int for example.

Filter plugins

One extension point that Rojo can have is the concept of a filter plugin.

When a file is read by Rojo into the plugin (or into an installation package) it can be run through a filter before it's processed.

Some good filter ideas:

• One that executes an external tool like moonc and uses its output as a replacement (one-way!)
• One that expands rbxmx and rbxlx files into objects, as well as writing back

Doc links in plugin

When the dev plugin is connected to a Rojo server, it should be possible to have links in the plugin that open in the user's browser.

This would be an API endpoint in the server that accepts a URL. From there, it should be trivial to implement in the server.

Handle protocol version mismatch

All currently released versions of Rojo use the same protocol version, 0.

However, it doesn't matter much, because the plugin doesn't check for it!

Before any breaking changes happen in the protocol, an upgrade policy should be put into place. Additionally, the plugin should be able to recommend that the user update their version of the Rojo server if it's out of date.

Use CollectionService to tag relevant instances

If all of the instances that Rojo is managing are tagged with CollectionService, it becomes easy to clean out and helps prevent Rojo clobbering things it shouldn't.

Automatically reload project file

Right now, when working on the configuration of a Rojo project using the server, the server has to be restarted.

It'd be pretty handy to have the server watch the project file and automatically restart itself if it changes.