lune-org / lune Goto Github PK
View Code? Open in Web Editor NEWA standalone Luau runtime
Home Page: https://lune-org.github.io/docs
License: Mozilla Public License 2.0
A standalone Luau runtime
Home Page: https://lune-org.github.io/docs
License: Mozilla Public License 2.0
I'm opening this specifically for emoji url encoding for discord, but url encoding has many usecases. Just a nice feature to have.
Our Luau type definitions are really small (less than 20kb), and also contain documentation and metadata for all Lune APIs, so we could generate both the selene yaml & docs json files from just the definitions file, instead of downloading from GitHub releases, which has been prone to errors in the past.
Right now the setup experience for generating type definitions and documentation is not great and relies on users reading the entire wiki docs. We could improve it quite substantially by making it interactive:
lune --setup
stdio.prompt("multiselect")
.gitignore
, defaulting to yesCode used:
local success, result = pcall(function()
local spawnResult = process.spawn("ffmpeg", '--dry-run', 'cmd', "inherit")
end)
print(success, result)
When trying to spawn a process using any value except for nil
for the 3rd shell argument, an error like this will be raised:
Failed to convert Luau type 'string' into Rust type 'table'!
[Stack Begin]
Script '[C]'
Script 'async', Line 2 - function process.spawn
Script '../test', Line 9
Script '[C]' - function pcall
Script '../test', Line 8 - function convert
Script '../test', Line 49
Script '[C]' - function pcall
Script '../test', Line 42
[Stack End]
This same issue also happens if the 2nd argument is set to an empty string ''
:
local spawnResult = process.spawn("ffmpeg", '', nil, "inherit")
Error message:
Failed to convert Luau type 'string' into Rust type 'Vec'!
Details:
expected table
[Stack Begin]
Script '[C]'
Script 'async', Line 2 - function process.spawn
Script '../test', Line 9
Script '[C]' - function pcall
Script '../test', Line 8 - function convert
Script '../test', Line 49
Script '[C]' - function pcall
Script '../test', Line 42
[Stack End]
Created this issue for the sake of record keeping and further discussion, but this was already very briefly discussed in the Roblox OSS Discord.
There are very few RegEx implementations written in Lua, and the ones that do exist are slow and don't have full feature coverage. Rust already has a performant and feature-complete RegEx implementation, and it just needs to be exposed in a Lua API.
Questions:
RegEx.new("...")
) or via a require (local regex = require("@lune/regex")
)?cc @matthargett
process.os = "linux" | "macos" | "windows"
process.arch = "x86_64" | "aarch64"
We shouldn't need to call out to some external process to determine what kind of machine our script(s) are running on. Luau and Lune currently only support the above operating systems and architectures so this should be pretty straightforward to add.
We could simplify the setup and running process for users with the help of an extension in their editor.
Setup and downloading could be prompted using similarly to how the Rojo VSCode extension does things, it also uses aftman which is already our preferred install method and listed under current installation instructions. This should only happen if lune-specific directories are present.
.gitignore
automatically--list
command showsThe os.date
/ os.time
functions included with Luau are very minimal, and properly dealing with ISO standards and timezones is notoriously difficult. The built-in functions also contain unexpected behavior that does not conform to ISO standards, one example of this being that the first day of the week is Sunday instead of Monday.
We could use the chrono
crate and reference some prior work and make something like a DateTime
library for Lune:
https://github.com/chronotope/chrono
https://tc39.es/proposal-temporal/docs/index.html
https://create.roblox.com/docs/reference/engine/datatypes/DateTime
This would also let us return much more ergonomic values for date/time in future additions such as #51 .
Right now we only have aftman and direct download from GitHub releases as viable installation methods.
It would be great to have support for more package managers, some examples could be:
Some APIs such as process.spawn
tell the user to look at docs for global types:
The third argument, `options`, can be passed as a dictionary of options to give to the child process.
Refer to the documentation for `ProcessSpawnOptions` for specific option keys and their values.
But global types are not currently included in the output for generated docs.
We should probably also show the luau type signature of functions and members somewhere.
In Lune, we currently have no way to dynamically run source/bytecode whatsoever; loadstring()
is omitted from the environment afaik.
In standalone Luau, due to multiple sandboxing reasons, loadstring()
can only run direct source code. (as opposed to loading raw bytecode in vanilla Lua) The function also deoptimizes the environment of the loaded closure, as it's directly returned and can be modifed/manipulated in any way:
local Closure = loadstring("print('hello world')")
-- Modifying the global env etc
setfenv(Closure, {})
Closure() -- Will err as "print" now doesn't exist in the env!
This default behavior isn't even intended for most using loadstring()
, and disables safeenv
internally in the VM.
I propose some sort of built-in library for Lune like "luau
", which could provide some fairly basic functions and utils for compiling/loading source code or bytecode as-is, as we don't really need to sandbox for this seperate behavior.
Also, these should probably all error for basic logical reasons, like incorrect type input etc. They shouldn't error by themselves though, as these functions would be how a script generally knows if compilation is successful.
function luau.compile(
source: string,
options: {
optimizationLevel: number = 1?,
debugLevel: number = 1?,
}?
): (
bytecode: string | nil,
compileError: string | nil
)
It would take in input source code, as well as an optional options
table for certain basic Luau::Compile
compilation options. If successful, it'd return the binary bytecode file/chunk as a string, and no compiler error ofcourse. (string, nil
) Otherwise, it'd just return nil, string
; string
being the Luau compiler error.
Example Usage:
local Bytecode, Error = luau.compile("do end", {
optimizationLevel = 2,
debugLevel = 0,
})
if not Bytecode and Error then
print("Compiler Error: " .. Error)
stdio.exit(1)
end
...
function luau.load(
source: string,
options: {
debugString: string | nil = nil?,
safeLoad: boolean = false?,
}?
): (
closure: (...any) -> (...any) | nil,
compileError: string | nil
)
This would be almost identical to loadstring()
's API, but also accepting bytecode for source
, just like vanilla Lua. (Luau doesn't have this behavior at all, it wouldn't be default expected behavior)
It would just return the closure as-is, like normal loadstring would. By default, this would mean safeenv
would naturally be disabled in the VM internally for this closure env. However, if options.safeLoad
is set to true
, the closure
return will just proxy back the return of a call, similarly to how require()
would, and not allow the raw closure environment to be modified from outside of itself. This would solve any safeenv VM optimization issues.
safeenv
for VM on Current Environmentfunction luau.setSafeEnv(newValue: boolean)
(Suggested from @TheGreatSageEqualToHeaven) Sets L->safeenv
directly on the current L
state.
There should probably be much more added to interact with low level Luau APIs, but these are just basic suggestions for such a library. If you have any disagreements or issues with these ideas, please lmk!
Similar to how Deno has a compile subcommand for generating a standalone executable, we could pack a Lune script into a standalone executable. Some notes on implementation:
.app
, .dmg
, .msi
, AppImage
, ...Add a REPL so that we can run luau code with the lune runtime without having to create a script, would be useful for modifying Roblox place files interactively from the command line. Suggested this to Remodel before, but since Lune is a successor to Remodel, I'd love for Lune to have it rojo-rbx/remodel#90
Hello ๐
I noticed that errors are getting automatically wrapped up into some opaque type. I can stringify the error and see what was originally thrown, but I can't get back the thrown value. For example, it would be great if I could do something like this:
local function throwError()
error({ message = "error" })
end
local success, err = pcall(throwError)
print("success", success) --> false
print("err", type(err)) --> userdata
print("err.message", err.message) --> throws: attempt to index userdata with 'message'
Remodel is a tool that is widely used in the Roblox community and was one of the motivations for making Lune. It would be great to have some kind of compatibility layer and/or some way to require a module that contains the same APIs that Remodel does. We could use the module support in mlua
for this.
Wally is a package manager specifically developed for the Roblox ecosystem - and as a consequence, require
paths must use the script
global. Requires are however still relative to file, and we could automagically translate paths such as script.Parent.Parent.Foo
to a Lune-compatible path such as "../../Foo"
if the user enables a configuration for it.
We could generate the entire wiki from the type definitions file in the same way we already generate the docs file. Type definitions already contain all globals and their members in a tree structure, similar to how a GitHub wiki repo is structured, with its markdown files and dirs.
It's shorter and vanilla Lua uses the same name.
The generated files from --generate-luau-types
, --generate-selene-types
and --generate-docs-file
don't include types nor documentation for roblox apis.
The ability to create TCP clients and servers.
Currently, I would like to be able to connect to an IRC server through Lune, but cannot due to the inability to open a TCP connection.
A TCP server is probably not required as much since it goes against what this project is trying to achieve, but a TCP client is often required to connect to systems like databases.
Roblox allows usage of _G but it seems like in Lune it is entirely walled off.
The docs says that print respects colors, but it does not. stdio.write does, however.
local stdio = require("@lune/stdio")
stdio.write(stdio.color("red"))
print("This text will be red")
stdio.write(stdio.color("reset"))
print("This text will be normal")
local stdio = require("@lune/stdio")
stdio.write(stdio.color("red"))
stdio.write("This text will be red")
stdio.write(stdio.color("reset"))
stdio.write("This text will be normal")
The library Process
should be merged with the os
library.
Grouping the process library's content inside the os
library would be a good idea for a variety of reasons.
Grouping them as seperate libraries does not make much sense as the purpose of each library is the same, yet they are in different global std libs.
It would make it Lune more familliar to vanilla Lua, lessening confusion to people coming from regular Lua. Also some other languages use the os
name as well.
Also os
is a much more ergonomic name than Process
. os
is much easier to type and remember than Process
.
Terrain.SmoothGrid
/ Terrain.MaterialColors
example:
local roblox = require("@lune/roblox")
local Instance = roblox.Instance
local Color3 = roblox.Color3
local game = Instance.new("DataModel")
local workspace = game:GetService("Workspace")
workspace.Terrain.SmoothGrid = roblox.readModelFile("TerrainRegion.rbxm")[1].SmoothGrid
workspace.Terrain.MaterialColors = "![CDATA[AAAAAAAAAIAAP39rf2Y/ilY+j35fi21PZmxvZbDqw8faiVpHOi4kHh4lZlw76JxKc3trhHtagcLgc4RKxr21zq2UlJSM]]"
roblox.writePlaceFile("test.rbxl", game)
Errors: SmoothGrid is not a valid member of TerrainRegion
We currently have a drop-in migration module for Remodel, which we added in Lune 0.7.0
- remodel.luau
There may be other use cases or tools that would suit Lune well and that are no longer maintained, we could also make migration modules for those. If anyone that has migrated to Lune sees this and would like to contribute to that, I would much appreciate it!
It would be great to have migration as part of our documentation site, too.
I have attached a repro file to this issue named file
. If you rename the file to file.rbxl
and open it with Studio, it opens without issue. Though lune's roblox library is unable to parse it.
To repro, run the following in lune
after unzipping the downloaded file:
local roblox = require("@lune/roblox")
roblox.readPlaceFile("file")
In this file, I get the error Failed to read document from buffer - stream did not contain valid UTF-8
but for other files I'll get Unknown document format
Info as a name is ambigious.
Please support this, I beg. This would be a life saver to me, and my tooling. Tell me if you need anything more information, and I'll be happier to provide it.
PlushNameGui.rbxm
When inserted in Studio, the font is Fredoka One
. When read and written to a place file, the font is Arial (Legacy)
.
When using --generate-selene-types
, the resulting lune.yml file has a couple of issues.
task.spawn
, task.defer
, and task.delay
all require the vararg to be used when it should be optional.task.cancel
takes an argument of type thread
which isn't actually defined anywhere. This actually causes an error within Selene that gives us a confusing error message and prevents it from running. (see below)Title essentially says it all, so here are some simple repro cases.
local stdio = require("@lune/stdio")
stdio.write("First")
stdio.ewrite("Second")
SecondFirst
local stdio = require("@lune/stdio")
stdio.write("First")
print("Second")
Second
First
module.luau
net.request("https://google.com")
return {}
script.luau
require("./module")
There are at least two ways to solve this:
require
async and use async filesystem APIs, but this is really tricky with caching and making sure requires only happen once (users can run require
in a task.spawn
, ..)Note that this bug does not occur unless it is in the main script of the required module, this works fine:
module2.luau
return function()
net.request("https://google.com")
end
script2.luau
require("./module2")()
Any script running in Lune can steal cookies with roblox.getAuthCookie()
, read and write arbitrary files with fs
, and send network traffic with net
.
It's very important to block access to these when running scripts that aren't your own.
An additional argument could be added to the Lune executable that specifies permissions, for example:
lune rwnc arg arg arg
where each letter in the arg rwnc
represents a feature that is allowed.
----
- No perms.
rw--
- File read / write perms.
r-n-
- File read and net perms.
r-nc
- File read, net, and cookie perms.
Etc.
Something like os.listdir()
in Python:
import os, sys
# Open a file
path = "/var/www/html/"
dirs = os.listdir(path)
# This would print all the files and directories
for file in dirs:
print(file)
Which would print:
stamp
faq.htm
_vti_txt
robots.txt
itemlisting
resumelisting
writing_effective_resume.htm
advertisebusiness.htm
papers
resume
Already been discussed in Discord, but creating an issue for formality.
Lune could be an excellent option for unit-testing Roblox scripts outside a Roblox environment. If scripts are decoupled from engine APIs, there's no reason they shouldn't be able to run in an environment like Lune. Lune could support Roblox-like require
s as an opt-in feature, using a Rojo source map to resolve file paths.
In the script environment, this would require a few things:
Instance
userdata which allows for indexing children and getting the parent. Specific instance types shouldn't be necessary; there just needs to be a way to traverse the datamodel.script
global, which is simply the current script's Instance
.game
global, an Instance
which represents the root of the datamodel.
GetService
should be supported so that datamodel services can be indexed in an idiomatic way.require
implementation which supports referencing Instance
s. Much of the existing async require
logic can be reused.In addition, under the existing roblox
global, Lune could support scripts implementing functionality for Roblox services. Lune can't reimplement everything (and it shouldn't aim to), but allowing users to reimplement what they need would be very powerful. Something like this:
roblox.implementService("HttpService", function(service)
service:AddMethod("JSONEncode", ...)
end)
One thing to consider is how this pairs with the existing DataModel implementation for reading place/model files. How much functionality there can be reused?
I don't wanna taint the math.random, and I want to have a psuedorandom thing for my code.
idk if its a me issue or whatever but when i use this the script stops directly
when i run this it just stops running directly after
local port = 12941
local handle = net.serve(port, {
handleWebSocket = function(socket)
print("Got new web socket connection!")
while true do
local message = socket.next()
if message ~= nil then
socket.send("Echo - " .. message)
end
end
print("Web socket disconnected.")
end,
})
print(`Listening on 127.0.0.1:{port} :3`)
Lune has ergonomic access for instances and their properties in the roblox
builtin, meaning you can create, get and set them the exact same way you would in the Roblox engine.
It could be beneficial to take this a step further and allow users to access metadata about these classes and properties, which is currently contained in the reflection database and only exists internally.
typeof()
, a function exclusive to Luau, seems to behave similarly or identical to type()
, which should not be the case.
One example is to feed an instance to typeof
, like typeof(Instance.new("DataModel"))
; this should return Instance
but userdata
is printed instead.
Process.cwd prepends "\?" to the path on Windows, which would force you to use workarounds such as stringLiteral:sub(5)
to be actually able to work with the given path.
For example, on this page I assume these methods take in arguments, but they are not named nor typed resulting in a lot of guessing and checking.
https://lune.gitbook.io/lune/api-reference/fs
fs.rename(from: string, to: string)
fs.rename(from: string, to: string, overwrite: boolean)
fs.rename(from: string, to: string, options: table)
This should throw an error by default if the path to
already exists, and this behavior should be able to be overridden by passing either true
for overwrite or some table of options, in case the user is 100% sure about what they are doing and don't want the extra overhead.
We currently leak web socket objects due to lifetime issues here:
There's no great solution using the mlua
crate directly since web socket streams do not implement Clone
and cannot be properly stored in the lua registry, and we can't use a LuaUserData
since the functions need to be async (without using the native mlua
async feature).
Most current usage of web sockets involves clients or servers with a small amount of connections during the life of the program (eg. things like Discord bots) but this still needs to be fixed as soon as possible.
Some code may be designed to run on both Roblox and on Lune. Right now the cleanest way to determine if the script is running on Lune is to do something like if pcall(require, "@lune/fs") then
which is what I'm doing now, but this seems a bit messy.
Currently there is no way to retrieve file metadata such as creation/modification date, file/folder sizes, and the like. At present Lune does not provide a method to get this information and a workaround is using powershell (or otherwise) and the "Get-Item" command, this does work to a point - retrieving the modification date & file size in bytes, though it's not nearly everything.
My use-case for this is to 'diff' two files (an asset and a place) to ensure that a file that's been modified will be updated and synced so long as it's modification date precedes the last modification date of a target place file, this would then allow me to move away from the not-so-agnostic powershell 'Get-Item' command, and also provide even more useful information.
It would also be helpful if the DateTime library came with this change so parsing of the dates is easier, attached is my solution to the DateTime and file metadata problem (thought do note these files are from when lune was moderately fresh and I was learning the ropes), the main file (for this issue) is canSyncAsset.luau
though to understand it's use I've provided the full sync and build scripts for context.
sync scripts.zip
This code would print Got Cookie but wouldn't print Complete
-- Main Module
local request = require("CookieRequest")
print("Complete")
-- CookieRequest
local roblox = require("@lune/roblox")
local cookie = roblox.getAuthCookie()
print("Got cookie!")
return function()
end
fs.copy(from: string, to: string)
fs.copy(from: string, to: string, overwrite: boolean)
fs.copy(from: string, to: string, options: table)
This should throw an error by default if the path to
already exists, and this behavior should be able to be overridden by passing either true
for overwrite or some table of options, in case the user is 100% sure about what they are doing and don't want the extra overhead.
We will need to take some extra care to handle symlinks properly here.
Lune Version: 0.6.7 (Win64)
Step 1: Create a very deeply nested Instance. Around 2,870 nests should be enough, but I'm using 32,768 nests as an example.
local j = {Instance.new("Part")}
for i=1,32768 do
local part = Instance.new("Part")
part.Parent = j[i]
table.insert(j, part)
end
Step 2: Write that instance to a file or model.
roblox.writeModelFile("./myNestedPlace.rbxm", {j[1]})
Actual result: Lune crashes with a fatal unrecoverable error: thread 'blocking-1' has overflowed its stack
Expected result: Lune either writes the file or complains that it's too deeply nested.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.