facebookexperimental / starlark-rust Goto Github PK
View Code? Open in Web Editor NEWA Rust implementation of the Starlark language
License: Apache License 2.0
starlark-rust's People
Contributors
Stargazers
Watchers
Forkers
qxtech jsgf philipcraig mythra aiuto pierd joelmarcey indygreg icodein willstott101 stepancheg cyberflamego dmitryvinn kcarretto alienscience dierksen ndmitchell isabella232 danieleliad hulto dtolnay-contrib lhvy pgavlin eastwooder console9 doytsujin benwr realtimetodie silvergasp stephanos maartenstaa sluongng mikelikespie ahupp bahdotsh iq-scm steve-downey bjulian5 penglei mapbased sunaemon olivercalder cameron-martin theonlykingpin yonasbsd dsheets kon72starlark-rust's Issues
Support error codes
Broken out from #7. @Mythra maintains a list of detailed descriptions for error codes for a project they are working on. These include error codes for Starlark errors. It would be great if Starlark could provide and integrate them, at low cost. The rough plan is:
- Each
enumvariant in our error enums becomes a named error code, e.g.ValueError::IntegerOverflowmight get the codeSTARLARK::INTEGER_OVERFLOW. For Lint errors we convert the enum into a code in a similar manner, so that code/approach can be reused. - We show those codes in the
Diagnostictype, either optionally or always (we'll see how messy they make things, but probably always). - @Mythra provides a markdown file with more information on each code. One potential format would be a single
error_codes.mdfile with section headings## STARLARK::INTEGER_OVERFLOWfollowed by arbitrary markdown content about the error. - The Starlark library uses
include_str!()to make that available, as a.describe(&self)method onDiagnostic, and also as aDiagnostic::describe_code(&str)method to enable implementing--describe=INTEGER_OVERFLOWon the command line.
While the steps are in logical order, in truth writing the Markdown file is the biggest effort by a huge margin, so that will probably come first - the actual technical tweaks are probably an hour in total, so I'll happily do them once a somewhat complete Markdown file arrives.
Given substructure positions for errors
Broken out from #7, using the idea from @Mythra. Once we have #11, we get a path of what was wrong with a value - e.g. we'll know that [1, "test"] has an error at index 1. Combined with positional information for the initial value (which is pretty easy to wire through), and an AST for the module (which we can recreate from the full original source in the codemap), we can observe if the argument is a literal list. If it is, we can then give a span that more precisely targets the field within that list that is wrong. That results in better error messages, e.g.
foo([1,"test"])
^^^^^^ -- expected an int
panicked at 'assertion failed: old_local.is_none()'
For given input file:
The following code:
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}
has output:
thread 'main' panicked at 'assertion failed: old_local.is_none()', /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:243:13
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::panicking::panic
3: starlark::eval::compiler::scope::Scope::collect_defines_in_def
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:243:13
4: starlark::eval::compiler::scope::Scope::collect_defines_recursively_in_expr
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:271:13
5: starlark::eval::compiler::scope::Scope::collect_defines_recursively::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:264:34
6: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut
at /home/sxia/code/rust-compiler/library/core/src/ops/function.rs:269:13
7: starlark::syntax::uniplate::<impl starlark::syntax::ast::StmtP<P>>::visit_children_mut::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/syntax/uniplate.rs:110:45
8: core::iter::traits::iterator::Iterator::for_each::call::{{closure}}
at /home/sxia/code/rust-compiler/library/core/src/iter/traits/iterator.rs:722:29
9: core::iter::traits::iterator::Iterator::fold
at /home/sxia/code/rust-compiler/library/core/src/iter/traits/iterator.rs:2159:21
10: core::iter::traits::iterator::Iterator::for_each
at /home/sxia/code/rust-compiler/library/core/src/iter/traits/iterator.rs:725:9
11: starlark::syntax::uniplate::<impl starlark::syntax::ast::StmtP<P>>::visit_children_mut
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/syntax/uniplate.rs:110:17
12: starlark::eval::compiler::scope::Scope::collect_defines_recursively
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:263:9
13: starlark::eval::compiler::scope::Scope::collect_defines_recursively::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:265:34
14: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut
at /home/sxia/code/rust-compiler/library/core/src/ops/function.rs:269:13
15: starlark::syntax::uniplate::<impl starlark::syntax::ast::StmtP<P>>::visit_children_mut::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/syntax/uniplate.rs:83:65
16: <core::slice::iter::IterMut<T> as core::iter::traits::iterator::Iterator>::for_each
at /home/sxia/code/rust-compiler/library/core/src/slice/iter/macros.rs:211:21
17: starlark::syntax::uniplate::<impl starlark::syntax::ast::StmtP<P>>::visit_children_mut
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/syntax/uniplate.rs:83:38
18: starlark::eval::compiler::scope::Scope::collect_defines_recursively
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:263:9
19: starlark::eval::compiler::scope::Scope::enter_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/compiler/scope.rs:195:9
20: starlark::eval::<impl starlark::eval::runtime::evaluator::Evaluator>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/mod.rs:98:25
...
Deal with printing recursive values
$> l = []
$> l.append(l)
$> print(repr(l))
thread 'main' has overflowed its stack
fatal runtime error: stack overflow
Would be good if that worked better.
ansi_term dependency issue
This project depends on structopt, which depends on clap v2, which depends on ansi_term which is unmaintained.
There's a migration guide available to be able to fix this.
Support Rust Stable channel
Opening a tracking issue here, as a carry-over from: google/starlark-rust#283 ๐ , as it seems there were a couple options discussed (RUSTC_BOOTSTRAP in this crate vs others, potentially maintaining a stable only interface, etc.) I'm just opening this to track that. As I'd like to switch over to this crate at some point rather than maintaining my own fork ๐
How should FileLoader::load be implemented?
The function's signature accepts &self and returns &FrozenModule, which I'm not sure how to satisfy with a loader that actually hits the filesystem. If FileLoader::load took &mut self, I could cache the module in a map in the loader and return a reference to that. If it returned an owned FrozenModule then that implementation would be obvious (load the file, parse the ast module, evaluate, freeze, return).
It looks like the only implementors of the trait in this repo work with already-constructed values. Is there a trick to implementing this with some kind of interning I haven't thought of? Or should the API be changed? If the latter, happy to send a PR.
panicked at 'assertion failed: self.kwargs.is_none()'
For given input file:
in.zip
The following code:
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}
has output:
thread 'main' panicked at 'assertion failed: self.kwargs.is_none()', /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/runtime/arguments.rs:155:9
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::panicking::panic
3: starlark::eval::runtime::arguments::ParametersSpec<V>::add
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/runtime/arguments.rs:155:9
4: starlark::eval::runtime::arguments::ParametersSpec<V>::required
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/runtime/arguments.rs:172:9
5: starlark::eval::fragment::def::DefCompiled::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/def.rs:210:56
6: starlark::eval::fragment::def::DefCompiled::as_compiled::ann_expr_def::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:29:33
7: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
8: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:95:54
9: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::ann_stmt_return::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:88:33
10: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
11: starlark::eval::fragment::stmt::StmtsCompiled::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:310:25
12: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
13: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:81:17
14: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:74:21
15: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:92:9
16: starlark::eval::<impl starlark::eval::runtime::evaluator::Evaluator>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/mod.rs:148:19
...
Not sure if it is related to #37
spec compatibility
Hi, Starlark spec author here. I'm very happy to see the Rust implementation improving, and that it now has a garbage collector.
In the Compatibility section of your main README, you list a number of places where the language differs from the spec. It would be unfortunate if the Rust implementation did not eventually converge on the spec, given that it was developed after the spec, and that two existing and largely conformant implementations are available for study.
Let's go through the list:
We have plenty of extensions, e.g. type annotations, lambda, recursion.
Lambda (along with nested def) is now a non-optional feature, so you need no longer mention it here.
The Go and Java implementations permit the dynamic check on recursive function calls to be disabled. Does the Rust implementation also allow it to be controlled? If so, you need not mention it here.
We don't have the common extensions of floats, bit operations, byte strings or sets.
Floats and bit operations are not extensions, they are core features. Fortunately they are easy to add. Consider this a request to do so.
Byte strings will be part of the spec fairly soon; they are somewhat tedious to add because they duplicate a lot of the logic of strings, but they don't pose any fundamental difficulty. The Rust implementation in particular sounds like it needs them because its string type is text, not binary. (In Go, text and binary are both represented using byte strings. The story in the Java implementation is more complicated because of entrenched bugs in Bazel.)
Sets are currently a Go-only optional extension.
Our strings are not compliant in several ways, often returning code points instead of singleton strings, and have poor performance.
I plan to change the spec for strings (which is currently true only for the Go impl, from which the spec was derived) so that that strings are defined as sequences of numeric UTF-K codes where K is implementation defined: 8 (byte) for Go, 16 (char) for Java, with no guarantee that the encoding is valid. Conversions to byte strings (encoding) and back (decoding) would be explicit operations.
What value of K would Rust use? I can imagine 8 might be the natural choice, because Rust strings are UTF-8, but they also require valid encodings, which would make the substring operation fail to cut a code point in half, contravening the spec. An alternative would be UTF-32 (in other words, a sequence of int32 code points), but that would require either an inefficient 4-byte representation, or indexing to take linear time. Another possibility is to represent strings as arbitrary byte arrays and to make the Starlark-Rust API do a validity check and conversion when accessing a Starlark string, either failing, or mending the encoding at that point.
Function arguments are sometimes evaluated in the wrong order, which is visible if their arguments have side effects.
This seems like a simple bug to fix. The Go and Java implementations can be studied as a model, and they contain additional comments that rationalize the spec on this point.
We follow proper lexical binding for list comprehensions, so each for clause introduces a fresh variable (deliberately).
I'm not sure what you mean here. How is this different from what the spec requires and the Go and Java implementations do, which is that the comprehension defines a new block, and a for clause is a binding use of its variables? The hacks in the Java implementation were removed a while back.
We allow comparison between None values (deliberately).
Do you mean ordered comparison? Why deviate from the spec?
In some cases creating circular data structures may lead to stack overflows.
Yeah, this is a problem in all the implementations, and in every imperative language.
We use 32bit fixed size integers. Constructing larger values will result in Starlark failing with an overflow error.
int32 is not enough to represent all the data types that appear in a protocol buffer, for which one needs at least uint64 and int64. Without an int65 data type, you could use int128, but it's easier to use bigint. It really is a nicer language for it, and it needn't be expensive.
I recently changed the Java implementation to use bigint, without loss of efficiency: internally int is a union of three classes similar to java.lang.{Integer,Long,BigInteger}. It was an API change, but not a very disruptive one. Also, I changed the Go implementations to use an mmap trick so that int32 values can be represented as pointers to a special segment, allowing a single pointer to represent a small int or a big int without ambiguity, and without allocation in the int32 case. You might be able to follow the same approach in Rust.
There are a number of minor incompatibilities or places where the spec is unclear, many of which are included in our tests.
Please regard each of these as a bug, either in your implementation, or in our spec, in which case report them to us and we will endeavor to fix them. Starlark users would best served by not having to remember which of three similar dialects they are using in each application, and it's easier to achieve convergence before you have made promises of stability.
cheers
alan
Integers are bounded to 32bit
The spec says:
Integers may be positive or negative, and arbitrarily large.
Currently, integers are limited to 32bit. We should make them arbitrarily large. Currently 32bit integers are packed into the Value pointer, which is something we want to continue doing, so they need to "overflow" into a Rust BigInt stored on the heap when necessary.
Add a ValueParser
Broken out from #7 and other discussions. Currently, values are "parsed" using UnpackValue, which for type T is fn(Value) -> Option<T>. That sucks because it doesn't say what you wanted or what didn't match. Imagine T is Vec<i32>, and the value is [1,"test"]. It should say the failure arises from "test" not being an Int at the 2nd element of the list. That means the return type should be a result with the error as Unexpected:
enum PathElement {
Index(i32),
Field(String),
}
struct Unexpected {
path: Vec<PathElement>, // The path I followed, e.g. index 1
expected: String, // What I wanted, e.g. Int
got: Value, // What I got, e.g. "test"
}
We want this to be the ubiquitous method of getting data from inside Value, to ensure it's used everywhere, and thus we get good error messages always. That requires making it the simplest method and providing good support for this by default - we want manually unpacking a value to be more work.
That means the Unexpected type should be easy to construct, and the ValueParser trait should do it automatically for Vec etc.
We probably also want to be able to generate documentation from ValueParser saying what type the value expects in advance, to be used for auto-documentation and function signatures. That gives us approx:
trait ValueParser {
fn parse(x: Value) -> Result<Self, Unexpected>;
fn sig() -> String;
}
Build broken on MacOs
Did something just update in the past few hours that breaks the build on macOs? My code now fails to compile, showing this error:
Basic Block in function '_ZN7lalrpop3tok9Tokenizer10take_until17h0db094dbe1039a64E' does not have terminator!
label %9
in function _ZN7lalrpop3tok9Tokenizer10take_until17h0db094dbe1039a64E
LLVM ERROR: Broken function found, compilation aborted!
error: could not compile `lalrpop`
...which $ cargo tree shows is being imported by starlark-rust:
โโโ starlark v0.4.0 (https://github.com/facebookexperimental/starlark-rust#094f681e)
...
โ โโโ lalrpop-util v0.19.6
โ โ โโโ regex v1.5.4 (*)
No support for bit operations
main branch doesn't compile. AstSmt & AstExpr size assertions.
Following the guide in the read me I ran into this error.
Updating the size assertions fixes the problem and starlark-rust seems to work as intended.
Rust version:
Implementing TypedValue and the git dependency on gazebo
It looks like right now to impl TypedValue that my type needs to also impl AsTypedValue and by implication AnyLifetime. IIUC, AnyLifetime is defined in gazebo which is published to crates.io but starlark uses it via git. So my crate needs to depend directly on both starlark and gazebo via two git deps. IMO in an ideal world the derive would come from crates.io or would be re-exported by the starlark crate. Do either of those options seem reasonable?
Separately, I think it would be nice if this repo's Cargo.toml could specify the exact gazebo revision it uses, but I'm not sure if y'all's open source export tooling can wire that up correctly. Without specifying the revision it's going to be hard to bisect through the crate's history using cargo. I think Cargo will support revision pins correctly so if I ever need to hold back starlark I should still be able to convince it to use an older compatible gazebo revision, but it would be easiest if starlark's commits included pointers to compatible gazebo commits.
EDIT: It seems Cargo gives me two different instances of gazebo if I specify a revision in my Cargo.toml without one being specified in starlarks. From Cargo.lock:
[[package]]
name = "gazebo"
version = "0.1.0"
source = "git+https://github.com/facebookincubator/gazebo?rev=604500f#604500f2775efa3c36bb53b5241a6911547fabdb"
dependencies = [
"gazebo_derive 0.1.0 (git+https://github.com/facebookincubator/gazebo?rev=604500f)",
]
[[package]]
name = "gazebo"
version = "0.1.0"
source = "git+https://github.com/facebookincubator/gazebo#604500f2775efa3c36bb53b5241a6911547fabdb"
dependencies = [
"gazebo_derive 0.1.0 (git+https://github.com/facebookincubator/gazebo)",
]
Which IIUC means that until starlark specifies a revision for gazebo that I can't pin a starlark version that uses an incompatible gazebo API. Is there a way to include the revision in starlark/Cargo.toml?
LSP: code completion
Hi,
Thanks to @DanielEliad's PR (#51), I've noticed the LSP code in the repo. It's very nice to see an update on this!
I think it's important to keep the LSP generic enough to accomodate multiple dialects of Starlark. I imagine you'll need it at Facebook too. Our approach at Google was to define a proto file that contains all the definitions for a dialect (all the builtins, all the types, all the methods on the types).
- Proto definition: https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/builtin.proto
- Executable that generates the proto definitions for Bazel: https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/docgen/ApiExporter.java
It would be very useful to align on this and use a compatible solution.
Fully featured support for float
From the spec:
The Starlark floating-point data type represents an IEEE 754 double-precision floating-point number. Its type is "float".
We don't yet support floating point numbers.
Please update dependencies to non-vulnerable versions
Currently, running cargo deny check advisories on a rust project that uses the starlark crate results in errors about RUSTSEC advisories, both of them fixed by now:
- nix-rust/nix#1541 via rustyline 0.7.1 (the current 0.9 doesn't depend on the vulnerable nix version)
- maciejhirsz/beef#37 via logos-derive via logos v0.11.4 (current version 0.12 doesn't depend on the vulnerable version of beef).
Would it be possible to bump those versions?
How to unpack list, dicts, and structs?
In the documentation at https://docs.rs/starlark/0.4.0/starlark/, there are several examples of reading simple Starlark types into Rust, then unpacking them to Rust primitives. But these are limited to unpack_str() and unpack_int(). I searched the #[test] code but didn't find anything. Please correct me if I've missed it.
What is the intended way of working in Rust with lists, dicts, structs? Do we read them into Rust Vecs, HashMaps, enums(?), or do we use starlark-rust API functions to access their parts? Naturally, these will likely be nested in practice, so can you show how to access parts of a nested object, like:
def animal(id):
return {
"kind": "giraffe",
"name": "giraffe-%s" % id,
"feeding": [
{
"name": "feeder",
"image": "photos-%s" % id,
"commands": [
"lift",
"roll-over",
],
},
],
}
Would you show some example code that illustrates how to work with the Starlark types parsed into Rust?
heap-buffer-overflow
For given input file in the zip file, the code compiled with AddressSanitizer reported heap-buffer-overflow error during runtime.
in.zip
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}Output
=================================================================
==23694==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000011b0 at pc 0x557edcf14bde bp 0x7ffccc795d70 sp 0x7ffccc795d68
READ of size 8 at 0x6020000011b0 thread T0
#0 0x557edcf14bdd in starlark::values::fast_string::skip_at_most_1byte::f::h0e1da77a9859a9a1 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:65:35
#1 0x557edcf14255 in starlark::values::fast_string::skip_at_most_1byte::h40f70214e41aab2e /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:84:14
#2 0x557edcf14f94 in starlark::values::fast_string::at::h317771503d053ab2 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:89:13
#3 0x557edd106f9e in starlark::values::types::string::_$LT$impl$u20$starlark..values..traits..StarlarkValue$u20$for$u20$alloc..boxed..Box$LT$str$GT$$GT$::at::hd2466a6c38857b75 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/types/string.rs:180:27
#4 0x557edd1498b2 in starlark::values::_$LT$impl$u20$starlark..values..layout..value..Value$GT$::at::h86d1c57ef797f103 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/mod.rs:454:9
#5 0x557edcdc9d9b in starlark::eval::expr::_$LT$impl$u20$starlark..eval..Compiler$GT$::expr::_$u7b$$u7b$closure$u7d$$u7d$::h97225333fb9eecfe /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/expr.rs:434:25
#6 0x557edcb3bb3c in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..Fn$LT$A$GT$$GT$::call::hfb7414494fc44b22 /home/xsh/code/rust-compiler/src/liballoc/boxed.rs:1095:9
#7 0x557edce16f27 in starlark::eval::stmt::_$LT$impl$u20$starlark..eval..Compiler$GT$::stmt::_$u7b$$u7b$closure$u7d$$u7d$::h823a805583856ead /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/stmt.rs:423:21
#8 0x557edcb3bb3c in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..Fn$LT$A$GT$$GT$::call::hfb7414494fc44b22 /home/xsh/code/rust-compiler/src/liballoc/boxed.rs:1095:9
#9 0x557edce437e1 in starlark::eval::_$LT$impl$u20$starlark..eval..context..Evaluator$GT$::eval_module::h1885c10571a8c91a /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/mod.rs:168:19
#10 0x557edd409ddb in starlarkfuzzvrf::main::hfa9f9430d0214a86 /home/xsh/code/rust-asan/fuzz/starlark-vrf.rs:30:21
#11 0x557edd409986 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hf7e2a6fbfb396fba /home/xsh/code/rust-compiler/src/libstd/rt.rs:67:34
#12 0x557edd966612 in std::rt::lang_start_internal::h32d9533fcd93e198 (/home/xsh/code/rust-asan/fuzz/target/shihao/x86_64-unknown-linux-gnu/debug/starlarkfuzzvrf+0x1690612)
#13 0x557edd409ac5 in std::rt::lang_start::hddcbf92bd8879287 /home/xsh/code/rust-compiler/src/libstd/rt.rs:67:5
0x6020000011b7 is located 0 bytes to the right of 7-byte region [0x6020000011b0,0x6020000011b7)
allocated by thread T0 here:
#0 0x557edc521cf0 in malloc /home/xsh/code/rust-compiler/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:145
#1 0x557edd97d721 in __rdl_alloc (/home/xsh/code/rust-asan/fuzz/target/shihao/x86_64-unknown-linux-gnu/debug/starlarkfuzzvrf+0x16a7721)
#2 0x557edd65a220 in _$LT$alloc..alloc..Global$u20$as$u20$core..alloc..AllocRef$GT$::alloc::hcad9d1152b08d2f0 /home/xsh/code/rust-compiler/src/liballoc/alloc.rs:174:49
#3 0x557edd69ad99 in alloc::raw_vec::RawVec$LT$T$C$A$GT$::allocate_in::hf0e3c52b7f9b2ee2 /home/xsh/code/rust-compiler/src/liballoc/raw_vec.rs:183:32
#4 0x557edd69a81f in alloc::raw_vec::RawVec$LT$T$C$A$GT$::with_capacity_in::h90673edd5154220e /home/xsh/code/rust-compiler/src/liballoc/raw_vec.rs:159:9
#5 0x557edcb7fd46 in _$LT$alloc..boxed..Box$LT$str$GT$$u20$as$u20$core..convert..From$LT$$RF$str$GT$$GT$::from::hab6c369a76dfc01e /home/xsh/code/rust-compiler/src/liballoc/boxed.rs:834:44
#6 0x557edd10b562 in starlark::values::types::string::_$LT$impl$u20$starlark..values..AllocFrozenValue$u20$for$u20$$RF$str$GT$::alloc_frozen_value::h0320a8d115d99a71 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/types/string.rs:298:24
#7 0x557edd12ab76 in starlark::values::_$LT$impl$u20$starlark..values..layout..heap..FrozenHeap$GT$::alloc::h0f81ce735feb37a2 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/mod.rs:143:9
#8 0x557edcdaf5ae in starlark::eval::expr::_$LT$impl$u20$starlark..syntax..ast..AstLiteral$GT$::compile::hc9b70e02f9b69980 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/expr.rs:181:45
#9 0x557edcdb700b in starlark::eval::expr::_$LT$impl$u20$starlark..eval..Compiler$GT$::expr::h85a66507d2df6b39 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/expr.rs:430:29
#10 0x557edce0ec2c in starlark::eval::stmt::_$LT$impl$u20$starlark..eval..Compiler$GT$::stmt::hf456df1940516028 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/stmt.rs:420:25
#11 0x557edce16317 in starlark::eval::stmt::_$LT$impl$u20$starlark..eval..Compiler$GT$::stmt::_$u7b$$u7b$closure$u7d$$u7d$::hd630830d91d07722 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/stmt.rs:406:52
#12 0x557edc82cf32 in core::iter::adapters::map_fold::_$u7b$$u7b$closure$u7d$$u7d$::h2dcb1f051bc7decd /home/xsh/code/rust-compiler/src/libcore/iter/adapters/mod.rs:833:28
#13 0x557edc7e15fa in core::iter::traits::iterator::Iterator::fold::hf811279acb3524f2 /home/xsh/code/rust-compiler/src/libcore/iter/traits/iterator.rs:2022:21
#14 0x557edcc08dcf in _$LT$core..iter..adapters..Map$LT$I$C$F$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::fold::h6574cb8d59d2ee82 /home/xsh/code/rust-compiler/src/libcore/iter/adapters/mod.rs:873:9
#15 0x557edc7fecf9 in core::iter::traits::iterator::Iterator::for_each::he221ccad8ce7ce7d /home/xsh/code/rust-compiler/src/libcore/iter/traits/iterator.rs:658:9
#16 0x557edcb47a1f in _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$alloc..vec..SpecExtend$LT$T$C$I$GT$$GT$::spec_extend::h6d9bbc281d6dd098 /home/xsh/code/rust-compiler/src/liballoc/vec.rs:2140:17
#17 0x557edcb5bbc7 in _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$alloc..vec..SpecExtend$LT$T$C$I$GT$$GT$::from_iter::h2aa07f3389d4a90c /home/xsh/code/rust-compiler/src/liballoc/vec.rs:2120:9
#18 0x557edcbf2db1 in _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..iter..traits..collect..FromIterator$LT$T$GT$$GT$::from_iter::h17f17a4a8c594510 /home/xsh/code/rust-compiler/src/liballoc/vec.rs:1995:9
#19 0x557edc7eaff7 in core::iter::traits::iterator::Iterator::collect::hc9377029bb0e0da2 /home/xsh/code/rust-compiler/src/libcore/iter/traits/iterator.rs:1671:9
#20 0x557edca4bcf4 in _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$gazebo..ext..vec..VecExt$GT$::into_map::h241f89a4238fcadd /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/gazebo-0.2.2/src/ext/vec.rs:170:9
#21 0x557edce0e7b7 in starlark::eval::stmt::_$LT$impl$u20$starlark..eval..Compiler$GT$::stmt::hf456df1940516028 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/stmt.rs:406:33
#22 0x557edce42f67 in starlark::eval::_$LT$impl$u20$starlark..eval..context..Evaluator$GT$::eval_module::h1885c10571a8c91a
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:65:35 in starlark::values::fast_string::skip_at_most_1byte::f::h0e1da77a9859a9a1
Shadow bytes around the buggy address:
0x0c047fff81e0: fa fa 07 fa fa fa 07 fa fa fa 06 fa fa fa 03 fa
0x0c047fff81f0: fa fa 02 fa fa fa 02 fa fa fa 06 fa fa fa 03 fa
0x0c047fff8200: fa fa 05 fa fa fa 02 fa fa fa 02 fa fa fa 05 fa
0x0c047fff8210: fa fa 05 fa fa fa 04 fa fa fa 02 fa fa fa 02 fa
0x0c047fff8220: fa fa 04 fa fa fa 03 fa fa fa 04 fa fa fa 03 fa
=>0x0c047fff8230: fa fa 04 fa fa fa[07]fa fa fa 00 fa fa fa 00 fa
0x0c047fff8240: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff8250: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff8260: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff8270: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff8280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==23694==ABORTING
access invalid pointer 0x000000000001
For given input file in the zip file, the code compiled with AddressSanitizer reported invalid address access error during runtime.
in.zip
Code
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}Output
AddressSanitizer:DEADLYSIGNAL
=================================================================
==21690==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x55ac52e1190a bp 0x7ffffab48110 sp 0x7ffffab47e80 T0)
==21690==The signal is caused by a READ memory access.
==21690==Hint: address points to the zero page.
#0 0x55ac52e1190a in starlark::values::fast_string::skip_at_most_1byte::f::h0e1da77a9859a9a1 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:55:34
#1 0x55ac52e11255 in starlark::values::fast_string::skip_at_most_1byte::h40f70214e41aab2e /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:84:14
#2 0x55ac52e11f94 in starlark::values::fast_string::at::h317771503d053ab2 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/fast_string.rs:89:13
#3 0x55ac53003f9e in starlark::values::types::string::_$LT$impl$u20$starlark..values..traits..StarlarkValue$u20$for$u20$alloc..boxed..Box$LT$str$GT$$GT$::at::hd2466a6c38857b75 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/types/string.rs:180:27
#4 0x55ac530468b2 in starlark::values::_$LT$impl$u20$starlark..values..layout..value..Value$GT$::at::h86d1c57ef797f103 /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/values/mod.rs:454:9
#5 0x55ac52cc6d9b in starlark::eval::expr::_$LT$impl$u20$starlark..eval..Compiler$GT$::expr::_$u7b$$u7b$closure$u7d$$u7d$::h97225333fb9eecfe /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/expr.rs:434:25
#6 0x55ac52a38b3c in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..Fn$LT$A$GT$$GT$::call::hfb7414494fc44b22 /home/xsh/code/rust-compiler/src/liballoc/boxed.rs:1095:9
#7 0x55ac52d13f27 in starlark::eval::stmt::_$LT$impl$u20$starlark..eval..Compiler$GT$::stmt::_$u7b$$u7b$closure$u7d$$u7d$::h823a805583856ead /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/stmt.rs:423:21
#8 0x55ac52a38b3c in _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..Fn$LT$A$GT$$GT$::call::hfb7414494fc44b22 /home/xsh/code/rust-compiler/src/liballoc/boxed.rs:1095:9
#9 0x55ac52d407e1 in starlark::eval::_$LT$impl$u20$starlark..eval..context..Evaluator$GT$::eval_module::h1885c10571a8c91a /home/xsh/.cargo/registry/src/github.com-1ecc6299db9ec823/starlark-0.4.0/src/eval/mod.rs:168:19
#10 0x55ac53306ddb in starlarkfuzzvrf::main::hfa9f9430d0214a86 /home/xsh/code/rust-asan/fuzz/starlark-vrf.rs:30:21
#11 0x55ac53306986 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hf7e2a6fbfb396fba /home/xsh/code/rust-compiler/src/libstd/rt.rs:67:34
#12 0x55ac53863612 in std::rt::lang_start_internal::h32d9533fcd93e198 (/home/xsh/code/rust-asan/fuzz/target/shihao/x86_64-unknown-linux-gnu/debug/starlarkfuzzvrf+0x1690612)
#13 0x55ac53306ac5 in std::rt::lang_start::hddcbf92bd8879287 /home/xsh/code/rust-compiler/src/libstd/rt.rs:67:5
#14 0x55ac53306af9 in main (/home/xsh/code/rust-asan/fuzz/target/shihao/x86_64-unknown-linux-gnu/debug/starlarkfuzzvrf+0x1133af9)
#15 0x7fd6c03e9bf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#16 0x55ac523a06f9 in _start (/home/xsh/code/rust-asan/fuzz/target/shihao/x86_64-unknown-linux-gnu/debug/starlarkfuzzvrf+0x1cd6f9)
str.find does not work correctly with non-ASCII characters
str.find and some other string operations do not work correctly with non-ASCII characters.
For example:
Integrate with google/oss-fuzz for continuous fuzz testing?
Hey I'd like to suggest adding starlark-rust to google/oss-fuzz. If you aren't familiar with fuzz testing, here is a bit of a run down (from Wikipedia);
In programming and software development, fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks.
Google offers a free continuous fuzzing service called OSS-fuzz. If starlark-rust is integrated into oss-fuzz, the (yet to be written) fuzz tests under starlark-rust will be built and then run once a day, to search for bugs and vulnerabilities in starlark-rust. This service can be integrated with the CI for starlark-rust, so that the fuzz tests are run for 10min or so for every pull request, preventing buggy code from being merged.
Here is an example of another FB project zstd that has been integrated into oss-fuzz https://github.com/google/oss-fuzz/blob/master/projects/zstd/project.yaml.
I've opened up a pull request to add a basic fuzz-testing harness here #64. If you are keen on adding starlark-rust to oss-fuzz I'd be happy to champion the integration :)
Other relevant issues:
Make a release
As promised in #28
Potential alignment UB for 0-size slices in `slice_cast`
In slice_cast, if the slice has 0 size then it directly casts the pointer type and constructs a new zero-len slice:
https://github.com/facebookexperimental/starlark-rust/blob/3bb20164a18f1e0fe65ba9cfc4c5bbfc7f41f621/starlark/src/values/layout/arena.rs#L76-L78
slice::from_raw_parts requires the pointer to be non-null and properly aligned, so if U does not have the same alignment as T then this is UB:
datamust be non-null and aligned even for zero-length slices
I think the alignment assertion below should apply to the whole function, not just the zero-length case.
This function is also a bit odd in that it loses the length of a slice of zero-sized objects. For example, a non-zero length slice of [()] has a zero mem::size_of_val, so this would turn it into a zero-length [U], rather than retaining the length (assuming U also has a zero size). I'm not sure if this is the intention; I guess it's OK if you're never expecting T to be zero-sized.
"Strict" equality testing of Values from the API?
Hi! Thanks for picking up the maintenance of Starlark's Rust implementation. I'm excited to see where this tech goes!
Is it possible to do "strict" equality testing of Values from the starlark-rust API, or would it be possible to expose that capability? I'm referring to something akin to JavaScript's === operator, or the default behavior of == in Python. I'd like to test primitive values (bool, int, string...) by value, but reference values (tuple, list, dict, record...) by reference, i.e., checking their identity only. Currently Value::equals tests the latter by value (except for functions), for example, walking over all elements in a list.
My use case is a system I'm fiddling with which will contain a large number of small, interconnected Starlark programs, where the outputs from one will be fed in as inputs to another. This graph of programs will be executed multiple times with changing sets of initial inputs. I'd like to take advantage of Starlark's determinism to memoize the execution of individual programs in the graph by their inputs, returning cached output values if the inputs given are unchanged from the previous execution. This requires equality-testing the inputs against previous values, which could become unintentionally expensive if it involves recursively descending into lists, dicts, etc.
Should there be an `impl<T: AllocValue> AllocValue for Option<T>`?
I have this helper function that seems like it should maybe be a trait impl:
fn opt_typed_val<'h>(heap: &'h Heap, v: Option<impl AllocValue<'h>>) -> Value<'h> {
if let Some(v) = v {
heap.alloc(v)
} else {
Value::new_none()
}
}Are there any risks to supporting this kind of automatic conversion? I haven't been able to think of situations where you'd want a different encoding of Option<T> but I could easily be missing something.
[Question] Creating a function using the `Dict` type as an argument.
Sorry if this is the wrong place but I'm having issues using the rust Dict type.
I'm creating a template function to expand the starlark stdlibrary.
The function will take a JINJA2 template and using Tera format it with arbitrary variables.
I'm currently using a serede_json object to pass the template vars but I would like to switch to the Starlark Dict type so the values can be more easily worked with.
file.rs
mod template_impl;
use starlark::environment::{Methods, MethodsBuilder, MethodsStatic};
use starlark::values::{StarlarkValue, Value, UnpackValue, ValueLike, dict::Dict};
use starlark::values::none::NoneType;
use starlark::{starlark_type, starlark_simple_value, starlark_module};
...
#[starlark_module]
fn methods(builder: &mut MethodsBuilder) {
...
fn template(_this: FileLibrary, template_path: String, dst_path: String, args: Dict, autoescape: bool) -> NoneType {
template_impl::template(template_path, dst_path, args, autoescape)?;
Ok(NoneType{})
}
fn write(_this: FileLibrary, path: String, content: String) -> NoneType {
write_impl::write(path, content)?;
Ok(NoneType{})
}
}This gives the following error.
I came across these two issues:
- #14 - My understanding is this is a good place to look if I wanted to create my own types but I'd like to use the Dict type.
- #22 - I see how I could unpack & create a Dict from a rust SmallMap but i'm still not sure how to define a rust function to take a Starlark Dict type.
What rust type should I specify in my function definition if I want to create a function that accepts a Starlark Dict?
Is VSCode extension functional?
I've compiled and installed it, but it does not seem to do anything... Is this expected?
Implement 32-bit Support
I have an use case (for a personal project) to deploy a starlark-parsing binary to a Raspberry Pi 3, which is a 32-bit ARM platform (Rust target armv7-unknown-linux-gnueabihf).
However, when I try and compile for armv7 (or any 32 bit platform, for example i586-unknown-linux-gnu) I get compile time errors in a few places:
https://github.com/facebookexperimental/starlark-rust/blob/main/starlark/src/values/layout/pointer.rs#L73-L78
https://github.com/facebookexperimental/starlark-rust/blob/main/starlark/src/syntax/ast.rs#L81-L83
https://github.com/facebookexperimental/starlark-rust/blob/main/starlark/src/values/layout/pointer_i32.rs#L38 (this constant value is simply too large)
That seems to be all of them at compile time, but due to an unrelated problem I don't have a cross-linker on the device I'm writing this from. I assume due to the first link above that there are runtime problems. So I'm wondering, are there any plans to support 32-bit? (I'm aware I'm asking to roll back the clock and that no one should use 32-bit any more, but it is what it is)
panicked at 'assertion failed: self.args.is_none() && !self.no_args && self.kwargs.is_none()'
For given input file:
in.zip
The following code:
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}
has output:
thread 'main' panicked at 'assertion failed: self.args.is_none() && !self.no_args && self.kwargs.is_none()', /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/runtime/arguments.rs:195:9
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::panicking::panic
3: starlark::eval::runtime::arguments::ParametersSpec<V>::args
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/runtime/arguments.rs:195:9
4: starlark::eval::fragment::def::DefCompiled::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/def.rs:215:54
5: starlark::eval::fragment::def::DefCompiled::as_compiled::ann_expr_def::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:29:33
6: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
7: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:95:54
8: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::ann_stmt_return::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:88:33
9: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
10: starlark::eval::fragment::stmt::StmtsCompiled::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:310:25
11: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
12: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:81:17
13: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:74:21
14: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:92:9
15: starlark::eval::<impl starlark::eval::runtime::evaluator::Evaluator>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/mod.rs:148:19
String.elems returns a list
Many of our string functions are wrong. Some are quite wrong, mixing int vs string. One particular example is that elems returns a string, rather than an opaque iterator. Something using an iterator would be much more memory efficient.
Starlark update throws errors related to the Allocative trait.
I'm working on updating to the latest version of starlark (acf6384) on the main branch so I can leverage the PrintHandle attribute and set a custom handler.
I was having some issues around allocative so I tried just building one of the example code sections but ran into the same compiler error.
https://github.com/facebookexperimental/starlark-rust/blob/acf638430a00ca3855862e8c669670e1adaa42aa/starlark/src/lib.rs#L287
I made some minor modifications like moving the struct definition and impl's out of the run function.
#![feature(trivial_bounds)]
use starlark::environment::{Globals, Module};
use starlark::eval::Evaluator;
use starlark::syntax::{AstModule, Dialect};
use starlark::values::{Heap, StarlarkValue, Value, ValueError, ValueLike, ProvidesStaticType, NoSerialize};
use starlark::{starlark_type, starlark_simple_value};
use std::fmt::{self, Display, Write};
use allocative::Allocative;
// Define complex numbers
#[derive(Debug, PartialEq, Eq, ProvidesStaticType, Allocative, NoSerialize)]
struct Complex {
real: i32,
imaginary: i32,
}
starlark_simple_value!(Complex);
impl Display for Complex {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{} + {}i", self.real, self.imaginary)
}
}
impl<'v> StarlarkValue<'v> for Complex {
starlark_type!("complex");
// How we add them
fn add(&self, rhs: Value<'v>, heap: &'v Heap)
-> Option<anyhow::Result<Value<'v>>> {
if let Some(rhs) = rhs.downcast_ref::<Self>() {
Some(Ok(heap.alloc(Complex {
real: self.real + rhs.real,
imaginary: self.imaginary + rhs.imaginary,
})))
} else {
None
}
}
}
fn run() -> anyhow::Result<()> {
let content = "str(a + b)";
let ast = AstModule::parse("complex.star", content.to_owned(), &Dialect::Standard)?;
let globals = Globals::standard();
let module = Module::new();
// We inject some complex numbers into the module before we start.
let a = module.heap().alloc(Complex {real: 1, imaginary: 8});
module.set("a", a);
let b = module.heap().alloc(Complex {real: 4, imaginary: 2});
module.set("b", b);
let mut eval = Evaluator::new(&module);
let res = eval.eval_module(ast, &globals)?;
assert_eq!(res.unpack_str(), Some("5 + 10i"));
Ok(())
}
fn main(){ run().unwrap(); }When building the project I get an error that the Allocative trait is not implemented for the Complex struct.
Downloads/starlark-test/starlark-test-proj ๎ฐ ๐ฆ 1.70.0-nightly ๎ฐ ๎ master โ โญ ๎ฐ
$ cargo run
Compiling rustix v0.36.11
Compiling terminal_size v0.2.5
Compiling clap_builder v4.2.1
Compiling clap v4.2.1
Compiling starlark v0.9.0-pre (https://github.com/facebookexperimental/starlark-rust?rev=acf638430a00ca3855862e8c669670e1adaa42aa#acf63843)
Compiling starlark-test-proj v0.1.0 (/Users/hulto/Downloads/starlark-test/starlark-test-proj)
warning: unused import: `ValueError`
--> src/main.rs:6:52
|
6 | use starlark::values::{Heap, StarlarkValue, Value, ValueError, ValueLike, ProvidesStaticType, NoSerialize};
| ^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
error[E0277]: the trait bound `Complex: allocative::allocative_trait::Allocative` is not satisfied
--> src/main.rs:25:32
|
25 | impl<'v> StarlarkValue<'v> for Complex {
| ^^^^^^^ the trait `allocative::allocative_trait::Allocative` is not implemented for `Complex`
|
= help: the following other types implement trait `allocative::allocative_trait::Allocative`:
!
&'static T
()
(A, B)
(A, B, C)
(A, B, C, D)
(A, B, C, D, E)
(A,)
and 118 others
note: required by a bound in `StarlarkValue`
--> /Users/hulto/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/acf6384/starlark/src/values/traits.rs:206:31
|
206 | 'v + ProvidesStaticType + Allocative + Debug + Display + Serialize + Sized
| ^^^^^^^^^^ required by this bound in `StarlarkValue`
For more information about this error, try `rustc --explain E0277`.
warning: `starlark-test-proj` (bin "starlark-test-proj") generated 1 warning
error: could not compile `starlark-test-proj` (bin "starlark-test-proj") due to previous error; 1 warning emitted
I tried implementing the Allocative trait manually instead of through a derive but got the same error.
Doing both, defining the Allocative trait manuallly and through derive throws an error that Allocative is defined twice.
cargo version
cargo 1.70.0-nightly (145219a9f 2023-03-27)
Cargo.toml
[dependencies]
starlark = { git = "https://github.com/facebookexperimental/starlark-rust", rev = "acf638430a00ca3855862e8c669670e1adaa42aa" }
allocative = { version = "0.2" }
allocative_derive = { version = "0.2" }
anyhow = "1.0.65"
Accessing ParametersSpec from a DefGen in a native function
I'm trying to access the static components of a ParametersSpec from a native method exposed to the Starlark evaluator when the input into the method is a method. Example:
def function_name(a):
return "Hello, higher order"
native_method(func=function_name)The Value received is a struct of type DefGen which is private in eval:
DefGen {
parameters: ParametersSpec {
function_name: "wip.function_name",
names: [("a", Required)],
indices: SmallMap { state: Vec(VecMap { hashes: [SmallHashResult(2767251955), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0), SmallHashResult(0)],
values: [("a", 0)] }) },
positional: 1,
no_args: false,
args: None,
kwargs: None
} ..rest omitted
}
Is there an approach to get access to the ParametersSpec for the passed in function that I am missing or another potential way to read it? If not, is that something that might be worth adding?
panicked at 'slice index starts at 2 but ends at 1'
For given input file:
in.zip
The following code:
use starlark::eval::Evaluator;
use starlark::environment::{Module, Globals};
use starlark::values::Value;
use starlark::syntax::{AstModule, Dialect};
fn main() {
// change filepath to the input file
let data = std::fs::read(filepath).unwrap();
let _ = match std::str::from_utf8(&data) {
Ok(d) => {
if let Ok(ast) = AstModule::parse("hello_world.star", d.to_owned(), &Dialect::Standard) {
let globals: Globals = Globals::standard();
let module: Module = Module::new();
let mut eval: Evaluator = Evaluator::new(&module, &globals);
eval.eval_module(ast);
}
},
Err(..) => return,
};
}
has output:
thread 'main' panicked at 'slice index starts at 2 but ends at 1', /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/values/index.rs:129:19
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::slice::index::slice_index_order_fail
3: <core::ops::range::Range<usize> as core::slice::index::SliceIndex<[T]>>::index
at /home/sxia/code/rust-compiler/library/core/src/slice/index.rs:238:13
4: core::slice::index::<impl core::ops::index::Index<I> for [T]>::index
at /home/sxia/code/rust-compiler/library/core/src/slice/index.rs:15:9
5: starlark::values::index::apply_slice
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/values/index.rs:129:19
6: <starlark::values::types::string::StarlarkStr as starlark::values::traits::StarlarkValue>::slice
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/values/types/string.rs:349:22
7: <starlark::values::layout::avalue::Wrapper<Mode,T> as starlark::values::traits::StarlarkValueDyn>::slice
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/values/layout/avalue.rs:598:9
8: starlark::values::<impl starlark::values::layout::value::Value>::slice
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/values/mod.rs:497:9
9: starlark::eval::fragment::expr::eval_slice::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/expr.rs:554:13
10: starlark::eval::fragment::expr::eval_slice::ann_expr_slice::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:44:33
11: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
12: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:95:54
13: starlark::eval::fragment::stmt::<impl starlark::codemap::Spanned<starlark::eval::fragment::stmt::StmtCompiledValue>>::as_compiled::ann_stmt_return::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/mod.rs:88:33
14: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
15: starlark::eval::fragment::stmt::StmtsCompiled::as_compiled::{{closure}}
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/stmt.rs:310:25
16: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
at /home/sxia/code/rust-compiler/library/alloc/src/boxed.rs:1652:9
17: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:81:17
18: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_top_level_stmt
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:74:21
19: starlark::eval::fragment::module::<impl starlark::eval::compiler::Compiler>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/fragment/module.rs:92:9
20: starlark::eval::<impl starlark::eval::runtime::evaluator::Evaluator>::eval_module
at /home/sxia/.cargo/git/checkouts/starlark-rust-59575ffdf833204c/458a203/starlark/src/eval/mod.rs:148:19
...
starlark-rust allows string.startswith to take a list as value of its prefix parameter, contrary to language spec
starlark-rust allows the following: 'a'.startswith(['a'])
By contrast, java and go starlark implementations allow only a string or a tuple of strings as the prefix parameter, in accordance with the spec: https://github.com/bazelbuild/starlark/blob/master/spec.md#string%C2%B7startswith
(Arguably, starlark-rust's behavior is better here - so maybe we ought to change the spec...)
add rdiv and friends (only radd is already implemented)
like defined in https://docs.python.org/3.11/reference/datamodel.html#emulating-numeric-types
this is helpful for emulating/ creating apis like the python ones of z3 or sympy, using starlark as the scripting engine instead of python, writing the core logic in rust
Documentation should include examples implementing TypedValue
It would be awesome if there were a canonical example so I could know if I'm doing it correctly! The crate root's docs cover calling Rust functions, and I think it would be nice to have an extended version of that example which returns a Rust-defined type.
test cases should not depend on @bazel_tools//tools/build_defs/pkg:pkg.bzl
pkg_tar from @bazel_tools//tools/build_defs/pkg might go away at Bazel 5.x.
It is referenced here and in a few other places
https://github.com/facebookexperimental/starlark-rust/blob/b5df9fe72616a5879e760d563b19a6103ce57168/starlark/testcases/parse/jenkins_base.star#L23
AFAICT, this is harmless, since you do not evaluate using bazel semantics, you are just checking that you can parse things. This is more of a heads up in case you think that will be an issue one day.
feat: support compiling to a Web Assembly (WASM) module
Feature request
Support compiling to web assembly so that it can be used embedded in a web page.
This feature is already supported by the Golang implementation of Starlark:
https://github.com/google/starlark-go
https://haribala.dev/starlark-webasm-demo
Environment
MacOS Monterey 12.5.1 (21G83) (Intel CPU)
$ rustup toolchain list
stable-x86_64-apple-darwin
nightly-x86_64-apple-darwin (default) (override)
Actual behaviour
Currently there are lots of errors, mostly due to dependencies on stuff like use std::os::unix::io::AsRawFd;
$ cargo build --target wasm32-unknown-unknown
Compiling memchr v2.5.0
Compiling serde v1.0.145
Compiling cfg-if v1.0.0
Compiling unicode-width v0.1.10
Compiling tinyvec_macros v0.1.0
Compiling serde_json v1.0.85
Compiling ryu v1.0.11
Compiling itoa v1.0.3
Compiling bitflags v1.3.2
Compiling crossbeam-utils v0.8.12
Compiling num-traits v0.2.15
Compiling ahash v0.7.6
Compiling log v0.4.17
Compiling percent-encoding v2.2.0
Compiling unicode-bidi v0.3.8
Compiling smallvec v1.9.0
Compiling tinyvec v1.6.0
Compiling once_cell v1.15.0
Compiling textwrap v0.11.0
Compiling form_urlencoded v1.1.0
Compiling nibble_vec v0.1.0
Compiling num-integer v0.1.45
Compiling unicode-linebreak v0.1.4
Compiling aho-corasick v0.7.19
Compiling libc v0.2.134
Compiling regex-syntax v0.6.27
Compiling atty v0.2.14
Compiling endian-type v0.1.2
Compiling vec_map v0.8.2
Compiling ansi_term v0.12.1
Compiling bit-vec v0.6.3
Compiling strsim v0.8.0
Compiling dirs-sys-next v0.1.2
Compiling crossbeam-channel v0.5.6
Compiling dirs-next v2.0.0
Compiling radix_trie v0.2.1
Compiling bit-set v0.5.3
Compiling clap v2.34.0
Compiling thiserror v1.0.37
Compiling hashbrown v0.12.3
Compiling anyhow v1.0.65
Compiling inventory v0.1.11
Compiling erased-serde v0.3.23
Compiling memoffset v0.6.5
Compiling num-bigint v0.4.3
Compiling os_str_bytes v6.3.0
Compiling fd-lock v3.0.6
Compiling gazebo v0.8.0
error[E0255]: the name `unsupported` is defined multiple times
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/mod.rs:15:17
|
14 | mod unsupported;
| ---------------- previous definition of the module `unsupported` here
15 | pub use unsupported;
| ^^^^^^^^^^^ `unsupported` reimported here
|
= note: `unsupported` must be defined only once in the type namespace of this module
help: you can use `as` to change the binding name of the import
|
15 | pub use unsupported as other_unsupported;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/read_guard.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/rw_lock.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/write_guard.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0365]: `unsupported` is private, and cannot be re-exported
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/mod.rs:15:17
|
15 | pub use unsupported;
| ^^^^^^^^^^^ re-export of private `unsupported`
|
= note: consider declaring type or module `unsupported` with `pub`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:20:43
|
20 | pub struct RwLockReadGuard<'lock, T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockReadGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:21:17
|
21 | guard: sys::RwLockReadGuard<'lock, T>,
| ^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockReadGuard;
|
1 | use std::sync::RwLockReadGuard;
|
help: if you import `RwLockReadGuard`, refer to it directly
|
21 - guard: sys::RwLockReadGuard<'lock, T>,
21 + guard: RwLockReadGuard<'lock, T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:24:21
|
24 | impl<'lock, T: sys::AsRaw> RwLockReadGuard<'lock, T> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockReadGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:25:35
|
25 | pub(crate) fn new(guard: sys::RwLockReadGuard<'lock, T>) -> Self {
| ^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockReadGuard;
|
1 | use std::sync::RwLockReadGuard;
|
help: if you import `RwLockReadGuard`, refer to it directly
|
25 - pub(crate) fn new(guard: sys::RwLockReadGuard<'lock, T>) -> Self {
25 + pub(crate) fn new(guard: RwLockReadGuard<'lock, T>) -> Self {
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:30:14
|
30 | impl<T: sys::AsRaw> ops::Deref for RwLockReadGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:40:14
|
40 | impl<T: sys::AsRaw> Drop for RwLockReadGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:13:27
|
13 | pub struct RwLock<T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLock` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:14:16
|
14 | lock: sys::RwLock<T>,
| ^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLock;
|
1 | use std::sync::RwLock;
|
help: if you import `RwLock`, refer to it directly
|
14 - lock: sys::RwLock<T>,
14 + lock: RwLock<T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:17:14
|
17 | impl<T: sys::AsRaw> RwLock<T> {
| ^^^^^ not found in `sys`
error[E0433]: failed to resolve: could not find `RwLock` in `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:34:24
|
34 | lock: sys::RwLock::new(inner),
| ^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLock;
|
1 | use std::sync::RwLock;
|
help: if you import `RwLock`, refer to it directly
|
34 - lock: sys::RwLock::new(inner),
34 + lock: RwLock::new(inner),
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:20:44
|
20 | pub struct RwLockWriteGuard<'lock, T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockWriteGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:21:17
|
21 | guard: sys::RwLockWriteGuard<'lock, T>,
| ^^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockWriteGuard;
|
1 | use std::sync::RwLockWriteGuard;
|
help: if you import `RwLockWriteGuard`, refer to it directly
|
21 - guard: sys::RwLockWriteGuard<'lock, T>,
21 + guard: RwLockWriteGuard<'lock, T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:24:21
|
24 | impl<'lock, T: sys::AsRaw> RwLockWriteGuard<'lock, T> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockWriteGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:25:35
|
25 | pub(crate) fn new(guard: sys::RwLockWriteGuard<'lock, T>) -> Self {
| ^^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockWriteGuard;
|
1 | use std::sync::RwLockWriteGuard;
|
help: if you import `RwLockWriteGuard`, refer to it directly
|
25 - pub(crate) fn new(guard: sys::RwLockWriteGuard<'lock, T>) -> Self {
25 + pub(crate) fn new(guard: RwLockWriteGuard<'lock, T>) -> Self {
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:30:14
|
30 | impl<T: sys::AsRaw> ops::Deref for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:39:14
|
39 | impl<T: sys::AsRaw> ops::DerefMut for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:47:14
|
47 | impl<T: sys::AsRaw> Drop for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
Compiling unicode-normalization v0.1.22
Compiling fnv v1.0.7
Some errors have detailed explanations: E0255, E0365, E0405, E0412, E0433.
For more information about an error, try `rustc --explain E0255`.
error: could not compile `fd-lock` due to 22 previous errors
warning: build failed, waiting for other jobs to finish...
$ echo $?
101
Clarification: "glob" / "native functions"
While playing around with "enhanced JSON"-like use-cases (as the documentation would call it), i stumbled upon the following interpreter-error:
Err(error: Variable `glob` not found
--> errors.star:3:10
|
3 | srcs = glob("*.md"),
| ^^^^
|
)
with something like:
pandoc_templating_indentation(
name = "bla",
srcs = glob("*.md"),
template = "blub.markdown",
indentation = 1
)
and:
#[starlark_module]
fn starlark_pandoc_templating_indentation(builder: &mut GlobalsBuilder) {
fn pandoc_templating_indentation (name: &str, srcs: Vec<&str>, template: &str, indentation: i32) -> anyhow::Result<NoneType> {
info!(" -> registered: pandoc_templating_indentation");
...
Now i wonder: Is glob not part of this interpreter? I guess, in bazel terminology it would be a native function.
On one hand,
- the testcases contain
globeverywhere - the intro blog-post contains a
glob
while on the other,
- it's not part of the specs
So:
- Is
globthere and i did miss something? - Is
globnot there and it's out of scope? - Is
globnot there and it will eventually be provided?
Thanks for reading.
- Sascha
How to return values from Starlark?
I'm creating my own values and functions and extracting them from Starklark for use in Rust. But what's the right way to return my objects from the Starlark modules? Eg, how should they be returned from something like:
pub fn read_file(fname: &str) -> anyhow::Result<Animal> {...}
The problem that I encountered was that my extracted values were local, so Rust complained about returning local variables. I found a simple solution: don't make the module local: Pass it in and let it contain the resulting value:
pub fn read_file<'a>(module: &'a Module, fname: &str) -> anyhow::Result<&'a Animal> {...}
However, now I'm using the load() Starlark command, following the example here: https://docs.rs/starlark/0.4.0/starlark/#enable-the-load-statement. But in that example, the get_module()` function creates a local module for each load. I'm able to load my object like:
let ab = get_module(fname)?;
let vv = ab.get("puppy").expect("error getting variable");
let val = Animal::from_value(vv.value()).ok_or(anyhow!("error extracting value"));
But trying to return Ok(val) of course results in a "cannot return value referencing local variable". I suppose I could modify get_module() to accept a module to contain the values, as above, or Box it, which would require my object to implement copy, but I wonder if there's a simpler or intended way for starlark-rust?
[Question] Using `Dict` as a function return type with string values.
This is dumb question but I'm having a hard time using the Dict or SmallMap<Value, Value> types as a return type for rust functions. Specifically when trying to return a Value holding a string.
- I see in this issue the suggestion to use the evaluator heap. Is it recommended to pass a reference to the eval heap in each functions and perform allocations there?
- Is there a better way in general to return a Map / dictionary from a starlark function?
Here's a test function i'm trying to add to the starlark library given a string, it should return a list of dictionary objects using Value<String> as the keys and values.
I'm able to use the frozen string value, and int but having issues with non-constant strings.
Thank you
lalrpop compile error with Rust 1.56+
I know version 0.3.1 of this crate is old. But I wanted to give a heads up that it stops building on Rust 1.56+ due to a build error in the lalrpop crate:
Compiling lalrpop v0.16.3
error[E0034]: multiple applicable items in scope
--> /home/gps/.cargo/registry/src/github.com-1ecc6299db9ec823/lalrpop-0.16.3/src/message/horiz.rs:25:14
|
25 | .intersperse(self.separate)
| ^^^^^^^^^^^ multiple `intersperse` found
|
= note: candidate #1 is defined in an impl of the trait `Iterator` for the type `std::iter::Map<I, F>`
= note: candidate #2 is defined in an impl of the trait `Itertools` for the type `T`
help: disambiguate the associated function for candidate #1
|
22 ~ Iterator::intersperse(self.items
23 + .iter()
24 + .map(|c| c.min_width()), self.separate)
|
help: disambiguate the associated function for candidate #2
|
22 ~ Itertools::intersperse(self.items
23 + .iter()
24 + .map(|c| c.min_width()), self.separate)
|
For more information about this error, try `rustc --explain E0034`.
error: could not compile `lalrpop` due to previous error
I'd love to port my code to version 0.5 of starlark-rust. But I haven't put in the time yet (and I'd really like #7 before I do so). The apparent imminent inability to build starlark-rust 0.3.1 on Rust stable is probably going to cause a number of bug reports for me :/
Would you be willing to publish a 0.3.2 release of this crate that uses newer versions of its dependencies so it continues to build on Rust stable / 1.56+?
(I wish I could commit to porting past the ancient 0.3 release, but I still haven't found the time to port the significant amount of code I have depending on the 0.3 semantics. But this lalrpop issue may force my hand...)
write/serialize starlark
Is it possible to write or serialize starlark that's been parsed in using this module to a file?
document the build instructions
It would be great if the project README linked to instructions---or simply demonstrated using an example---how to build it, ideally assuming only a minimum of Rust knowledge.
I cloned the repo, installed cargo using brew install rust, and attempted to build the REPL using variations of these commands:
$ cargo build --bin=starlark
$ cargo build
$ cargo build --release --bin=starlark
$ (cd starlark/bin/ && cargo build)but in each case got this error:
error[E0554]: `#![feature]`
--> /Users/adonovan/.cargo/registry/src/github.com-1ecc6299db9ec823/gazebo-0.6.0/src/lib.rs:10:49
|
10 | #![cfg_attr(feature = "str_pattern_extensions", feature(pattern))]
I'm sure I'm missing something elementary. Following clues from StackOverflow, I tried removing the homebrew rust and installing rustup, eventually fumbling my way to success:
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ ~/.cargo/bin/rustup install nightly
$ ~/.cargo/bin/cargo +nightly install starlark
$ ~/.cargo/bin/starlark --repl
$> print(1+2)
3Perhaps someone could turn my drunken walk into a signpost. Thanks!
No support for bytes
From the spec:
A bytes is an immutable sequence of values in the range 0-255. The type of a bytes is "bytes".
We don't yet support bytes.
Dependency on starlark fails to resolve
I added a dependency on starlark-rust:
starlark = { git = "https://github.com/facebookexperimental/starlark-rust.git", optional = true }
but this fails to resolve with:
[jeremy@aerolith reindeer]$ cargo +nightly check
Updating git repository `https://github.com/facebookexperimental/starlark-rust.git`
Updating crates.io index
Updating git repository `https://github.com/facebookincubator/gazebo`
error: failed to select a version for `syn`.
... required by package `starlark_module v0.4.0-pre (https://github.com/facebookexperimental/starlark-rust.git#bf2f90dd)`
... which is depended on by `starlark v0.4.0-pre (https://github.com/facebookexperimental/starlark-rust.git#bf2f90dd)`
... which is depended on by `reindeer v0.0.0 (/home/jeremy/git/reindeer)`
versions that meet the requirements `^1.0.57` are: 1.0.60, 1.0.59, 1.0.58, 1.0.57
all possible versions conflict with previously selected packages.
previously selected package `syn v1.0.33`
... which is depended on by `auto_impl v0.4.1`
... which is depended on by `tower-lsp v0.13.3`
... which is depended on by `starlark v0.4.0-pre (https://github.com/facebookexperimental/starlark-rust.git#bf2f90dd)`
... which is depended on by `reindeer v0.0.0 (/home/jeremy/git/reindeer)`
failed to select a version for `syn` which could resolve this conflict
which seems to be a version conflict on syn between tokio-lsp and starlark_module. I don't see this when I build in the starlark-rust repo itself, so I'm not sure why I'm seeing it here - perhaps there's another version of syn involved via one of my other dependencies.
repr("\x01") does not conform to spec
Stack overflow bug
I found a bug in the starlark parser using the fuzzing harness in #64. I'm guessing there is some recursive parsing of unary operators and as a result, you can get a stackoverflow if you chain a bunch of unary operators. e.g. attempting to parse;
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++''
Will result in a stackoverflow. It looks like a fairly unlikely scenario, and likely won't impact usage much if at all. But I thought I'd mention it anyway :)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.