expr-lang / expr Goto Github PK
View Code? Open in Web Editor NEWExpression language and expression evaluation for Go
Home Page: https://expr-lang.org
License: MIT License
Expression language and expression evaluation for Go
Home Page: https://expr-lang.org
License: MIT License
Assuming I have a document map[string]interface{}{ "Age" : ..., "Income": ..., "other" : .... }
, and a generic expression.
I want to be able to know all the names
that are in the expression.
> expr.GetNames("Age > 25 AND Income > 25,000" )
[ "Age", "Income"]
https://github.com/antonmedv/expr/blob/a97d10e66a207bffb67f503fe58297fb14f3ee1c/vm/vm.go#L248
There are other similar heap allocations. This could easily be replaced with a pre-allocated or stack allocated fixed array of some typical size, and if the callsite is bigger (if you happen to have some crazy number of arguments) then you could fall back to allocating. Otherwise you are heap allocating on every call
First, congrats for this really nice project :)
For my use case I need to disallow access to private fields (recursively). Is there currently any way to do that? Thanks.
Hi @antonmedv
we tried to pass a map literal to a function in an expression, and the compiler/vm seems to be uncertain about its type (map[interface{}]interface{} or map[string]interface{}.
I add following unit test to illustrate the problem:
package score
import (
"github.com/antonmedv/expr"
"testing"
)
type UserEnv struct {
Var map[string]interface{}
}
func (self *UserEnv) With(userEnv map[string]interface{}) *UserEnv {
self.Var = userEnv
return self
}
func (self *UserEnv) Do(expressionTxt string) interface{} {
node, err := expr.Compile(expressionTxt, expr.Env(self))
if err != nil {
fmt.Printf("%v",err)
return nil
}
out, err := expr.Run(node, self)
if err != nil {
fmt.Printf("%v",err)
return nil
}
return out
}
func TestWith( t *testing.T){
userEnv := UserEnv{
Var : map[string]interface{}{},
}
expressionTxt := `With({'score':3+2+1}).Do("Var['score']*2") `
node, err := expr.Compile(expressionTxt, expr.Env(&userEnv))
if err != nil {
t.Errorf("Error parsing expression (%v): %v", expressionTxt, err)
t.FailNow()
}
out, err := expr.Run(node, &userEnv)
if err != nil {
t.Errorf("Error running expression (%v): %v", expressionTxt, err)
t.FailNow()
}
t.Logf("%v", out)
if out != 12 {
t.Logf("expected 12 got %v", out)
t.Fail()
}
}
I was able to get the desired behaviour by changing line 16 in types.go (package checker). But that might break other stuff.
Can you take a look at it?
Regards,
Bart
On update to go1.13, a package using generated code begins to error
/home/o/go/src/github.com/antonmedv/expr/parser/gen/expr_parser.go:2231: constant 4228579072 overflows int
/home/o/go/src/github.com/antonmedv/expr/parser/gen/expr_parser.go:2249: constant 4228579072 overflows int
/home/o/go/src/github.com/antonmedv/expr/parser/gen/expr_parser.go:2300: constant 4228579072 overflows int
There was no problem previously until the update, i was using the expr package and have almost zero experience with the details this deep but thought it might be wise to post an issue here.
installed the wrong version of go nvm
Right now it looks like a VM instance cannot be re-used (is this correct?), it seems like there is no way to reset the instruction pointer / stack / etc, so the only option is to dynamically allocate and construct and entirely new VM instance for each execution (which also involves dynamically allocating heap memory for the stack, making channels, etc). If you want to run the same expression over large amounts of data it seems like you would want to be able to reuse the VM.
Seems straightforward, but for the life of me, can't get this to work. Keep getting ca_test.go:19: Expected no errors. Got: undefined: beliefs
:
func TestExpr(t *testing.T) {
expression := "beliefs['temp'] > 60 and beliefs['temp'] < 76"
env := map[string]interface{}{
`beliefs`: map[string]interface{}{
`temp`: 58.6,
},
}
out, err := expr.Eval(expression, expr.Env(env))
if err != nil {
t.Errorf("Expected no errors. Got: %v", err)
}
if out != false {
t.Errorf("Expecte `false`, got: %v", out)
}
}
this is the proposal for the following API
type Place struct {
Code string
}
type Segment struct {
Origin Place
}
type Helpers struct {
PlaceEq func(p Place, s string) bool
}
type Request struct {
Segments []*Segment
Helpers
}
code := `Segments[0].Origin == "MOW" && PlaceEq(Segments[0].Origin, "MOW")`
program, err := expr.Compile(code, expr.Env(&Request{}), expr.Operator("==", "PlaceEq"))
if err != nil {
fmt.Printf("%v", err)
return
}
this should pass the type checker and allow to compare Place and string using "==" opertator
Hello,
First of all thanks for this great project :)
I was wondering if Expr can be used to return a pointer to a field of a structure rather than its value ?
Let's say I let the user specifiy source and destination fields in some kind of configuration (strings only for the example).
When it comes to specifying the source, expr already handles this, as the user can write foo.bar.baz
and expr will return the string contained in baz
.
However, I'd like for expr to be able to return the address of baz
so I can do other stuff with it (for example write to it).
Is it something possible ?
Thanks,
Refactor String() methods to print less brackets for binary operators.
Example:
((foo && bar) && baz)
Can be printed as
foo && bar && baz
This optimization should respect operators precedence.
I'm trying to learn to use this package but struggling to understand some basics
A very basic example extracted from custom code I attempted
env := map[string]interface{}{
"distance":"far,
}
expr.Eval("distance==far", env)
You get an error to the effect of "env doesn't contain far"
This is a reduction of what I attempted which has something like
type Attr []byte
type Task struct {
v map[string]Attr
}
where I pass in an expression e.g. "distance==far" to an env of map[string]Attr where I ended up with a swamp of my own code parsing the expression passed in to convert to methods that were unwieldy to work with or not working at all (methods on struct error if they take an interface{}) and still couldn't get it to work as I expected. I am not attempting to be aggressive/accusatory here I sort of just need clarification and pointed to more documentation/examples to find out why I cannot accomplish what I want to accomplish. I might not understand what is going on here and have the wrong package for my use case.
The point being, what is the point? I thought this package was supposed to relate arbitrary expressions to arbitrary data structures, and I have to write a whole other package of parsing and supplementary methods, I might as well start from scratch and do it myself.
It would be great to be able to use date operations like transaction.Date > now+5d
Perhaps using https://github.com/timberio/go-datemath ?
Replace foo in [...]
with foo in {...}
if possible.
I see in doc and usage examples Compile
calls but trying to run an example fails with undefined expr.Compile
.
I have github.com/antonmedv/expr v1.1.4
go doc expr
says:
// Or precompile expression to ast first.
node, err := expr.Parse("expression")
May be docs should be updated?
@antonmedv: First of all, thank you for a great library! I played with it a bit and it is very nicely done.
One thing that looks a bit limited is Regexp support. From what I see the only supported operation is currently string match
which takes a regexp string on the right-hand side.
Would you be interested in enhancing Regexp support? I will be happy to contribute PRs that implement new features. Here is what I would love to see added:
Add Regexp as native supported type with a set of simple functions, e.g. FindMatches, FindSubmatches, etc. that implement more complex functionality related to regexes.
Regexp literal support (e.g. JavaScript-like forward slash-based).
Yes, one can define a custom Regexp()
function which accepts a regex string and creates a Regex type, but that will only work dynamically. With literal support it would be possible to pre-compile regexes.
Is this something you would be interested in adding to the language? I am currently considering using Expr in a product I work on and these capabilities are what I am missing to move forward. I can implement the features myself but wanted to check first if this is inline with how you want to evolve the library.
Thanks again :)
Using expr.Eval seems to work fine though.
I will also submit the following example as a pull request:
`type AnInterface interface {
Method(x string) []AnInterface
}
type A struct{ I int }
func (self A) Method(x string) []AnInterface {
return []AnInterface{self}
}
func ExampleCompileInterfaceMethodCall() {
type B_Env struct{ Ai AnInterface }
env := B_Env{A{6}}
prog, err := expr.Compile(`Ai.Method("x")`, expr.Env(env))
if err != nil {
fmt.Printf("%v", err)
return
}
output, err := expr.Run(prog, env)
if err != nil {
fmt.Printf("%v", err)
return
}
fmt.Printf("%v", output)
// Output: 6
}`
Automatically convert ast for foo in 1..10
from:
binaryNode{operator: "in",
left: nameNode{name: "foo"},
right: binaryNode{operator: "..",
left: numberNode{value: 1},
right: numberNode{value: 10}}}
To:
binaryNode{operator: "and",
left: binaryNode{operator: ">=",
left: nameNode{name: "foo"},
right: numberNode{value: 1}},
right: binaryNode{operator: "<=",
left: nameNode{name: "foo"},
right: numberNode{value: 10}}}
in func getFunc(val interface{}, name string) (interface{}, bool)
there is a switch on d.Kind()
is it possible to add a default for the switch? ie:
default:
method := v.MethodByName(name)
if method.IsValid() && method.CanInterface() {
return method.Interface(), true
}
this way it would be possible to allow methods on arbitrary types,
type X int
func (x X) PrintFormat() { ... }
type B struct { A X }
exp.Eval(`A.PrintFormat()`,B{A:X(10)})
Given bytecode:
0 OpTrue
1 OpJumpIfTrue 2 (6)
4 OpPop
5 OpTrue
6 OpJumpIfTrue 2 (11)
9 OpPop
10 OpTrue
11 OpJumpIfTrue 2 (16)
14 OpPop
15 OpTrue
Can be optimized to
0 OpTrue
1 OpJumpIfTrue 2 (16)
4 OpPop
5 OpTrue
6 OpJumpIfTrue 2 (16)
9 OpPop
10 OpTrue
11 OpJumpIfTrue 2 (16)
14 OpPop
15 OpTrue
Hi, how i can get an error from result of function inside expression language?
Case:
package main
import (
"errors"
"fmt"
"github.com/PaesslerAG/gval"
"github.com/antonmedv/expr"
)
type A struct {}
func (a *A) Test() (string, error) {
return "test", errors.New("it`s error!")
}
func main() {
a := A{}
val, err := expr.Eval(`tx.Test()`, map[string]interface{}{
"tx": &a,
})
fmt.Println(val) //print test
fmt.Println(err) //print <nil>
}
//need:
val, err = gval.Evaluate(`tx.Test()`, map[string]interface{}{
"tx": &a,
})
fmt.Println(val) //print <nil>
fmt.Println(err) //print can not evaluate tx.Test(): it`s error!
}
Error reporting when I run the above code.
error:
invalid operation: "a" + "b" (mismatched types string and string)
如果不能序列化的话,那在分布式系统中应用的话,会是一个大麻烦
Add support for constexpr:
expr.Compile("foo(...)", expr.Env(env{}), expr.ConstExpr("foo"))
If helper declared as constexpr with expr.ConstExpr(string)
, it (helper) should be treated as a pure function. Compile will evaluate it during compile and replace it with the result.
We have been using v1.1.4 of the 'expr' package, which is great, by the way. We finally updated to GO 1.13 and at the same time started migrating to GO modules.
However, when upgrading, our process is failing because of a missing "/v2" in your go.mod module path.
$ go get github.com/antonmedv/[email protected]
go: finding github.com v2.1.1
go: finding github.com/antonmedv v2.1.1
go: finding github.com/antonmedv/expr v2.1.1
go: finding github.com/antonmedv/expr v2.1.1
go get github.com/antonmedv/[email protected]: github.com/antonmedv/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2
I was able to download the source and modify the "go.mod" file to be:
$ cat go.mod
module github.com/antonmedv/expr/v2
go 1.12
require (
github.com/antlr/antlr4 v0.0.0-20190518164840-edae2a1c9b4b
github.com/gdamore/tcell v1.1.2
github.com/rivo/tview v0.0.0-20190515161233-bd836ef13b4b
github.com/sanity-io/litter v1.1.0
github.com/stretchr/testify v1.3.0
golang.org/x/text v0.3.2
)
This helped me get past the above error.
Is there something I am doing wrong? Or is this an issue with the "expr" module. I am new to "go modules" and dealing with the learning curve.
var environment = map[string]interface{}{
"percent": 10,
"amount": 1,
}
program, err := expr.Compile("amount * (percent / 100)", expr.Env(environment))
output, err := expr.Run(program, environment)
if err != nil {
panic(err)
}
fmt.Println("Result : ", output)
Exprected Result : 0.1
Actual Result: 0
It looks like division is always being done using an integer data type instead of a float
Is it possible to use regexes/contains methods on slice elements?
for example (altering the demo code):
package main
import (
"fmt"
"github.com/antonmedv/expr"
)
var expressions = []string{"'^hel.*' in names"}
var environment = map[string]interface{}{
"names": []string{"hello", "hey", "world"},
}
func main() {
for _, input := range expressions {
program, err := expr.Compile(input, expr.Env(environment))
if err != nil {
panic(err)
}
output, err := expr.Run(program, environment)
if err != nil {
panic(err)
}
fmt.Println(output)
}
}
will output true
Is this something we can do in the new v2 version?
When I used go get -u github.com/antonmedv/expr, I met vendor/github.com/antonmedv/expr/type.go:9: syntax error: unexpected = in type declaration this error, after troubleshooting go1 .9 is supported, but our production environment is go1.8, how can I solve it?
Hello guys!
Would like to do simple assignation évaluation, something like:
Var lib string
Code = ‘lib="ok"’
... expr code to parse this thing...
Font.Println(lib) // should print ok
Is it something doable? If yes what code should i use to achieve that?
Hi, I have got the following results from Compile
method
env := map[string]interface{}{
"foo": 20,
"bar": 10,
}
statements := []string{
`foo + bar`, // 30
`(foo) // bar`, // 20
`foo ++++++ bar`, // 30
`foo +++- bar`, // 10
`/foo * bar`, // Compile error: syntax error: extraneous input '/' expecting {'len', 'all', 'none', 'any', 'one', 'filter', 'map', '[', '(', '{', '.', '+', '-', Not, '#', 'nil', BooleanLiteral, IntegerLiteral, FloatLiteral, HexIntegerLiteral, Identifier, StringLiteral} (1:1)
`foo * bar/`, // Compile error: syntax error: extraneous input '/' expecting {'len', 'all', 'none', 'any', 'one', 'filter', 'map', '[', '(', '{', '.', '+', '-', Not, '#', 'nil', BooleanLiteral, IntegerLiteral, FloatLiteral, HexIntegerLiteral, Identifier, StringLiteral} (1:11)
`foo +++-* bar`, // Compile error: syntax error: extraneous input '*' expecting {'len', 'all', 'none', 'any', 'one', 'filter', 'map', '[', '(', '{', '.', '+', '-', Not, '#', 'nil', BooleanLiteral, IntegerLiteral, FloatLiteral, HexIntegerLiteral, Identifier, StringLiteral} (1:9)
}
Is there a way to make it more strict?
The documentation states that marshaling and unmarshaling of a program is supported.
However, the following test fails:
func Test_marshalRegexp(t *testing.T) {
prog, err := expr.Compile(`"hello" matches "h.*"`)
if !assert.NoError(t, err) {
t.FailNow()
}
marshaled, err := json.Marshal(prog)
if !assert.NoError(t, err) {
t.FailNow()
}
unmarshaledProgram := &vm.Program{}
err = json.Unmarshal(marshaled, unmarshaledProgram)
if !assert.NoError(t, err) {
t.FailNow()
}
output, err := expr.Run(unmarshaledProgram, nil)
if !assert.NoError(t, err) {
t.FailNow()
}
assert.Equal(t, true, output)
}
For my use case I need to allow only small subset of operators, is there currently any way to do that?
Windows 7 32-bit
go1.11.2 windows/386
Trying to do:
package main
import (
"github.com/antonmedv/expr"
)
func main() {
p, err := expr.Compile("1+2")
}
Build fails with multiple errors:
Installation failed: # github.com/antonmedv/expr/parser/gen
..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2231: constant 4228579072 overflows int
..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2249: constant 4228579072 overflows int
..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2300: constant 4228579072 overflows int
In python, you can do [i ** 2 for i in range(10)]
to iterate through each value in range and square each one and get a list of squares all in one expression. I propose a similar syntax for expr:
[v ** 2 for _, v := range 0..9]
This would return the slice of all the numbers from 0 to 9 squared.
Another thing you could do is a conditional comprehension, for example
[[i, v] for i, v := range [1, 4, 7, 2, 3] if v <= 5]
This would return a [][]int
of the numbers and indexes of those numbers where the number is less than or equal to 5. In this case it would be [[0, 1], [1, 4], [3, 2], [4, 3]
.
This feature would make this package more powerful and greatly increase the potential of it, as currently there is no way to return a slice outside of the range operator, which is limited.
Add support for tags expr:"..."
for names aliases.
type Env struct {
Value int `expr:"value"`
}
And now it's will be possible to use value
instead of Value
in expressions.
var timeNil *time.Time
var out interface{}
out, err = expr.Eval("t == nil", map[string]interface{}{"t": timeNil})
fmt.Println("Output", out, err)
// Output false <nil>
go.mod
github.com/antonmedv/expr v0.0.0-20190710105945-85ffe5789329
Hi @antonmedv. I have a question about ast
. In this example:
import "github.com/antonmedv/expr/ast"
type visitor struct {
ast.BaseVisitor
identifiers []string
}
func (v *visitor) IdentifierNode(node *ast.IdentifierNode) {
v.identifiers = append(v.identifiers, node.Value)
}
program, err := expr.Compile("foo + bar", expr.Env(env))
visitor := &visitor{}
ast.Walk(node, visitor)
fmt.Printf("%v", visitor.identifiers) // outputs [foo bar]
How do we convert program into node in order to use ast
?
https://github.com/antonmedv/expr#examples
excample "Using embedded structs to construct env." has error: "out, err := expr.Run(p, env{"world"}) " should be "out, err := expr.Run(p, env{Name: "world"}) " ?
Implement slicing operator array[start:end]
With same as Go semantics:
b := []byte{'g', 'o', 'l', 'a', 'n', 'g'}
// b[1:4] == []byte{'o', 'l', 'a'}
// b[:2] == []byte{'g', 'o'}
// b[2:] == []byte{'l', 'a', 'n', 'g'}
// b[:] == b
https://github.com/antonmedv/expr/blob/33a808e9f8835ba017273759a1441fb256b22ae4/vm/runtime.go#L44
Interfaces store a pair of either (type, ptr) or (type, value), the current runtime code will convert (type, ptr) to (type, value) and then recurse, but that conversion means creating a value copy, meaning value := *ptr. This copy is definitely unnecessary and a performance concern for large/deep structures.
The solution is quite simple, the fetch function can unpack the ptr without going through an additional interface/function call, ex:
func fetch(from interface{}, i interface{}) interface{} {
v := reflect.ValueOf(from)
kind := v.Kind()
if kind == reflect.Ptr {
v = v.Elem()
kind = v.Kind()
}
switch kind {
case reflect.Array, reflect.Slice, reflect.String:
index := toInt(i)
value := v.Index(int(index))
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
case reflect.Map:
value := v.MapIndex(reflect.ValueOf(i))
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
case reflect.Struct:
value := v.FieldByName(reflect.ValueOf(i).String())
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
}
return nil
}
Additionally, you can support chasing through **T like:
for ; kind == reflect.Ptr; kind = v.Kind() {
v = v.Elem()
}
This will then follow any amount of indirection until you get a type, meaning the environment could be a pointer to a pointer to a struct.
I'm happy to create a pull request with the change.
Thanks
I'm wondering if version v2 of the project is available as a go module?
See the following error:
$ take foo
$ go mod init
$ go get gopkg.in/antonmedv/expr.v2
go: gopkg.in/antonmedv/[email protected]: go.mod has non-....v2 module path "github.com/antonmedv/expr" at revision v2.0.5
go: error loading module requirements
Thanks!
I used ast.walk function, I can find all identifier nodes in expression.
But I want to replace some identifier nodes, then generate new expression, then compare it.
How is this achieved?
Is it possible to disable some language features by settings the VM (or parser?). My use case is I have a simple expression evaluator which supports only number/string operations (no map support, for example).
Allow this for in
operator:
key in {a: 1, b: 2}
At v1.5.7 the parser no longer accepts trailing commas for maps and lists:
[1, 2, 3,]
{"a": 1, "b": 2, "c": 3,}
Now running these expressions returns:
unexpected token Bracket(\"]\")
a map key must be a quoted string, a number, a identifier, or an expression enclosed in parentheses (unexpected token Bracket(\"}\"))
Is this expected? I got these errors after upgrading from v1.4.1 and there was no problem before.
Thanks in advance.
100*1.05
vs 100+5%
I'm thinking about adding this +n%
syntax to expr library. What do you think about it? Please share your opinion.
Expr has already %
sign as mod: 42 % 5 == 2
. But new syntax can be ambiguous. To deal with it it will be handled on the parse phase and allowed only in A+N%
and A-N%
constructions.
Expr language is also made for people without programming background and for them, it will be more easy to deal with such syntax:
111 * (1 - 0.11)
vs 111 - 11%
Please, share your opinion.
Currently, returned errors are of type *errors.errorString
. This prevents inspection of the error to get specific details. For example, to isolate the message, location, and source code of the error. Something along the lines of:
result, err := expr.Eval(`"test" + 3`, nil)
if err != nil {
if exprErr, ok := err.(expr.Error); ok {
// exprErr.Message
// exprErr.Location
// exprErr.Source
// ...
}
}
Currently there is no type checks for function arguments:
https://github.com/antonmedv/expr/blob/425d51925274929795656c4ed3065079d68551ad/type.go#L150-L163
Will be cool to add those. Use reflect for getting function arguments and compare with node's arguments types. Gather types at line 152, get f
arguments in line 159 and check.
Same for builtins:
https://github.com/antonmedv/expr/blob/425d51925274929795656c4ed3065079d68551ad/type.go#L143
func TestNumericKey(t *testing.T) {
var request = struct {
Msg map[string]interface{}
}{
Msg: map[string]interface{}{
"a": 150,
"4": 150,
"a4": 150,
"4a": 150,
},
}
{
// Works
rule, err := expr.Compile(`Msg.a > 100`, expr.Env(&request))
require.NoError(t, err)
require.NotNil(t, rule)
output, err := expr.Run(rule, &request)
require.NoError(t, err)
require.Equal(t, true, output)
}
{
// Works
rule, err := expr.Compile(`Msg['4'] > 100`, expr.Env(&request))
require.NoError(t, err)
require.NotNil(t, rule)
output, err := expr.Run(rule, &request)
require.NoError(t, err)
require.Equal(t, true, output)
}
{
// Works
rule, err := expr.Compile(`Msg.a4 > 100`, expr.Env(&request))
require.NoError(t, err)
require.NotNil(t, rule)
output, err := expr.Run(rule, &request)
require.NoError(t, err)
require.Equal(t, true, output)
}
{
// Fails
_, err := expr.Compile(`Msg.4a > 100`, expr.Env(&request))
require.Error(t, err)
}
{
// Fails
_, err := expr.Compile(`Msg.4 > 100`, expr.Env(&request))
require.Error(t, err)
}
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.