open-policy-agent / npm-opa-wasm Goto Github PK
View Code? Open in Web Editor NEWOpen Policy Agent WebAssembly NPM module (opa-wasm)
License: Apache License 2.0
Open Policy Agent WebAssembly NPM module (opa-wasm)
License: Apache License 2.0
It's now expected to be false instead of undefined.
When fixing this, we should be able to bump the version of OPA used in CI to 0.43.0 (or 0.44.0 when it's released).
Hi hi. Master has a different (and better) interface than the current npm release. It also seems fairly stable. Any chance for a release? At least a canary one? I'm relying on master atm.
Type are referenced in the package.json
however they are not include in the build. This was working previous at version 1.4.0
So this is the error when using with Typescript
Could not find a declaration file for module '@open-policy-agent/opa-wasm'. '..../node_modules/@open-policy-agent/opa-wasm/src/index.mjs' implicitly has an 'any' type.
There are types at '.../node_modules/@open-policy-agent/opa-wasm/dist/types/opa.d.ts', but this result could not be resolved when respecting package.json "exports". The '@open-policy-agent/opa-wasm' library may need to update its package.json or typings.ts(7016)
Requested resolution is to include the types again
I've seen projects who render their API docs into the README, that would be a simple first step, I think. Eventually, these could be documented in some way that's "native" for NPM packages...
We don't have anything like that, do we? 🔍 👀
I am trying to build policies with jwt decoding and verify functions (io.jwt.decode, io.jwt.verify_rs256) in a node js application. However I am getting an error not implemented: built-in function io.jwt.verify_hs256
the same goes for not implemented: built-in function io.jwt.decode
.
I followed the examples available in the /examples/nodejs-app. For simple policies, it is working like a pro, but with the JWT processing functions.
Am I missing something here or these functions are not supported?
Beginner here.
Not sure why using 'data.example = x' instead of 'data.example' here.
Any advice?
Thanks.
Any test cases from OPA that include x in xs
(internal.member_2) or x, y in xs
(internal.member_3) are skipped right now, because the capabilities.json of this project doesn't include those built-ins.
These two builtins, however, have native wasm implementations, so we should be able to just run their tests by adding their definitions to the capabilities.json.
Right now, the test runs report, for example,
✎ todo aggregates/member simple, set: error: 1 error occurred: /tmp/tmp-3091-1fakAHvoZuVN:4: rego_type_error: undefined function internal.member_2
and skip the test case that includes the call.
The current version of the NPM module on npmjs is still at version 1.1.0 (10 months ago) while new changes were added to the master branch 4 months ago. Bumping the version of the master branch and pushing that new version to the NPM repository would solve this issue!
Hello, this might be obvious, but can you document or add an example of using this package to filter a query and not just allow/deny?
Also from #36:
This module is based on commonjs and considering ES Modules is the standard, providing support for that would be great. Currently, working with esModuleInterop and importing like this:
import opa from "@open-policy-agent/opa-wasm"; const { loadPolicy } = opa;
Howdy! Stumbled across this project and thing it's a really cool way to open up WASM to NodeJS and also web. Had a few bits of feedback if that's ok:
loadPolicy
instead of load_policy
for example to be a bit more javascript-y?I'd also be willing to help with #13, have set up quite a few packages & js builds recently.
I'm attempting to use a policy in a flutter application, using JavascriptCore.
I had a bit of trouble with the promise returned by loadPolicy
which was removed when I loaded using new WebAssembly.Instance
.
I'm guessing there are performance reasons why you can't do that in a browser, but maybe there could be a loadPolicySync
?
const policy = opa.loadPolicySync(new WebAssembly.Module(wasmBinary))
const wasm = new WebAssembly.Instance(policyWasm, {
env: {
memory,
opa_abort: function (addr) {
throw addr2string(addr);
},
opa_println: function (addr) {
console.log(addr2string(addr));
}...
I'm happy to do a pull request if it's something you'd accept.
the below policy rule is working in OPA as server and rego Playground.
But the same policy is failing below error, when I deploy as wasm file and calling it from nodeJS.
{ message: 'not implemented: built-in function 1: startswith' } the same issue i see for contains function too.
package example
isPrechecked {
startswith(inputs.country, "AT")
}
With open-policy-agent/opa#3627 making its way into a release soon, it would be nice if this SDK could make use of it for faster evaluations.
Thanks for creating this, trying it out and would like to make use of the newer customBuiltins
option but I notice the latest release to npm is from February which does not have this functionality.
I can't find any information about when releases will happen but in any case are there plans on continuously releasing as pre-releases to npm so newer functionality can be made use of even though it may not be yet stable?
OPA has been adding more builtins to the WASM binary (implemented in C and included in any compiled query). We should synchronize the NPM module to remove any builtins that are no longer required.
The current map of supported builtins from OPA can be found here: https://github.com/open-policy-agent/opa/blob/master/internal/planner/planner.go#L26
Note: This will likely need to wait for the v0.20.0 release, they are only in master
as of right now.
When my policies use sprintf
I receive the following error. This is using the code in master.
ERROR: TypeError: s.sprintf is not a function
at sprintf (/Users/garethr/Documents/opa-wasm/node_modules/@open-policy-agent/opa-wasm/src/builtins/strings.js:7:28)
at _builtinCall (/Users/garethr/Documents/opa-wasm/node_modules/@open-policy-agent/opa-wasm/src/opa.js:101:18)
at opa_builtin2 (/Users/garethr/Documents/opa-wasm/node_modules/@open-policy-agent/opa-wasm/src/opa.js:137:16)
at <anonymous>:wasm-function[557]:0x357df
at eval (<anonymous>:wasm-function[558]:0x35891)
at LoadedPolicy.evaluate (/Users/garethr/Documents/opa-wasm/node_modules/@open-policy-agent/opa-wasm/src/opa.js:233:31)
at /Users/garethr/Documents/opa-wasm/app.js:15:2
Today, the output of opa build -t wasm
is a bundle.tar.gz file.
It will be really nice if we will be able to load a bundle, like in the Go SDK.
Often, the data to be set on a policy instance is already present as a string of JSON. Right now, we'll have to parse that, pass it on as object, and have setData
marshal it into a string again.
This could be avoided if setData
optionally accepted a string and would use that as-is.
There's an edge case of data = "foo"
, where marshalling it would set data to \"foo\"
. I think this is very uncommon... but we could still, to keep backwards-compatibility, have another method, setDataString
or setDataRaw
, instead... 🤔
RangeError: Source is too large
at Uint8Array.set ()
at LoadedPolicy.evaluate (/opt/server/node_modules/@open-policy-agent/opa-wasm/src/opa.js:303:13)
I got this error when call evaluate with some payload. I'm using v1.4.0. Is there any solution to solve this issue?
Have the examples been tested recently and are they supposed to work?
I have not been able to get any to work:
With nodejs-ts-app-multi-entrypoint I am getting:
[email protected] start
ts-node app.ts
Running multi entrypoint demo suite
Iterations: 100000 iterations of 10 inputs for 1000000 total evals per entrypoint
default entrypoint: 10.013s
ERROR: entrypoint example/one is not valid in this instance
With nodejs-ts-app I am getting an empty result:
[email protected] start
ts-node app.ts "{"message": "world"}"
[]
With nodejs-app I also get an empty result.
I am running Node.js v14.17.5 on Windows 10.
It would be great if users looking at this library found starting points for different settings.
These come to mind, but I'm sure there are more:
I wanted to give calling a WASM-compiled policy from C# a try; is there documentation about the ABI & how to use it aside from trying to reverse engineer from https://github.com/open-policy-agent/npm-opa-wasm/blob/master/src/opa.js?
The plan would be to use wasmer (via WasmerSharp).
There currently isn't any CI configured for the project. As a good starting point what we want is
These should be done for each PR as a check, and post merge on master.
Longer term we would want to support automatically building/pushing the module itself to automate the release process. We don't need it in the initial configuration but whatever tooling we pick needs to support it.
The easiest thing would likely be to do this with Github Actions.
Hi folks, This is a neat project. I've run into the OPA wasm incompatibility addressed in another issue. I can point at the git repository for now, but it looks like y'all haven't published to NPM in quite some time. Is it not ready for another release to NPM?
I'm getting Error: Cannot find module 'utf8'
with both nodejs-app
and nodejs-ts-app
Also with nodejs-ts-app
it isn't resolving the type definitions properly so I made these changes to tsconfig:
"strict": false,
"noImplicitAny": false,
And the docs for nodejs-ts-app
says to do npm run -- '{\"message\": \"world\"
but that is wrong, it should be npm start -- '{\"message\": \"world\"
If you try to build using the new OPA build command and load it with our current examples you get an error like:
ERROR: LinkError: WebAssembly.instantiate(): Import #7 module="env" function="opa_println" error: function import requires a callable
Reference: #18 (comment)
First off, just want to say that I appreciate the work you guys have been doing. Hoping you guys can help me with an issue. Apologies if this is not an appropriate place to post this (feel free to redirect me if so).
The issue: My policies output messages using sprintf. For input, I have a YAML file (specifically, an AWS CloudFormation template) that uses functions like !Ref. In the case of !Ref, I want to output the variable name. When I evaluate my policies in terminal, my message returns what I want: if the BucketName value is “!Ref Something”, it returns “Something”. However, when I evaluate using the same policies and same input through the SDK, it returns [object Object].
Any ideas? It’s definitely a TypeScript/JavaScript thing, but because it’s already returned as “[object Object]” in a String upon evaluation, I don’t have any way to inspect it.
Executing policy.evaluate(...)
multiple times results in a memory leak.
If the memoryDescriptor has no maximum specified, this results in the wasm module growing the number of pages internally (via __opa_malloc_new_allocation
). If a maximum is specified this would eventually result in an opa_abort once the maximum page size is reached.
Test case replicating the issue:
it("memory leak reusing the same policy multiple times", async () => {
const policy = await loadPolicy(policyWasm, { initial: 2, maximum: 8 });
const input = "a".repeat(2 * 65536);
for (const iteration of new Array(16)) {
expect(() => policy.evaluate(input)).not.toThrow();
}
});
Unsurprisingly calling setData(...)
before each evaluation makes the test pass, since the heap pointer is set back to the baseHeapPtr
.
it("memory leak reusing the same policy multiple times", async () => {
const policy = await loadPolicy(policyWasm, { initial: 2, maximum: 8 });
const input = "a".repeat(2 * 65536);
for (const iteration of new Array(16)) {
policy.setData({})
expect(() => policy.evaluate(input)).not.toThrow();
}
});
I was experimenting with loading a wasm-compiled policy using Cloudflare and ESM-worker. Basic setup something like this:
import { loadPolicy } from "@open-policy-agent/opa-wasm";
import _wasm from "../opa/policy.wasm";
export default {
async fetch(request) {
const policy = await loadPolicy(_wasm);
policy.setData({ world: "world" });
const input = await request.json();//{"message": "world"};
const resultSet = policy.evaluate(input);
return new Response(JSON.stringify(resultSet));
},
};
Unfortunately I got an error that I traced to wasm.instance
being undefined.
I tested using the opa-wasm js-files locally, and it worked when using only wasm
instead of wasm.instance
.
Since I see this env.instance = wasm.instance ? wasm.instance : wasm;
elsewhere in the code, I guess it is sort of known issue, but not handled everywhere in the code.
ABI version 1.2 adds a method, and keeps all the existing ones around. This shouldn't explode on an ABI 1.2 module.
As developer, I want to use Compile API so that I can use result for query filtering.
We're developing a node/typescript app on a serverless stack (Google Cloud Run, Could Functions). We would like to use OPA for centralized API security enforcement. Could you please recommend a way to partially evaluate policy with unknown parts of the data to build data filters from rego policies and an incoming request?
Possible workaround(?): run OPA as separate service like in: https://github.com/zotoio/sls-lambda-opa.
Thank you!
The LoadedPolicy class is not exported so its type cannot be used in Typescript based code.
It would nice to have it exported for Typescript users.
I'd have created a PR for that but I'm fairly new to this process here. Do you have any recommendations?
I am trying to create an Angular service that evaluates an OPA policy but have not been able to get it to work and I have not come across any samples. First question: is this a valid/supported use case?
It is very simple, basically a "Hello World". The equivalent in Node (replacing 'fetch' with reading the file via 'fs') works without a problem but in Angular, I get the following:
Error: export 'loadPolicy' (imported as 'loadPolicy') was not found in '@open-policy-agent/opa-wasm' (possible exports: default)
At design time, I can clearly see that the package does indeed export 'loadPolicy'.
Here is the code:
import { Injectable } from '@angular/core';
import { loadPolicy } from "@open-policy-agent/opa-wasm";
@Injectable({
providedIn: 'root'
})
export class RulesService {
constructor() { }
async evaluateFilterRules(context: string): Promise<string> {
var result = '{}';
await fetch('/assets/wasm/filter.wasm').then(response =>
response.arrayBuffer()
).then(bytes =>
loadPolicy(bytes)
).then(
policy => {
result = policy.evaluate(context);
},
);
return JSON.stringify(result, null, 2);
}
}
Any pointers would be much appreciated.
Hello Team,
Can you help me with example on how to compile policy using REST API instead of opa cli?
Thanks.
For context: We can release whenever from master, but there are a few outstanding issues that have been holding up some of the progress. At a high level the thinking is that until we have #13 #14 and maybe #15 completed it doesn't make much sense to publish another release via npm. The implication being that the tagged/published release should work correctly and will be supported, but without the testing in place from those issues it's hard to be confident on that.
In the meantime, for anyone looking to use the npm module, please use the git ref until the next release is cut.
Contributions towards those blocking issues are of course welcome 😄
As highlighted in the report #36, it would be useful if multiple entrypoints were supported by the SDK: You could define one or more entrypoints for policy evaluation, and add a few for checking the state of your WASM instance's data.
It's been a WASM feature since OPA v0.25.0 (open-policy-agent/opa#2844).
This might tie in with #3
https://github.com/open-policy-agent/npm-opa-wasm/blob/master/src/opa.js#L61
Haven't run it, but the comment looks weird, and a couple lines later
var argArray = Array.prototype.slice.apply(arguments);
Looks very much like arguments comes out of thin air.
We currently have unit tests for the javascript and documented steps with an example to do the full Rego -> WASM -> evaluate in node flow, but we don't have any automated tests for that.
Long term we would want to essentially use all of the tests that OPA has for its topdown
package (which is a comprehensive set of tests for the language). However there is quite a bit of work that would need to be done in OPA to separate them all.
Shorter term we can potentially look into using the WASM tests it uses https://github.com/open-policy-agent/opa/tree/master/test/wasm/assets and evaluate them using the NPM module to ensure compliance. We would need to enrich these with tests for additional builtins supported by the NPM module versus the ones baked into the WASM binary.
As a first step lets just automate the example we currently have, plus coverage for the builtins we implement, to ensure end to end functionality of the SDK.
This would ideally be integrated with the CI checks from #13
For OPA as of v0.20.0 there is a newer opa build
command that works differently than the older one that only generated WASM binaries.
We will need to update the examples and documentation to show using the newer version.
For the example app it should now be something more like:
opa build -t wasm ./example.rego -e 'data.example'
tar -xzf bundle.tar.gz
And then there will be a policy.wasm
file in the current directory
👉 reported in #89
loadPolicy
should take another argument indicating the maximum memory to be usedevaluate
should figure out if the current memory is large enough to write the input to,
Just wondering if I coud use this package to consume the APIs Policies in a Wasm bundle version on the browser.
Would there be any security concers ?
As-is the SDK has a few builtins added:
And the list that OPA includes baked into the WASM binary is increasing, current list at: https://github.com/open-policy-agent/opa/blob/master/internal/planner/planner.go#L26
But there are still a number of missing ones (ex: #22 ).
Before we do anything we need to identify what functions are missing, and how we want to stay in sync with OPA. Over time it will include more builtins, so we will need to slowly deprecate and remove the ones we provide in the javascript SDK. We should figure out a plan for that..
We also need to document any missing functions to help users of the SDK understand what they can and cannot do with the SDK in its current form.
Hi. I have been trying to use this module with Node and Typescript with WASM but have been facing quite a few challenges. Thought of adding all of it in this issue.
I initially tried installing the package from NPM and the examples did not work since the package was last published 1 year ago and there has been some breaking changes since then. For instance, load_policy
has become loadPolicy
and so on. Currently, downloaded this repo and working with it. I guess it is tracked by #31
This module is based on commonjs
and considering ES Modules is the standard, providing support for that would be great. Currently, working with esModuleInterop
and importing like this:
import opa from "@open-policy-agent/opa-wasm";
const { loadPolicy } = opa;
While using this, I wanted to check whether the data has actually been loaded when I do setData
but this module currently does not expose any API for getting the data which is loaded. I wanted to know the structure of the data loaded and also confirm that the data is available but all I am able to get back now if I do console.log(policy)
is this from which I am not sure how to infer:
I am not sure if I am doing something wrong here, but when I do evaluate
the response I get is just one of these:
[ { result: true } ]
or
[ { result: false } ]
which is inconsistent with the results I am getting in the Playground or VSCode extension for the same rego files, data and input.
For instance, my output in playground looked something like this:
{
"myrule": [
{
"msg": "test failed"
}
],
"testOutput": "test failed"
}
Also, while OPA WASM supports non-boolean results now as per this: open-policy-agent/opa#1118 I am not sure if this module does support it.
Thanks.
I am trying to integrate the OPA in an Angular codebase. I referred to the node-js-app
from the /examples
folder.
Following is the Angular code to load the wasm
file:
constructor(private http: HttpClient) {
this.loadWasmFile("./policy.wasm");
}
private loadWasmFile(url: string): void {
this.http.get(url, { responseType: "arraybuffer" }).subscribe(
(buffer: any) => {
loadPolicy(buffer)
.then((policy) => {
const input = '{"message": "world"}';
policy.setData({ world: "world" });
const result = policy.evaluate(input);
console.log(JSON.stringify(result, null, 2));
})
.catch((err) => {
console.log("ERROR: ", err);
});
},
(error) => {
console.error("Error loading WASM file:", error);
}
);
}
}
On load I am getting the following error:
ERROR: CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 3c 21 44 4f @+0
I generated the wasm
file using the opa build
cmd given in the README of node-js-app
.
Can we support Partial evaluation?
https://www.openpolicyagent.org/docs/latest/policy-performance/
https://blog.openpolicyagent.org/partial-evaluation-162750eaf422
When can a new release be expected?
The tagged version is broken when using loadPolicy
.
Would be happy to help and contribute to push that.
I get error: illegal call: unknown operator contains
when trying to build the policy.
package package.name.here
default pass = false
pass {
string_input := "Full String Here"
contains(string_input, "String Here")
}
Tested it in the Rego playground here https://play.openpolicyagent.org/
EDIT: more clarification, the policy works in the playground but fails to build with opa
to generate .wasm
file
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.