metatypedev / metatype Goto Github PK

• Expose more parameters
  • Queues
  • Namespaces
• Read temporal address from secret
• More tests

Currently, eget is used to install and update the meta-cli from GitHub releases. However eget must be installed at first, and may not work in all cases. Let's replace it by a custom installation script over https. Take inspiration from https://github.com/jpillora/installer.

curl https://metatype.dev/meta-cli.sh | bash 

The issue seems to be around the custom meta extension. At startup, deno tries to locate the runtime.js initializer at the path it'd seen it during compilation instead of bundling it within.

I believe, in order to solve this, we'll need to start using snapshots. Or use them properly as the deno crate's using them by default, it's just that the meta extension isn't included in that snapshot. This will mean a few more changes in embedded deno repo as well.

On the very off chance something goes wrong when loading the typegraph...which entails shelling out to deno or python, timing out can provide a path to sane error messages or recovery.

Some of the logs/traces emitted seem very adhoc. They contain very little context info and what ought to be trace level logs being present in debug prove a challenge to debugging.

Edit: worst offenders seem to be the ones present in the typescript section of the typegate. The rust logs have the function's type_path.

A general improvement in instrumentation and maybe OpenTelemetry for the typegate seems like a good idea. Instrumenting a large code-base in general is a bit daunting so best to start picking away at the mountain early.

Consider moving from the log crates to the tracing as well as they have better mind share/more features for otel and whatnot.

For each target in metatype.yml:

• check host reachability and typegate status
• check secrets accesses

Sample command invocation in this case:

$ meta deploy -f ./orcman/api/typegraph.ts --allow-dirty --create-migration --target dev --gate http://localhost:8907

Sample output (notice the EXITED value):

Tracking issue for:

• https://github.com/metatypedev/metatype/security/code-scanning/2

meta new should install typegraph based on pyproject.toml (if any) or in the python env. Always showing what is automatically done to the user by logging at the correct level.

Updating Cargo.lock whenever dev/lock.ts runs increases maintenance burden as breaking changes despite semantic versioning. Not API breakages ofcourse but bulid breakages. This usually comes from the deno dependencies due to the nature of the integration.

Solving this could look like:

• Update dev/lock.yml to minimize changes to lockfiles
• Weekly or monthly CI job that creates a PR just for this purposes.

To be specific, this specific alpine image used by the deno-alpine image as a base. (Also used by the ghjk docker based test suite).

Since the difference between the two's the full and thin version of the cli is the bundled typegate, this is technically an issue within typegate.

Exposing the v8 flags to end users allows tuning the engine for specific workloads. Most critically, increasing the heap size. Hopefully, memory optimization will mean this won't be necessary for most but it's always good to have the option.

Ideally, the flags ought to be configurable from env vars since the primary method of distribution for the typegate are container images.

...for the typegate that exposes simple metrics like sizes of uploaded typegraph and resident memory usage. To be consumed by the meta doctor command.

Note the following run:

It doesn't have a cache hit despite:'

Which was generated from this run mere minutes earlier.

meta upgrade currently only update itself. The command can update:

• the compose.yml and reload the typegate
• the python module and pyproject.toml/requirements.txt (if any)

As for the executable, the user should be prompted with the suggested action.

JSR.io is a new package registry by the folks at Deno. It seems to have great compatibility story across runtimes and in theory, publishing only there should be enough. It has a concept of scores for packages to describe some measure of quality (like documentation and good typing) so maybe a few hours to improve those might be a good idea..

What it says on the tin can. Error is shown here:

Consider exposing all the dev/*.ts scripts under ghjk task as well.

Followed tutorial:

[ERROR] Failed to load typegraph(s) from "api/example.py": error while loading typegraph(s) from "/home/oli/metatype/demo/api/example.py":   File "/home/oli/metatype/demo/api/example.py", line 8
[ERROR]>     ) as g:
[ERROR]>     ^
[ERROR]> SyntaxError: unmatched ')'

Changed code to:

  ) as g:
  # declare runtimes and policies

Now I get:

[ERROR] Failed to load typegraph(s) from "api/example.py": error while loading typegraph(s) from "/home/oli/metatype/demo/api/example.py": Traceback (most recent call last):
[ERROR]>   File "/home/oli/metatype/demo/api/example.py", line 4, in <module>
[ERROR]>     with TypeGraph(
[ERROR]> TypeError: TypeGraph.__init__() got an unexpected keyword argument 'allow_origin'

0.3.3 carries a bug where prisma migrations are broken because the temp dir root isn't created before hand. The mktmpd calls don't recursively create the parent directories which is reasonable behavior I suppose.

A fix would be to either make sure the tmp dir is present at typegate initialization or make all locations where tmp dirs are created recursive. (mkdir -p behavior)

Some tasks that should be considered for any CI rework effort:

• Replace rust-cache by something better, possibly mozilla/sccache. Something that works well with very large and numerous build artifacts.
• Replace all action-rs actions as they've been archived by the authors
• Fix dev/lock.ts to avoid dependency churn
• Use ghjk envs for more things.

Bring up any ideas for improving the dev workflow, improve codebase quality and get rid of paper cuts.

Ideas like:

• Embrace cargo workspaces
• Move on to the tracing ecosystem from log
  • Setup a guideline for instrumentation of the code to be followed pro-actively (needs a separate issue ig)