h3ku / starlarky

VGS edition of Google's safe and hermetically sealed Starlark language

Starlarky
VGS edition of Google's safe and hermetically sealed Starlark language

• Description
• Project overview
  • Libstarlark
  • Larky
  • Runlarky
  • Pylarky
• Developer setup
• Depoyment process

Description

Starlarky is VGS in-house edition of Bazel hermetically-sealed language created by Google called Starlark. This language is used to run "unsafe" user-submitted code without exposing service at whole to possible attack and/or vulnerabilities. Starlark has Python-like syntax and is created to support same structure of additional libraries. Key differences between Starlark and Python can be found here

Project overview

Starlarky is presented as a monorepo with different modules

Libstarlark

Libstarlark is a maven module, that contains Starlark compiler from bazelbuild This module is being periodically updated from bazelbuild via this script to maintain relevancy.

See more at Libstarlarky README

To build run this command:

mvn versions:set -DnewVersion=<your-version> -pl libstarlark (optional)
mvn clean package -pl libstarlark

Larky

Larky is a maven module, that contains VGS additions to Starlark language. Some additions ispired and taken from Copybara

Here are some of them:

• JSR223 script engine
• Annotations to define additional libraries
• Extension modules

To build run this command:

mvn versions:set -DnewVersion=<your-version> -pl larky (optional)
mvn versions:set-property -Dproperty=libstarlark.version -DnewVersion=<larky-version> -pl larky
mvn clean package -pl larky

Runlarky

Runlarky is an example Larky invocation application It builds as a Quarkus executable and gives ability to run Larky with input parameters.

To build run this command:

mvn versions:set -DnewVersion=<your-version> -pl runlarky (optional)
mvn versions:set-property -Dproperty=starlarky.version -DnewVersion=<larky-version> -pl runlarky
mvn clean package -pl runlarky -Pnative

This would build larky-runner executable in runlarky/target directory, that can be run from terminal

Pylarky

Pylarky is pip lib-wrapper for runlarky to make larky calls conviniently from Python.

Building and Running Tests

docker-compose build
docker-compose run local bash /src/build-and-test-java.sh
docker-compose run local bash /src/build-and-test-python.sh

Run individual larky stdlib test

mvn -Dtest='StdLibTest*' -Dlarky.stdlib_test=test_bytes.star org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test -pl larky

Developer setup

In addition to having Maven installed, it must be configured to retrieve artifacts from Github.

1. Generate an access token using Github's instructions. The token needs read:packages scopes.

2. If you're on Github Enterprise (which VGS employees are), follow the image below:

3. Place the token in your ~/.m2/settings.xml file. For example (look for github-username and github-api-key to be replaced with your values):

<?xml version='1.0' encoding='us-ascii'?>
<settings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0                           https://maven.apache.org/xsd/settings-1.0.0.xsd">
      <localRepository />
      <interactiveMode />
      <usePluginRegistry />
      <offline />
      <pluginGroups />
      <servers>
          <server>
              <id>github</id>
              <username>github-username</username>
              <password>github-api-key</password>
          </server>
      </servers>
      <mirrors />
      <proxies />
      <profiles />
      <activeProfiles />
    </settings>

Deployment process

To rollout a new verion of libstarlark/larky/larky-api create a new tag

git tag x.x.x
git push origin x.x.x

Than, after CircleCI build, publish the draft release