software-mansion / scarb

The Cairo package manager

Scarb

Scarb is the project management tool for the Cairo language. Scarb manages your dependencies, compiles your projects and works as an extensible platform assisting in development.

Documentation

All information about Scarb is available on project's website.

• Installation
• Cheat sheet
• Documentation

Changelog

All notable changes to this project are documented on the GitHub releases page.

Roadmap

Scarb is under active development! Expect a lot of new features to appear soon! 🔥

• Building Cairo packages
• Pulling dependencies from local filesystem
• Pulling dependencies from Git
• Machine-readable workspace metadata generation
• Built-in Cairo compiler plugins as packages (the starknet package)
• Feature parity with Cairo compiler CLIs
• Scarb installer
• ASDF plugin
• GitHub action
• Workspaces
• Nightlies
• Scarb.lock
• scarb update
• scarb check
• Standardized test target 🏗️
• Package registry 🏗️
• PubGrub implementation for version resolution
• Dynamic loading of custom Cairo compiler plugins as Scarb packages 🏗️
• Dynamic loading of custom targets aka code generators
• Dependency overrides
• Signing & notarization of Windows & macOS binaries
• scarb doc

Credits

This product includes modified portions of code of Cargo, developed by the Rust project contributors.

This product includes modified portions of code of hex_solver, developed by Six Colors AB.

scarb's Issues

Support `scarb metadata` in CairoLS

Create Scarb website

Pull corelib from Git

Scripts runner

Add [scripts] section to Scarb.toml that fulfills same role as NPM scripts. Use https://github.com/denoland/deno_task_shell and in general see how Deno implements this.

Perform downloads in parallel

Sign & notarize Windows and macOS binaries

Canonicalize Git URLs in scarb add

We have a logic for canonicalization of Git URLs, primarily for GitHub. It's done by CanonicalUrl struct.

scarb add currently pastes Git URL to the manifest verbatim. In order to encourage consistent style of everything, I think it'd be nice to canonicalize these URLs that users are passing.

Support silencing Ui with clap verbosity quiet

Problem

The main cli arguments

scarb/scarb/src/bin/scarb/main.rs

Line 17 in afbbedd

let args: Args = Args::parse();

accept verbosity flag

scarb/scarb/src/bin/scarb/args.rs

Line 24 in afbbedd

pub verbose: clap_verbosity_flag::Verbosity,

as defined by clap-verbosity-flag package. The flag is currently used for filtering tracing info.
I suggest silencing the Ui interface

scarb/scarb/src/ui/mod.rs

Line 36 in afbbedd

impl Ui {

when the verbosity flag is set to silence (is_silent).

Proposed Solution

No response

Notes

No response

Targets source isolation

Currently, if a package has multiple targets, all of them are in parallel compiled over the same source root. This is different from what Cargo does, and it comes from the difference in models of compilation units between Rust (each crate into separate unit and then linked) and Cairo (all crates creating a target go into single unit) worlds.

There are several disadvantages with this model though, and we might want to mimic Cargo's behaviour:

1. It's basically impossible to mix target kinds, because often they employ dedicated compiler plugins, specific modules and so. For example, it is not possible to write a [lib] which also has a [[target.starknet-contract]].
2. It may be impossible to isolate target-specific modules from general purpose library-ones. Leading to enforcing bad practices.

There are problems though:

1. Isolating targets would require creating distinct project configs, which is unknown how it will behave.
2. How should the directory layout look like? Rust-style (see how [[bin]] targets are laid out) have ambiguity with declaring submodules via mod x syntax.

Allow providing custom `core` version

1. Stabilize no-core field in the manifest, i.e. document it properly.
2. If the main package of a compilation unit has no-core = true, then make sure the compilation unit has no implicit core package in its components, and instead require the main package to provide custom core dependency.

scarb add: try to pull library to get its current version

...and write that in the manifest instead of user-provided version requirement

Targets mechanism

Provide a pluggable mechanism for specifying package targets:

Builtin:

• lib with a flag to allow outputting sierra and/or casm

Plug-in (but may be temporarily built-in as a hack):

• starknet-contract
• test

This should be similar to Cargo's targets, but some deviations are expected down the line.

Add option to configure target directory

Upstream proper fix for stdlib detection in Cairo compiler

starkware-libs/cairo#1595

Publish Scarb to crates.io

Initialize Git repository in scarb new/init

Problem

scarb new and scarb init currently create .gitignore file, but it would be also nice to let it automatically call git init for the user.

Proposed Solution

Add new function init_vcs in new ops file and run it just before write_vcs_ignore. This should be on-by-default behaviour, with opt out via --no-vcs CLI flag.

Notes

We do not want to bother yet with non-Git VCS systems, like Cargo for example is. Nobody uses anything else than Git in Cairo ecosystem as far as I am aware.

User interface

1. Implement user interface abstraction.
2. Use it for communicating downloads and builds etc.
3. Use console and indicatif libraries

Implement `add` command for manifest file manipulation

Take a look at how cargo add from cargo-edit work. Hint: use toml_edit crate.

Compile command should compile entire workspace

Verify releases in CI

1. Build artifacts and upload to release from single job.
2. Add a set of jobs which run scarb --version on as most diverse platforms as possible.

Implement `new` and `init` commands

Document usage of Scarb in readme

Implement proper support for workspaces

Allow virtual manifest files which have a [workspace] table. Allow Workspace structure to really contain multiple members. Functionally this should work similar to Cargo, but implementation-wise I think this can be done in a more clean way by perhaps defining separate struct for virtual manifest.

Move exit code handling to error types system, basically introduce `ErrorWithExitCode` struct using `thiserror` or smth like this. See `CargoResult` in Cargo. This is to be done in separate PR.

Move exit code handling to error types system, basically introduce ErrorWithExitCode struct using thiserror or smth like this. See CargoResult in Cargo. This is to be done in separate PR.

Originally posted by @mkaput in #66 (comment)

Lock registry and target directories when doing operations within them

Cargo does this (Waiting for lock...).

Scarb deps don't work when using Cairo test runner

Problem

Cairo test runner doesn't know how to pull in Scarb dependencies (building does work)

Steps

1. Repo reproduction is here: auditless/cairo-template#9.
2. Just need to uncomment the commented bits in Scarb config/code/tests if you want to run it).

Possible Solution(s)

Wait for Protostar?

Notes

No response

Version

scarb 0.1.0-alpha.2 (d5553ded6 2023-02-22)
cairo: 1.0.0-alpha.3

Add update informer

Problem

Add update information logic so that users will get notified about new project releases.

Proposed Solution

Perhaps https://github.com/mgrachev/update-informer

Notes

No response

Set up release pipeline

Document installation and getting started in readme

Fix tests ignored on Windows

Implement `fmt` command

Take a look for cargo fmt for CLI interface.

Keep mind to use exactly same version spec for formatter crate as we use for the compiler one! Rust happily compiles multiple versions of a single library into a single target.

`starknet-contract` target kind

Implement registry caching

Perhaps techniques used in dataloader could help here?

• https://docs.rs/dataloader/latest/dataloader/
• https://github.com/cksac/dataloader-rs/blob/master/src/non_cached.rs

Pass `PATH`, `*_LOG` and other vital envvars to subcommands

1. Write a test which ensures this behaviour (could be a unix-only test which runs a shell script which outputs these envs values)

Make `create-output-dir` crate not depend on `cargo-util`

Implement PubGrub version resolver algorithm

As title says

Implement `CorelibSource`

Implement `GitSource`

This should work identically to Cargo's one.

Note that recently there appeared an effort to integrate gitoxide in place of libgit2 in Cargo itself. https://github.com/Byron/gitoxide/blob/main/git-repository/examples/clone.rs

Try to use async for these operations. With gitoxide there's a change we might need to change the runtime to Tokio

Flock tests are flaky

Problem

Tests for file locking mechanisms are flaky, because they have fixed timeouts.

Steps

https://github.com/software-mansion/scarb/actions/runs/3987362765/jobs/6837077213

Possible Solution(s)

I think we should run Scarb via std Command and react to stdout lines on demand.

Test can timeout with ntest::timeout macro.

Notes

No response

Version

No response

Enforce UTF-8 in user-provided paths

Primarily enforce this in manifest and ops arguments and binary args parsing

https://github.com/camino-rs/camino

Extend package metadata with fields like `authors`, `description` etc.

1. Basically copy-pase these fields from Cargo
2. Don't forget to include these in metadata output

Remove trailing slash from path sources IDs Display representation

Compiling ci_testing v0.1.0 (D:\a\scarb\scarb\ci_testing\Scarb.toml\)

should look like this:

Compiling ci_testing v0.1.0 (D:\a\scarb\scarb\ci_testing\Scarb.toml)

this happens on all platforms

Hide `Summary` behind `Arc` like `Package` is to reduce clone costs

Plugin dependencies fundamentals

Allow packages to depend on Cairo compiler plugins, and use this to allow [lib] packages use Starknet plugin. Only Scarb-crate-level APIs to register custom plugins as for now.

Implement `list-commands` command

It should do the same thing the cargo --list does. Perhaps we should also replace commands command with --list argument?

Include information what Cairo version is bundled in `--version` output

Add scarb rm command

Like cargo rm for Scarb.

• Add RemoveDependency Op, just like Add one is done in manifest_editor
• Add rm command
• Try to preserve Cargo's behaviour as much as possible

Make Scarb independent of OpenSSL 1.1

Support namespaced package names

[dependencies]
"org/package" = "1.0.0"

On both cases, only package word as package name should be fed to the compiler. The org part is purely for registry purposes and the resolver will reject the following manifest with duplicated package conflict error:

[dependencies]
"argent/account" = "1.0.0"
"openzeppelin/account" = "6.0.0"

Reason: Support publishing forks to the registry. Suppose package company/X has a critical bug, but the maintainer is not responsive. Community can create a fix, push it to a fork and publish this it as community/X. Users will only have to change their manifests, instead of having to also rename all references to X in their codebases.

Note: I would rather avoid Javaesque/Apple domain-styled organization names (i.e. prefer starkware instead of co.starkware etc.) and in order to enforce this I plan to ban names like x.y.z

Update Cairo to 1.0.0-alpha.2