axodotdev / harbor Goto Github PK
View Code? Open in Web Editor NEW🦭 supply chain security GitHub integration for Rust, via cargo-vet
Home Page: https://harbor.axo.dev
🦭 supply chain security GitHub integration for Rust, via cargo-vet
Home Page: https://harbor.axo.dev
Right now we have one file that handles both get
and put
and we can divide it using https://nuxt.com/docs/guide/directory-structure/server#matching-http-method
When working on #33 I noticed that some baseline responsiveness is missing here. These are the two big things that should be fixed.
The text is being ingested with fixed line-breaks. We should find out why and not do that. We can either stop them where they are coming in and leave white-space-pre-wrap
for paragraph breaks, or we can find a way to break out the paragraphs and wrap each in a p
and then the internal line-breaks will be ignored.
This is how it currently looks. I would probably come up with something so that the sidebar is a separate layout that is navigated to, but one could also use something similar to the header list and let users pick an item from that. I don't have a real vision but it seems like we should do something.
I'm curious why we use safe-to-run
and safe-to-deploy
, which map to safe for dev
and safe for production
instead of straightforwardly saying the latter?
Also, in some cases couldn't safe for dev
be a higher standard because it can access your file system? I guess in Rust that's always assumed, so no?
Currently the UI frames the Solutions/Recommendations as if they are Goals (or the updates that actually happened to your dependencies). In reality they are just "the easiest fix" and may be fairly disjointed from the goal, especially with imported audit files.
The "failures" array in the Report (which we currently ignore) contains the "goals". I have some local experimental changes to merge that info in and tweak the UI.
For instance you may have a situation like:
And your audits/imports look like:
cargo-vet will likely end up recommending 1.0->2.0 as a way to achieve an audit for 3.0. The goal and the solution are quite disjointed (and there's basically no relation to what actually changed in your dependencies, which not even cargo-vet knows)!
(I dream of cargo-vet emitting graphs to explain this stuff but that's way out of scope).
Say a user has a TODO list of "audit A and B". They opt for an incremental approach, auditing A and hitting "Commit" before working on B.
They are now in a problematic situation as the UI currently works. They need to remember to manually "clear" A before submitting B, or we when they hit "Commit" it will attempt to add another audit for A to the ledger.
More problematically, they may just realize "oops I forgot to add a note!" and intentionally resubmit A with the new comment.
There are many possible solutions to this, in increasing levels of difficulty:
Github has a safety valve against workflows-triggering-workflows that extends to commits created by workflows. This is Good because it prevents infinite loops and infinite CPU waste.
However we specifically do want this, especially since cargo-vet-certify is only triggered by humans, so there's no loop.
It seems doing a git commit "properly" "with oauth" may bypass this safety valve: https://stackoverflow.com/questions/72421030/merge-by-github-action-does-not-trigger-ci-that-listen-by-branch
This might be a weird one, but I thought it would be worth discussing.
Rather than throwing an "audit failed" error when packages are unaudited, could be call it something like "audit outstanding"? I know that computer people are used to the first phrasing, but framing it as something that is fixable is a nicer vibe and I like the idea we could do nice vibes.
Is this totally silly?
PROBABLY NOT NEEDED FOR MVP
If cargo-vet says you only need safe-to-run but you decide to audit for safe-to-deploy anyway that's good and should be encouraged/allowed. Ultimately the UI will need two bits of state per criteria:
This also necessitates "showing" non-recommended criteria somehow (maybe collapsed? in a sidebar?).
The "shields" in the sidebar for whether you've done something could also potentially be more subtle:
If you try to submit with yellow shields there should probably be an "are you sure?" prompt.
Currently we use the generic github actions account which is ~fine but is bad branding and also may have issues (see #6).
The Sourcegraph button needs a hover state and needs to be ported to Fringe. Part of this is also considering what kinda of buttons we want: primary/seconday, success/error.
Don't really fully understand the implications of this but presumably this is better than "copy these files into your CI, hope you never have to update them!".
Various organizations will have standards for "well-formed commits" and not adhering to them can break CI.
Related to #4.
Based on discussion about the text newlines in #35 and meetings, we should interpret criteria text as markdown.
Include text to preface suggestions that makes the status clear. One way might be to insert between toggle and text either:
Toggle on to agree to text or Currently this package is marked as or something like that.
We don't need to implement these other platforms but we should make sure we understand how we would add them and don't over-hardcode github things. We should do an audit pass over the endpoints/schemas to make sure there's a reasonable path to add "gitlab" or "bitbucket" or "mozilla's phabricator" as an integration (if only so it's easier for folks to self-host this app with a bit of config).
Potentially relevant extension points off the top of my head:
https://phabricator.mozilla.com/cargo-vet-api
")https://github.com/axodotdev/cargo-vet-school/tree/main/.github/workflows
Now that I've learned github-script lets you do inline JS, we can just kill the certify
dir which exists to create a single JS workflow. Of course this would lock the current impl even more into github but it's also not exactly complex logic. The challenge is mostly getting github to not freak out about permissions and to trigger in the right situations, which is necessarily domain-specific.
There's also just a bunch of commented out gunk from experiments and bad formatting/style.
Not sure how much of this stuff should be "published actions".
Not sure how much should be abstracted into a new rust cli (or made into builtin functionality of cargo-vet!).
We need to work out the UX for when two people submit audits at the same time (or one person submits, possibly changes, then resubmits).
Another case that may scope-creep #4.
Right now on error in the frontend we just show There has been an error
and we should be showing the actual error so the user knows what is happening
API routes that need to be checked:
We need to differentiate between the commit form and the note form, and add affordances to show the note has been saved.
.
The UI components for this may be taken care of via Sunfish and then ported here through Fringe.
There's a lot of random crap I did in the PR that I need to go over and and cleanup now that I Understand what I want out of it.
Anywhere there's ${{ something }}
is a place where Bad Shit can happen. Some of them are Fine, others Aren't. Ideally I should extract more from the github
context object that gets injected into github JS workflows, so that strings are already strings and not something I need to inject into quotes!
Do an audit pass before shipping.
We need to work out exactly what to do in the cross-repo commit+push situation.
It's possible dealing with this will scope-creep some aspect of #4 into the mvp.
(Also going off of what @SaraVieira said, I haven't tested that this is busted myself.)
The current design has two background tasks:
cargo-vet-check-submit-failure task responds to cargo-vet-check failing, unpacks the zip, POSTS to the webapp, and posts a comment
cargo-vet-certify is triggered by the webapp when you hit the "Commit" button, runs cargo vet certify
in a loop, then commits and pushes
In both of these cases there can be Something Important And Useful Related To The PR happening and the the only way to know is to open the Actions admin panel on the repo (but not always the same repo!).
Potential improvements:
Probably lots of docs need to be written for lots of things?
I notice in https://gist.github.com/Gankra/d9ffe2cde001c6a615a3a9cf3928cddf, you mention the following filenames:
config.toml: your project-private configuration, similar to Cargo.toml
audits.toml: your audits and auditing-criteria, ready to be checked and shared
imports.lock: the audits you've imported from other organizations, similar to Cargo.lock
I think we'd benefit from naming these somewhat differently both for differentiation and for grouping, something like: vet.toml
vet-audits.toml
vet.lock
.
Right now it's not obvious what SourceGraph is, and its position in the top-right feels a bit de-emphasized. I'm not really sure what the Ideal design is, but it should probably more loudly scream "this is the button to do the review!!!".
For reference, the ostensible workflow of a user would be:
OUT OF SCOPE FOR THE MVP
I expect in the future we'll want this button (final submit button for the GUI):
To be a lot more like this button (github "merge PR" button):
in that it could support a few different options:
Currently the green being used for the commit button and for the toast are different — they should be the same to cutdown on the number of green variations on the screen.
Some errors:
cargo-vet has a mechanism where you can import audits.toml
files from other projects (e.g. "I peer with mozilla, so I import their audits.toml").
cargo-vet was originally designed on the premise that humans would be running it on their local machines without --locked
so we can opportunistically update imports. A mechanism was also added to minimize the churn for this: the imports.lock file only contains required information. We GC anything that wasn't (seemingly) necessary to get things audited.
CI would then run --locked
which only used the imports.lock to avoid spurious breakage.
However this limits the capacity for ci/bots to know about additional useful information. In the simplest case there may simply be a new import that covers a new requirement. More subtly it may just make a requirement simpler (smaller diff). Even more subtly an import may introduce a violation
and we won't know, and may even suggest audits that overlap with that violation!
Also the bot is actually better poised to update imports, as it's more trustworthy than "some human pushed a commit with imports.lock updated, trust me it's legit". Clearly the bot should have a mechanism for pulling in imports and updating them.
Before the imports GC thing I had suggested it periodically opening dependabot-style PRs, but now I think it would have to just surreptitiously insert them as part of other ops. Although maybe a cron job for checking violations is merited.
No idea what this entails yet, but probably involves #17!?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.