digitalcredentials / credential-status-manager-git Goto Github PK

Using git has served very useful in many use cases but we're finding that we're hitting some limitations. For instance, the git api may change without notice and there may be limits on requests.

This issue is to explore the next phase of how DCC will suggest management of credential status.

If I pass a badly formed VC to the allocateStatus method, it fails, but leaves the repos in a state such that when I try to restart, it gives me an error. I think the problem is that when allocating a status, the manager first updates the config, and in particular increments the number of credentials issued and saves this back to Github:

And then when it goes on to issue the credential, that fails, and so the credential never gets issued and the log doesn't get updated, which leaves the number of credentials issued in the config ('credentialsIssued') one greater than are in the log. Which is what later I think causes the error when restarting, specifically because of this check:

Fundamentally I think the problem is that the two operations (updateConfig and updateLog) aren't atomic and so the log and config can get out of whack. I could imagine this causing other problems too.

If the 'credentialsIssued` in the log is only used to know when the current list is full and so that a new list needs to be created, then maybe better to just calculate the # of credentials issued from the log?

Context: Currently, we use API tokens with global access to create credential status artifacts. This design has dangerous security implications, as a compromised token could lead to corruption/destruction of the entire organization's codebase.

Deliverable: Refine scope of API tokens.

I get this error when starting up the status manager, if my GitHub access token is missing, incorrect, or expired:

Error: The credential status repo ("status-test-three") and the credential status metadata repo ("status-test-meta-three") must be manually created in advance.
at createStatusManager (file:///Users/jameschartrand/Documents/github/dcc/status-service/node_modules/@digitalcredentials/credential-status-manager-git/dist/esm/credential-status-manager-index.js:61:15)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async initializeStatusManager (file:///Users/jameschartrand/Documents/github/dcc/status-service/src/status.js:16:31)
at async build (file:///Users/jameschartrand/Documents/github/dcc/status-service/src/app.js:10:5)
at async run (file:///Users/jameschartrand/Documents/github/dcc/status-service/server.js:11:15)

The repositories do exist, it is just the GitHub access token that is incorrect.

Could at some point maybe return a more specific error.

Context: Currently, this library can only create credential status artifacts within an organizational workspace. This design shirks DCC's mandate to provide simple and accessible tools that enable more issuers to get started as quickly as possible.

Deliverable: Add support for creating credential status artifacts within a personal workspace.

Reproduction: Run yarn test-karma.

Deliverable: Fix karma tests and remove FIXME comment in .github/workflows/main.yml.

Context: After refactoring the status library to be compatible with vanilla Javascript client code, we are encountering the following error with the node tests: TypeError: ES Modules cannot be stubbed. This error originates from the Sinon library and relates to the fact that ESM modules cannot be modified.

Reproduction: Run yarn test-node or simply yarn test.

Deliverable: Fix node tests and remove FIXME comment in .github/workflows/main.yml.

Context: allocateStatus first calls the GitHub api to get the current position, then increments the value locally, then calls the Github api again to set the updated value, and then returns the new position for use in a credential.

The problem then is:

1. Two (more or less) simultaneous calls (callA and callB) are made to the allocateStatus method (to allocate a position for two different credentials).
2. Both calls read the same status value from the metadata config in GitHub, e.g, 5
3. Both calls increment the value locally to 6.
4. Call A writes 6 back to the metadata config
5. Call B also writes 6 back to the metadata config

Now both original calls to allocateStatus return 6 as the new position. And so, the two issuers that made the calls both assign position 6 to their respective credentials. Which means two different credentials share the same status position, which they shouldn't.

This might seem an unlikely scenario, but it might not be as unlikely as we think because there could at times be a network lag between the call to the the current position and the call to set the updated position. Even if that is only a second, that is more than enough time for another thread/process to have read the 'old' status position.

We might need some kind of lock (e.g., semaphore) to prevent the problem.

If I delete all files in my two status repos, I'd expect the credential-status-manager to re-initialize the repos on restart, but it doesn't because of how the manager checks to see if the repo is empty: it looks for an empty result on a call to the octokit getContent method:

And, for whatever reason, after the repos have had files - even though they've been deleted - the getContent call doesn't return an empty result.

Could there be some other way to confirm the repos are empty, or at least empty enough that they can be used? So, for example, checking for the existence of the log.json and config.json files.

Alternatively, we could update the README to make clear to people that the repos have to be brand new and that simply deleting all contents isn't sufficient.

sdasdsdf

Extract the credential status manager client (for Gitlab and Github) from the sign-and-verify PR https://github.com/digitalcredentials/sign-and-verify/pull/55/files, into this standalone library.

• If possible, make this library isomorphic (or at very least, able to run from React Native, so we can use it from Learner Credential Wallet for self-issued credentials, in the future).
• Only support Status List 2021 for now (we don't need 2020 support)
• Use Typescript declarations for VCs and VPs from https://github.com/digitalcredentials/vc-data-model

Usage

import { createStatusManager } from '@digitalcredentials/status-list-manager-git'

const options = { /* access token, repo name, visibility, etc */ }
const statusListManager = createStatusManager({ service: 'gitlab', ...options })

// Ensure status list repo exists (this will be called on Issuer service startup)
// this will check if repo exists, if not, generate and write a blank config, etc, like it currently 
// does in src/app.ts in PR #55
// otherwise, will sync repo state
await statusListManager.initialize()

// Now you can use it to implement VC-API status-related endpoints
statusListManager.readStatusData()

statusListManager.embedCredentialStatus({ credential })

await statusListManager.createStatusData(statusCredentialData)

// etc