Delivering plugins should come with a proper demo to show off the plugin SDK features. The suggestion is to make a plugin to expose the Qiskit simulator, to replicate at least the first two simulator tutorials at https://qiskit.github.io/qiskit-aer/tutorials/1_aersimulator.html in Aqueduct.

Decide with @tomhartley and @samiralavi what it should look like.

1. • A class which represents and runs a plugin
2. • A class which enumerates plugins
3. • A GraphQL endpoint to get list of plugins
4. • A GraphQL endpoint to get plugin info (maybe merged with (3))
5. • A GraphQL endpoint to launch a plugin with provided parameters

When/if Airflow is supported in Aqueduct, plugins should work transparently as Airflow nodes.

Define and implement how a separate plugin can/should be delivered to a installation of the aqueduct.
Take into account:

1. Probably, installation is a docker container.
2. Uploading via UI will require to create additional frontend screen.
3. Uploading via pyaqueduct is simpler.

Add some query and mutations mocked in the Frontend to them ready for Unit tests.

• query: plugins
• mutation: executePlugin

Now we have the requirements captured for experiments data import/export feature, the functional specification needs to be documented as a guideline for the development of this feature.

Hi, would you be willing to add a changelog file describing changes between releases? I noticed that new features have been added to the development version, however to find what features have been added I need to look through the PR/commit history. Would you consider adding a changelog?

Regards,
Jake

Extensions list button in the experiment page and the function of opening the modal.

Describe the bug
It's a request to modify API fields for better simplicity and common sense alignment.

Current API
1- The unique identifier which is referred to by the name id is better to be changed to uuid
2- The experiment id which is referred to by the name alias is better to be changed to eid
3- Also, it's good to separate the favourite and archive functionality from being a special tag and being considered as a field in the Database which improves query efficiency and API usage.

Additional context
I also see that there will be breaking changes with Aqueduct-core and pyAqueduct.

I want Aqueduct being able to execute plugin-like code. This includes:

• having infrastructure (GraphQL endpoints) to store and list plugins
• having code to execute a particular plugin function (also, GraphQL endpoint).
• having simple examples to implement my own plugins.

As a user, I want to get back to the previous state when I opened any experiment. but if I change filters in the experiments list the pagination should be reset.

Implement different types of input while filling in the parameters including

• int
• str
• textarea
• float
• experiment
• file
• bool
• select

there is a screen which takes plugin list from the endpoint and represents them together with their functions. Probably this will be a 2-level tree-like structure: each plugin has a list of functions it implements.

Implement the Modal component which is responsible for the extension's input, including

• Run button
• Left-side function list (Card)
• This ticket task also relies on #108 to have different extensions' input components

Aqueduct branding such as logo and style is not yet reflected on the documentation and readme file. Also, some important CI statuses such as tests and coverage should be added to the main readme file. This is also the case for PyAqueduct (AqueductHub/pyaqueduct#21).

Exported Artifact

The exported artifact from Aqueduct will have two main data items:

Experiments metadata: This is mainly database records and their relations alongside extra information bits about the version of Aqueduct, the user that has created them, etc.

Experiments' files: These are the experiment files uploaded by the users or generated by the plugins.

The exported artifact is going to be a single tar.gz* file that has the following directory structure:

artifact
|   metadata.json
|
----experiment_files
|
--------[UUID of experiment]
|       |   ...

• Python supports most of the main compression methods through different libraries. The trade-off is mainly between compression speed versus compressed size. I think gunzip (tar.gz) has a good balance, which is why it is pretty common, but this is open to discussion. More here: All The Ways to Compress and Archive Files in Python

Exported Experiments metadata Format

Based on the requirements of exporting data into a text document, the usual options are JSON and YAML. JSON is a better solution for data transfer as it can be minimised by removing white spaces and there is more support for parsing JSON rather than YAML, for example, Pydantic is based on JSON. Furthermore, if size was a problem. JSON can be converted to CBOR, which is an IETF standard for binary representation of JSON documents.

The JSON schema is version-less on its own, but it can include metadata to show the version and other information about the Aqueduct instance it was exported from.

• RELEASE on Mondays every 2 weeks at the start of sprint with a patch version increment with the option to pause this.
• Automated release notes to the repo and documentation

Once the import/export functionality is developed, its use should be documented.

Come up with a documentation page with screenshots which guides through plugin execution. For example, we may use qiskit simulation plugin, or wolfram alpha plugin

To provide new text/x-markdown as content-type in response of .md, .mdown, ... files.
Ref: https://mimetype.io/text/x-markdown

Backend changes for the removal of experiment is implemented and can be done using GraphQL mutation of removeExperiment. We need to have some frontend functionality like a button on experiment details page or some other UI component.

DoD:

Interacting with the experiment removal component results in deletion of experiment from the database and deletion of all the files related to that experiment.

1. Should we restrict the executing user in the final version?

2. Should we isolate execution environment?

• Non-isolated environment allow users be sure that whatever works on their server machine is available to plugin context. If we isolate the environment, we put additional burden on developers to take care of dependencies separately.

3. Which libraries/binaries should we make available for plugin developers? And how?

• If we do not isolate, we may define dependencies as dependencies of Aqueductcore – plugins are executed in its context. If we isolate, probably some kind of dockerfile should be given to users.

Describe the bug
Some links in README.md are not taking you to a correct place.

Steps to Reproduce

1. Click through all of them.
2. Update the right destination.

addTagToExperiment mutation only accepts a single tag as an argument. If the API users want to add multiple tags in an atomic way, it is not possible. (updateExperiment mutation can be used but it reassigns the tags, not addition). A new or updated mutation is needed to add a list of tags to experiments in a transactional way.

This issue is related to this use case in PyAqueduct: AqueductHub/pyaqueduct#12

Once the development is finished, the feature is merged into main, and its documentation is complete, a new release should be created for the users to be able to access this feature

Currently I've added an example of populating venv by hand. But we may natively support this in PluginExecutor class as a preliminary step.

As a user, I want to be able to sort my files based on the table header shown at the top of the table.

The default sort should be newest first.

Is your feature request related to a problem? Please describe.
I wanted to be able to preview Markdown files in the GUI

Describe the solution you'd like
To add a new feature which enables .md files to be shown in the file explorer inside experiment details.

Additional context
It also should understand relative links to the files living next to the markdown file in the same experiment.

Plugin execution may take time. To track plugin execution and be able to address plugin execution history, we should be able to execute plugins as jobs. This includes:

1. Running them asynchronously in a job pool.
2. Have job IDs.
3. Storing job IDs in a database.
4. Storing job statuses in a database (?).
5. Storing job execution result in a database.
6. Exposing job listing and job info access.

Pushes to main deploy the aqueductcore-dev docker image to dockerhub: https://hub.docker.com/r/aqueducthub/aqueductcore-dev . This should be added to the README.md to tell users that this exists. Probably some disclaimer that this may be unstable.

Interface for Importing/Exporting Data

Import and Export functionality runs by the admin user through the CLI interface of Aqueductcore. Depending on the installation of Aqueductcore (docker container, python package, source code, etc…), the user first needs to establish a terminal to the environment where Aqueductcore is installed. The environment should already have the required prerequisites and variables. Then the specified command can be executed like this:

> aqueductcore export [file path] -e/--experiment-files
> aqueductcore import[file path]

The -e/--experiment-files argument is a flag that determines if experiment files should be included or not. By default, if the flag is not provided, it has false value and experiment files won’t be exported.

The decision of partial (without experiments' files) or full (with experiments' files) export/import is left to the user. Partial export is useful when the file server is a NAS (network-attached storage) and it will be used for the new instance.

Frequently, we might want to review the software dependencies and their licenses used in Aqueduct Core. An automated Github action that gets triggered manually is helpful to achieve this. The exported file should be in readable text format.

As a user, I want to be able to delete my archived experiments from the list which also asks for confirmation. and the button only appears when the experiment is already archived.

Implement plugin functions of QEC-X which are exposed to QEC-X.
This task should carefully consider authentication and data processing. Having local Aqueduct installation and cloud QEC-X installation:

1. QEC-X will need a separate authentication mechanism.
2. Aqueduct should be accessible from the outer network.

Partial solution is DataString, but this still require QEC-X authentication.

Exported Artifact

The exported artifact from Aqueduct will have two main data items:

Experiments metadata: This is mainly database records and their relations alongside extra information bits about the version of Aqueduct, the user that has created them, etc.

Experiments' files: These are the experiment files uploaded by the users or generated by the plugins.

The exported artifact is going to be a single tar.gz* file that has the following directory structure:

artifact
|   metadata.json
|
----experiment_files
|
--------[UUID of experiment]
|       |   ...

• Python supports most of the main compression methods through different libraries. The trade-off is mainly between compression speed versus compressed size. I think gunzip (tar.gz) has a good balance, which is why it is pretty common, but this is open to discussion. More here: All The Ways to Compress and Archive Files in Python

Exported Experiments metadata Format

Based on the requirements of exporting data into a text document, the usual options are JSON and YAML. JSON is a better solution for data transfer as it can be minimised by removing white spaces and there is more support for parsing JSON rather than YAML, for example, Pydantic is based on JSON. Furthermore, if size was a problem. JSON can be converted to CBOR, which is an IETF standard for binary representation of JSON documents.

The JSON schema is version-less on its own, but it can include metadata to show the version and other information about the Aqueduct instance it was exported from.

1. Support default values in variable definition
2. Support bool type (checkbox)
3. multiline -> textarea
4. Dropdown strings (select -> option?)
5. Save log into an experiment; return a file name
6. Expose a single experiment for a function
7. implement display_name field for parameters

After collection of the parameters and building an interface we have to pass these values to a server endpoint and expect it to return results. Results will consist of generic int (result code) and text fields (result code, std-err, std-out).

Hi,

For larger experiments that generate a lot of data, it would be useful to group related files (e.g. plots) in subdirectories for easier browsing of the data. Currently, I have to look through a few dozens of files to find the correct one.

Is this something you are planning to support?

Cheers,

Jerome

The github actions that are supposed to run for tests and PRs are not run when the branch is different than feat-*. As the guideline suggests using feat/* for branch naming, the workflows have to be updated to understand this pattern instead.

The timestamp of the experiments when they are stored in the database is localized, which could be problematic for certain use cases that require showing the experiments in different time zones. This can be fixed by saving the UTC datetimes in the database and only convert them to the local time zone when displaying them.

Definition of Done:

• Stored datetime in database in UTC
• Display time on the frontend in local time zone

As a user, I want to be able to use characters like “-”, “:” and “/” in the tag’s name.

The documentation of Aqueduct can not be previewed in a PR that has changed the docs because there is no staging environment or workflow for that. An idea taken from FastAPI repository is to use Cloudflare Pages service which is a free service to host staging static website. Further information about how it works is here:

https://github.com/cloudflare/pages-action
https://github.com/tiangolo/fastapi/blob/master/.github/workflows/deploy-docs.yml

The screen is generated according to endpoint declaration and builds a form to populate with values.
To start these might be:

• Experiment dropdown (with search) -- PROBABLY if we don't add this to experiment UI
• File dropdown inside experiment
• string, multiline string, float, int.

As a user, I want to have access to the documentation of the Aqueduct and it’s a new item in the left-side bar which should have an external link icon to indicate it’s an eternal link.