nevermined-io / gateway Goto Github PK

Is your feature request related to a problem? Please describe.

The intention of this issue is to provide the mechanism to encrypt messages by the gateway, allowing later that only the gateway can decrypt them.

Describe the solution you'd like

A new endpoint /encrypt-rsa where the messages sent by the user are encrypted

Additional context

N/A

Is your feature request related to a problem? Please describe.

The intention of this issue is to provide the mechanism to encrypt messages by the gateway, allowing later that only the gateway can decrypt them.

Describe the solution you'd like

A new endpoint /encrypt-ecdsa where the messages sent by the user are encrypted

Additional context

N/A

Is your feature request related to a problem? Please describe.

Migrate ocean brizo

Describe the solution you'd like

A clear and concise description of what you want to happen.

Additional context

Add any other context or screenshots about the feature request here.

The gateway should expose a method allowing a user to upload the content of a file for further retrieval.

This functionality should be able to be enabled/disabled by configuration.

What

Fix how the gateway advertises the contracts on the root endpoint.

Why

Some components including the javascript sdk rely on that information to see if the network is operating correctly. Currently the contracts being displayed are hardcoded which makes it difficult to keep up to date when contract_libs_py changes.

How

In the gateway we should replace the hardcoded contracts and and instead fetch the list of deployed contracts from the keeper. That way every component using the same keeper will be in sync about what contracts are deployed and what are their addresses.

Describe the bug

The gateway is giving access when LockPayment condition is fulfilled. But because this could allow to give access to assets with a mistaken payment, before fulfilling any Access like condition, the gateway must:

1. Validate the LockPayment condition was fulfill (like until now)
2. Validate that LockPayment condition parameters (amounts + token address) used are the same than in the DDO

This needs to be resolved for any gateway flow (NFTs) before fulfilling the access condition

To Reproduce

Steps to reproduce the behavior:

1. Publish an asset with price > 0
2. Create a service agreement and lock 0 as payment
3. Get access through the gateway

Expected behavior

The gateway doesn't fulfill the access condition if the payment is not correct

Screenshots

Additional context

The gateway is showing the keeper url with the infura token. This is not recommended because a third-party could copy and re-use that token.

It would be recommended to trim the url and remove any token from there

Is your feature request related to a problem? Please describe.

It is need it to update the config to use the variable name nevermined-compute-url and use the environment variable NEVERMINED_COMPUTE_URL in the dockerfile to be consistent with the new names.

Is your feature request related to a problem? Please describe.

Running the gateway without the correct artifacts results in errors like:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/nevermined-gateway/nevermined_gateway/routes.py", line 384, in nft_transfer
    keeper = keeper_instance()
  File "/nevermined-gateway/nevermined_gateway/util.py", line 285, in keeper_instance
    return Keeper.get_instance(external_contracts=external_contracts)
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/keeper.py", line 161, in get_instance
    return Keeper(artifacts_path, contract_names, external_contracts)
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/keeper.py", line 104, in __init__
    self.did_registry = DIDRegistry.get_instance()
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/contract_base.py", line 46, in get_instance
    return cls(cls.CONTRACT_NAME, dependencies)
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/contract_base.py", line 33, in __init__
    self.contract = dependencies['ContractHandler'].get(contract_name)
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/contract_handler.py", line 32, in get
    return (ContractHandler._contracts.get(name) or ContractHandler._load(name))[0]
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/contract_handler.py", line 83, in _load
    contract_definition = ContractHandler.get_contract_dict_by_name(
  File "/usr/local/lib/python3.8/site-packages/contracts_lib_py/contract_handler.py", line 138, in get_contract_dict_by_name
    raise FileNotFoundError(
FileNotFoundError: Keeper contract DIDRegistry file not found in /usr/local/nevermined-contracts using network name mumbai
2022-05-18 15:46:23,191 - nevermined_gateway.myapp - ERROR - Exception on /api/v1/gateway/services/nft-transfer [POST]

Describe the solution you'd like

In my opinion it most times better if the process exits on some scenario like this. Reasons:

• Easier to identify failing services (with alerts to failing pods)
• Some spurious errors (like limited network connectivity) can cause that contracts cannot be allocated correctly (or other situations that could cause a incorrect state of the process), and restarting the process (as would happen if running on docker/kubernetes) would fix the issue.

What

Add a single source of truth for the gateway version that is less error prone on releases.

Why

Currently there is no single source of truth for the version of the gateway. Previously we were using bumpversion to bump the version numbers but that is not always followed. Lately we have been manually updating the version on setup.py and using the git-release script for releases. The problem here is that the gateway info endpoint is still reading the version number from bumpversion.cfg.

How

I think we should get rid of bumpversion since it's not compatible or overlaps with the git-release script and use another way of handling the version.

There are several ways of single-sourcing the version on python. I personally prefer option 3 .

We also need to update the release process on the README.

A data science or data engineer executing a compute job via Nevermined is gonna need to fetch the status about a job execution. This issue should provide a new gateway endpoint where the issues can ask for the status of a specific execution. If the execution resulted in the creation of new assets, this result should indicate the DID of the assed created too.

Download functionality needs to be more resilient and return meaningful error messages depending on different scenarios: 404, couldn't resolve url, error, connection refused, etc

when i do a version request against version 0.8.3 i get back 0.8.2. It is caused by this file not being updated:

https://github.com/nevermined-io/gateway/blob/v0.8.3/nevermined_gateway/version.py

A data science or data engineer executing a compute job via Nevermined is gonna need to fetch logs information about a job execution. This issue should provide a new gateway endpoint where the issues can ask for the logs of a specific execution.

Is your feature request related to a problem? Please describe.

The gateway checkPermissions and if you don’t, check if the agreement was fulfilled. If that’s the case it grants access. We reduce the flow to initialize and consume.

Describe the solution you'd like

A clear and concise description of what you want to happen.

Additional context

Implements: https://github.com/keyko-io/nevermined-internal/issues/23

Right now the gateway loads the contract informations (addresses, ABIs) from the file system. It has a reference to the contracts lib and the code is tested against this lib, therefore it should also take the contract information from the contracts lib instead of loading them from the file system somwhere else.

This lead to a problem on production where the gateway was run with a wrong version of the artifacts. Also it would make the installation less complex (no need for minio then anymore, nothing loaded sideways).

https://github.com/nevermined-io/gateway/runs/5510023133?check_suite_focus=true

Describe the bug

To validate the conditions the gateway is using the NVM token as a default value because the parameter with a specific token is not sent.

To Reproduce

Steps to reproduce the behavior:

1. Create an order with a token address different that NVM address
2. Try to consume the assets

Expected behavior

The gateway should validate that the order was created using a different token address than NVM and generate the conditions ids based on that.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

In the function validate_access on token.py

There is a call to
self.check_ddo(did, agreement_id, asset_id, consumer_address, keeper, cond_ids, ServiceTypes.ASSET_ACCESS)

That is calling to
(id1, id2, id3) = aservice.generate_agreement_condition_ids(agreement_id, asset_id, consumer_address, keeper)

And the parameter token_address is not sent. So the function is using token_address=None default, this drives to use always the nvm token because this is is checking that this parameter is None.

if token_address is None: token_address = keeper.token.address

Something changed recently with the estuary api and when downloading a file the gateway is actually returning only metadata about the file.

We need to fix the python filecoin drivers.

Currently the Filecoin upload method is using the metadata-driver-filecoin driver depending on a Powergate node.

Currently there are better integration mechanisms without that dependency allowing to publish files into Filecoin network via HTTP integration.

Two good candidates are :

• Web3.storage
• Estuary

More information here:
https://docs.filecoin.io/store/#apps