sygmaprotocol / sygma-fee-oracle Goto Github PK

• test feeOracle on testnet environment and make sure that all configuration is correctly configured
• make all necessary changes and updates (check contract configuration, env variables...)

Implementation details

Testing details

Acceptance Criteria

feeOracle can be used on testnet for GMP

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Versions

ChainBridge commit (or docker tag):
chainbridge-solidity version:
chainbridge-substrate version:
Go version:

Expand fee oracle API to support fee data requests where the destination chain is a substrate-based chain with deployed Sygma pallet.

Implementation details

• Details of fee data message format can be found in technical documentation (will be available after this PR is merged)
• API endpoint handler function should be expanded so it takes into consideration the destination domain (network) type when constructing a response
• Additional query property should be supported (length_fee) when the user requests a generic call fee estimate with the substrate-based destination chain.

Testing details

• Add unit test

Acceptance Criteria

• Fee Oracle should return appropriate fee data when requested for the destination chain being substrate-based

Add specific tags for each service (Etherscan, Coinmarketcap...) to logs when the query is successfull and it fails (not 200).

Implementation details

Add custom Str() field to the logs with the tag for service.

Testing details

Acceptance Criteria

Seems to be more correct to filter gas price by domainID instead of domain name which seems arbitrary and is not used anywhere else.

Will be a lot more crucial when we implement proper database support and currently we have to edit shared config names to match gas price domains instead of matching domain IDs which are always correct.

Implementation details

Testing details

Acceptance Criteria

The Swagger API documentation is currently not automatically updated upon new releases. This results in the link in the README file pointing to v1.0.0 of the API, which is hardcoded. Even if we change the URL to point to the latest version, the Swagger docs will still show v1.0.0 as it is the last released version of swagger docs.

Implementation details

To resolve this issue, we should automate the release of new versions of the Swagger docs using a GitHub action. This will ensure that the latest version of the documentation is always available and up to date.

Acceptance Criteria

• The link inside the README file should point to the latest version of the Swagger API documentation.
• A GitHub action should be created to release new Swagger docs versions automatically.

Related to #24. Avoid using domain names to match domains and assign gas price APIs per domain.
Also, remove custom polygonscan implementation as it is the same as Etherscan.

Currently, it limits future possibilites to resuse Etherscan implementations on networks outside of ethereum (there are many which have Etherscan supported) and it cannot match gas prices per domainID which will be important for proper database support.

Implementation details

Testing details

Acceptance Criteria

Currently, we need to have a hardcoded mapping for resources on testnets for which we want to query values from APIs like coinmarketcap. The reason behind this is that price APIs are being queried with the token symbols, so we need to remap testnet token symbols to something that actually has a price.

Suggested solution

The problem with this is that currently, we have a hardcoded remap inside code. My idea would be to add an additional work mode for fee oracle. Currently, fee oracle works in two modes:

• debug => local setup/e2e tests
  Always returns hardcoded response
• release => devnet/testnet/mainet

After refactoring, it would have three different modes:

• debug => local setup/e2e tests
  Always returns hardcoded response
• test => devnet/testnet
  Same as release, just on startup fee oracle can accept additional mapping of resourceID<>symbol that would allow us to remap test tokens to some actual mainet tokens for which we can fetch price data.
• release => mainet

Testing details

• Manually test different modes localy
• Add unit test for any new logic

Acceptance Criteria

• Removed hardcoded mapping for testnet tokens
• Working e2e tests
• Working devnet fee oracle
• Working testnet fee oracle

explorer link
https://moonbase.moonscan.io/

The fee oracle must make an RPC call to the substrate to fetch inclusion_fee when providing a fee estimate for a bridging request to the substrate-based chain. Support should be added in a general way so that fee oracle can call arbitrary substrate RPC calls.

Implementation details

Testing details

• Add unit tests

Acceptance Criteria

• Fee oracle is able to make substrate RPC calls
• Fee oracle can call Phala<>Sygma pallet to fetch inclusion_fee

explorer link
https://moonbase.moonscan.io/

Implementation details

Testing details

Acceptance Criteria

Move local config.yaml into env var

Implementation details

Read the data that is currently in config.yaml from env var
Make necessary changes in devops repo also

Testing details

Acceptance Criteria

config.yaml file is removed
configuration data is read from env var

Fee oracle needs to be refactored so it supports the new shared configuration.

Implementation details

• Load shared configuration from IPNS URL
• Restructure fee oracle codebase so it only uses one configuration (currently, it loads domains and resources separately)
• Extract address prefix information to in-code constant

Testing details

• Update all unit and e2e tests regarding made changes
• Add additional tests for loading shared configuration

Acceptance Criteria

• Passing unit and e2e tests
• Successful devnet shared configuration loading

Implementation details

Expand endpoint from: /{version}/rate/from/{fromDomainID}/to/{toDomainID}/resourceid/{resourceID}
to something like this:
/{version}/rate/from/{fromDomainID}/to/{toDomainID}/resourceid/{resourceID}/gasLimit/{msgGasLimit}

• Add this gas limit information to the signed fee data.
• Do some basic value checks on this parameter

Testing details

• Expand unit and e2e tests based on changes

Acceptance Criteria

• Passing unit/e2e tests
• Working endpoint

Currently, SSM params are being fetched by hardcoded value

Implementation details

• Add env Params for remoteParamDomainData and remoteParamResourceData
• hardcode this param into ECS task definition PR https://github.com/sygmaprotocol/sygma-fee-oracle/pull/11/files#diff-276fe3cf568afda7033fcc8889aff02c5d11d9977ae4c261572d3d491d2bee99R98
• Do not forget about the local setup of FeeOracle, for this we probably need to check REMOTE_PARAM_OPERATOR_ENABLE, and (probably) when it is set to false (that is used for local setup) these params should be fetched from local files (this fetching should be already done maybe you do not need to make even any changes)

Testing details

• check thet local setup works
• Unit tests
• e2e tests

Acceptance Criteria

• Code changed
• Ronny's PR with new task definition updaated with new params

Having fee oracle periodically fetch shared configuration would enable us not to have downtime on fee oracle service.

Implementation details

Testing details

Acceptance Criteria

Currently, fee oracle is using levelDB, but we should migrate to something that will scale better with the growing data set.

Implementation details

Ensure that communication toward DB is abstracted so we can always exchange it with another implementation.

Testing details

• Add units tests

Acceptance Criteria

• Fee oracle supports persisting data to MongoDB
• Passing unit and e2e tests

Implement Datadog APM with support for API error rate, latency, requests per second.

Implementation details

https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/go/?tab=containers

Testing details

Acceptance Criteria

• APM working
• API metrics sent to datadog

Update config.yaml

Implementation details

Update config.yaml so that is up to date with shared config(add additional config if necessary)
Update ecs in devops repo if necessary

Testing details

Acceptance Criteria

devnet is working without errors

Add PHA asset support in fee oracle.

Implementation details

As PHA is supported on the marketplace, this change should be just a matter of expanding configuration, so information on PHA token price is periodically fetched and persisted.

Testing details

• Manually test

Acceptance Criteria

• Fee oracle should be able to return the PHA price rate

Currently, fee oracle requires us to define all possible conversion pairs manually. This, paired with the format of this configuration property, makes it really hard to maintain this in any real production environment.

It would be ideal for removing this as a configuration property fully.

Implementation details

I see two approaches here:

To further facilitate these approaches, we can expand the shared configuration for each individual resource.
Add property feeType - "basic" / "oracle". This would mark all tokens that are supported by fee oracle

1. Currently, we are calculating rates for each defined pair and persisting it. We could parse shared config, extract tokens that are marked to use oracle, and then only fetch the value of each supported token (in $) and persist these values. Then, when someone requires specific pair, we just divide $ values to get the rate. This would significantly reduce complexity, as adding a new token to fee oracle requires us just to add one more query toward a service like coinmarketcap. The biggest con to this approach is that we are using $ price as a medium for calculating rate (but probably services like coinmarketcap are also using this under the hood).

2. Second approach would be to parse shared config, extract tokens that are marked to use oracle, and then query rates for all possible pairs between them. This approach removes the necessity for us to define all the pairs for fee oracle manually but still has this problem of rapidly increasing the number of queries as we start supporting new tokens.

cc: @freddyli7 @P1sar @mpetrun5, what do you think about this? I am leaning more toward the 1) option.

Testing details

TBD

Acceptance Criteria

TBD

Instead of fetching each param separate from environment use something like viper (relayer has custom logic because of chain configs) to do that to remove ifology when fetching params.

Implementation details

Testing details

Acceptance Criteria