api3dao / airseeker-v1 Goto Github PK

API3 Market team requires a testing/dev beaconSet. Here's the slack discussion about this: https://api3workspace.slack.com/archives/C037NLX1R8T/p1663330087379479

Tasks to be performed:

1. pick 3 random beacons from amberdata (let's say eth/usd, btc/usd and matic/usd)
2. create a beaconSet out of these 3
3. modify the current deployed airseeker.json file locally
4. run airseeker locally
5. deploy airseeker to aws with the new changes

It'd better to mention about flow of updating beacon using Airseeker:

1. Get transactionCount (RPC call)
2. Read on-chain data (RPC call)
3. Check update conditions
4. If 3., Get gasPrice (RPC call)
5. Submit tx (RPC call)

For situations where the update condition isn’t satisfied (most frequent), we call RPC 2 times, if we can change order to 2-3-1-4-5, we’ll call RPC one time in that situation.

This improvement doesn't look essential when only one sponsorWallet is used for beacons, because transaction count fetcher is invoked one times. But the situation where each beacon has distinct sponsorWallet, it effectively reduces the total number of RPC calls.

But still executing on serverless services.

Probably using Airnode's TF files as a template (so we can easily support GCP and AWS).

Both updateBeacons and updateBeaconSets functions currently call initializeUpdateCycle with almost identical parameters, except for dataFeedType. However, dataFeedType is only used in logOptions, which is not crucial and can be moved into the respective functions. This implementation has a disadvantage. For instance, if we have 8000 sponsors, each with only one beacon (as is the typical case), we would fetch 8000 transaction counts in updateBeacons and another 8000 transaction counts in updateBeaconSets unnecessarily, even though we don't have any beacon sets. To address this issue, I suggest the following modification:

1. Remove initializeUpdateCycle from both updateBeacons and updateBeaconSets functions.
2. Remove parameter dataFeedType for initializeUpdateCycle which is used only for logOptions
3. Move it just before where updateBeacons and updateBeaconSets functions are called, here
4. Extend logOptions in updateBeacons and updateBeaconSets functions by adding dataFeedType

For context:

https://api3workspace.slack.com/archives/C037NLX1R8T/p1653813343064269

Strategies will be applied in the order they are available in the configuration file.

We want to support 3 strategies, latestBlockPercentileGasPrice, providerRecommendedGasPrice, and constantGasPrice all of which are already implemented in gas price oracle. What is needed is to move them into independent building blocks so they can be used in any order.

The fee type will be deduced based on the gas price strategy.

#183

Relates to api3dao/operations-deprecated#136

This section describes what beacons (array elements are beacon IDs) belong to a specific beacon set (the object keys are beacon set IDs):

"beaconSets": {
  "0x924b5d4cb3ec6366ae4302a1ca6aec035594ea3ea48a102d160b50b0c43ebfb5": [
    "0x924b5d4cb3ec6366ae4302a1ca6aec035594ea3ea48a102d160b50b0c43ebfb5",
    "0x924b5d4cb3ec6366ae4302a1ca6aec035594ea3ea48a102d160b50b0c43ebfb6"
  ]
},

In the validation, we need to make sure that every beacon listed has its counterpart in the beacons section.

Currently the Airseeker updates each Beacon with an individual transaction. This is inefficient because the 21,000 base transaction gas cost is being paid for each update. We should instead batch these updates in a single transaction. This includes the development of a proxy Multicall contract that doesn’t revert if an individual update fails. See the thread for more information

https://api3workspace.slack.com/archives/C037NLX1R8T/p1653479838267289

Currently Airseeker calls a remote HTTP signed data gateway in order to retrieve signed API responses.

We'd like Airseeker to have the ability to call an API directly and sign the result with the existing Airseeker mnemonic.

This will practically create a hybrid between Airseeker and Airkeeper in terms of functionality.

This should be implemented as a method in the trigger configuration, for instance:

  "triggers": {
    "dataFeedUpdates": {
      "1": {
        "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC": {
          "beacons": [
            {
              "beaconId": "0xbc80cbd7a8b8180e11d189a8334814a44a69c4d083b31305ecf67a3a3ea0fd9a",
              "deviationThreshold": 0.1,
              "heartbeatInterval": 86400
              "method": "direct"
            },

method should be capable of supporting other arbitrary retrievers through a common interface, eg.

// something like...
interface DirectRequest {
  template: Id<Template>;
}

interface GatewayRequest extends DirectRequest {
  gateways: Gateway[];
}

interface Request {
  (request: DirectRequest | GatewayRequest): Promise<SignedData>
}

export const makeSignedDataGatewayRequests = async (
  gateways: Gateway[],
  template: Id<Template>
): Promise<SignedData> => {};

Methods could be:

• direct - call the API directly
• v0.6.5 - call the remote gateway using an adapter for v0.6.5 Airnode
• v0.9.0 - etc.

The above will require Airseeker to have access to the OIS. This could be included at the top level of airseeker.json as an array of OISes:

{
  "triggers": {},
  "templates": {},
  "oises": [
    {
      "oisFormat": "1.1.1",
      "title": "<FILL_*>",
      "version": "<FILL_*>"
    }
  ]
}

Airseeker should search the available OISes for the first OIS that contains a matching endpoint and use the associated details to execute the direct call. The API response data should be signed with the Airseeker's mnemonic.

This will also require Airseeker to include an apiCredentials field in the config file.
Mnemonic derivation should be done up-front (like Airnode's deployer) to reduce duplication of processor usage.

Nodary largely operates single-source feeds.
The dAPIs team needs to run multi-source/aggregated feeds, this usually doesn't involve only updating single-source feeds.

The addition of this line in a recent commit causes Airseeker to exit early:
https://github.com/api3dao/airseeker/blob/7f0bd378e30062527bdb78afcf85707ae739c6b0/src/main.ts#L33

with the error "No beacons to fetch data for found. Stopping."

I noticed that those were not implemented when #163 and #164 were being done.

Current implementation has two fetchMethod, api and gateway. When gateway is selected, the requester function tries to parse response with legacy schema as in here. If it fails, then it tries with the current schema. Adding an extra fetchMethod like gateway-old, we can distinctly select the schema going to be utilized for parsing.

Currently, when filtering empty wallets, Airseeker attempts to retrieve balances using all providers simultaneously using Promise.any approach. This means that the response from the first successful RPC call is accepted, and the filtering is performed based on that response. However, in certain situations where there are malicious or out-of-sync RPC providers, they may incorrectly report non-empty wallets as empty. This can cause Airseeker to become blocked.

To address this issue, instead of using Promise.any, the responses from RPC providers can be logically combined using an OR operation. By doing this, Airseeker can avoid the problem of relying on a single potentially unreliable provider. It's important to note that the opposite scenario (reporting empty wallets as non-empty) is also possible, but it will result in behavior similar to Airseeker without the filteringEmptyWallet feature, which is not problematic.

Blocked by https://github.com/api3dao/airseeker/issues/447 and https://github.com/api3dao/airseeker/issues/446

With self-funded Beacons, we also want to omit fulfillmentGasLimit (same reasons as https://github.com/api3dao/airseeker/issues/447), but we don't want to call estimateGas() for each update because it's wasteful (they will all return the same value). As a solution, in cases that fulfillmentGasLimit is not defined and the transaction to be made is a single Beacon update, we can use the estimateGas() result we got from https://github.com/api3dao/airseeker/issues/446

Refer to Slack thread here.

Currently, the DapiServer contract uses averaging to determine the output timestamp across a set of input beacon values when updating a beacon set. The problem with this approach is that a malicious Airnode could misreport the timestamp of their data, (eg. they could double the timestamp value). This would greatly increase the average timestamp value across the beaconset, causing the isValidTimestamp check to reject the value and the entire beacon set failing to update.

An interim solution is to do a check within Airseeker to confirm that the timestamp values are sane. A longer-term solution is to make the DapIServer using median instead of averaging for calculating the output timestamp.

Airseeker's config file had triggers for beacon sets defined and no triggers to the beacons inside those beaconsets. The result was a bunch of failed beaconset update transactions. This seems problematic, but perhaps the product of Airseeker being misconfigured 😅

https://mumbai.polygonscan.com/tx/0x8e09525f6025897bc3bbb73503edc457e87dc9d302f93d90893263027aa46c38

When you run create-airseeker-config it sets txTypes to be eip1559. It should use legacy instead, see the config files belonging to previous Airseeker deployments.

See the thread: https://api3workspace.slack.com/archives/C043YH02PGQ/p1681511459412509

This isn't a complete solution for the problem described in slack thread but it'll help to alleviate occurrence probability for the problem.

I'd say that the implementation will be different enough to be done separately from update-beacons.ts. I would prefer starting with a separate implementation in update-beacon-sets.ts and then refactor both and reuse common parts.

The flow of beacon set update:

• retrieve values for each beacon within the set from the cache (common memory)
• IF there's no value for a given beacon, fetch it from the chain
• IF the value is not available on the chain skip the update
• fetch beacon set value & timestamp from the chain
• calculate beacon set timestamp from beacon timestamps (https://github.com/api3dao/airnode-protocol-v1/blob/main/contracts/dapis/DapiServer.sol#L443)
• IF new timestamp is older than the on-chain one skip the update
• IF the last update is older than now + heartbeat interval force update
• IF the deviation threshold is reached do the update, skip otherwise

Use the readDataFeedWithId function to rad the data (both beacon and beacon sets) and updateBeaconSetWithSignedData function to update beacon sets.

Airseeker checks the sponsor wallets specified in its config file at the start and discards the ones that are not funded api3dao/airseeker#309 This is especially useful for self-funded, Nodary-style usage where an Airseeker has hundreds and thousands of sponsor wallets.

This can be further improved by multi-calling the balanceOf() calls to the sponsor wallets, resulting in fewer RPC calls. Api3ServerV1 inherits ExtendedSelfMulticall, so the multicall can be done to that contract.

I presume this will be similar to the triggers.beaconUpdates section, grouped based on the chain and sponsor. Deviation threshold and heartbeat interval will be beacon set bound. The list of beacons included in the set is available in the beaconSets field. I would imagine something like:

"beaconSetUpdates": {
  "1": {
    "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC": {
      "beaconSets": [
        {
          "beaconSetId": "0x924b5d4cb3ec6366ae4302a1ca6aec035594ea3ea48a102d160b50b0c43ebfb5",
          "deviationThreshold": 0.1,
          "heartbeatInterval": 86400
        }
      ],
      "updateInterval": 30
    }
  },
  "3": {
    "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC": {
      "beaconSets": [
        {
          "beaconSetId": "0x924b5d4cb3ec6366ae4302a1ca6aec035594ea3ea48a102d160b50b0c43ebfb5",
          "deviationThreshold": 0.2,
          "heartbeatInterval": 86400
        }
      ],
      "updateInterval": 30
    }
  }
}

In the validation, we need to make sure that the beacon set present in this section has its counterpart in the beaconSets section.

There are stale PRs from dependabot updating Airnode (and other) dependencies that are failing. They require some manual changes.

Airnode v0.12 will make fulfillmentGasLimit in chains an optional fields. Airnode needs to jump through a lot of hoops to support that. Airseeker just needs to omit the gasLimit field in the transaction, which will tell ethers to make an estimateGas() call beforehand. (see below) This should be useful for (multicall-based) Beacon set updates, which will differ in the required gas limit.

This also makes chain integrations easier and safer, as the required gas limit floats in some rollupsL2s, making hardcoding a gas limit a risky strategy.

Currently, sponsor wallet balances are fetched at the beginning of the coordinator, and the sponsor wallets that have exactly zero balance are filtered out to reduce the Airseeker load. Self-funded sponsor wallets update until the funds run out, but the funds leave some dust instead of running out completely, and the filtering strategy isn't suited to address this.

Proposed method:

• At the start of the coordinator, generate a random wallet (to use as airnode), and call estimateGas() with it to estimate the gas of initializing a Beacon
• As the second step, fetch the gas price as described by the gas price strategy
• Filter the sponsor wallets whose balances are lower than the multiplication of the above two values

Since we're not running Airseeker as a long-running service, there are parts that can be removed:

• Terraform recipes
• pm2 setup
• Docker containerization
• some helper scripts

We can (and probably should) remove these to keep the repository clean and less confusing at first sight.
@aquarat @bbenligiray What do you think about this? I'd say we should remove what we can, we can always dig it back up from the git history if needed 🤷‍♂️

We have added protocol indices as part of v0.8 Airnode release. We should use these constants inside Airseeker.

api3dao/airnode#1195

We’ll need a way of handling the secrets. We’d probably want to trigger a build from Github but not from within Github.

Once #166 is done.

Airseeker calls APIs, Airnode gateways and blockchain providers to read things. It does these in bursts, which creates instantaneous load. If these calls are staggered, it would help. However, these two future features need to be considered: api3dao/airseeker#256 api3dao/airseeker#216

A socket used inside a go-promise with a timeout >= to the go-promise's timeout will result in a dangling file descriptor.
This ultimately results in file descriptor exhaustion, which DoS's Airseeker.

This specifically impacts ethers Provider instances and axios request calls.

Currently, node version is specified as 18 in package.json however 14 in serverless.aws.yml. It can cause unexpected behavior between development (node18) and production deployment (node14).

DapiServer signed data update interface changed a bit. Airseeker should be updated accordingly. There are two ways of doing this

• Do a breaking change
• Add a new trigger type that uses the new interface for backward/forward compatibility

Currently, we wait for one more "cycle" after the stop signal is received. This is not needed, it prolongs tests and is in general annoying. We can remove it and (presumably) shorten some E2E tests.

Remote gateways do sometime die, but they also often arbitrarily fail but come back to life.

This issue calls for a tally to be kept in Airseeker monitoring which counts the number of failures on each gateway. If a gateway exceeds 5x failures, raise an alert.

Everytime a gateway call is successful, reset the tally to 0.
If the tally was non-zero to begin with, close any existing alerts.

Work on this issue should be placed in a branch that branches off the branch of PR #432 (assuming it isn't already merged).

This adds an unnecessary gas cost overhead for individual Beacon updates, which is quite important for self-funded feeds

https://github.com/api3dao/airseeker/blob/main/src/constants.ts#L3

Even though we have E2E tests we from time to time need to test new Airseeker features so we need to set up an environment (prepare blockchain, run Airnode(s), etc.) to do it. It would be nice to have this at least semi-automated/scripted.

Currently, Airnode signed gateway URL changes require Airseekers to be redeployed. Would it be possible to fetch gateways from the database or through an endpoint every cycle?

Refer to #113

Change Airseeker's gas strategy to what Nodary uses currently.

Update: Airseeker should bump dependency once this PR is released with Airnode 0.12

Update: Airnode v0.12 is released so this is not blocked anymore.

When you run create-airseeker-config it asks

How many RPC provider (per chain) you want to use in the configuration?

If you hit enter, the providers are empty. Since there is no point in having a chain without any providers for an Airseeker it should ask this instead

How many RPC provider (per chain) you want to use in the configuration? (default: 3)

In the case that a sponsor wallet is not funded, Airseeker still goes through the entire flow until it attempts to make a transaction, which will fail. This is very wasteful for a bring-your-own-gas kind of scenario, where most sponsor wallets will be unfunded and you don't want to make unnecessary API calls for them. The suggested solution:

Keep a isFunded flag for each sponsor wallet. When Airseeker encounters a "trigger" for a sponsor wallet, it first checks if it has already checked if it is funded. If it has and it was funded, continue. If it has and it wasn't funded, skip. If it hasn't, check if the sponsor wallet is funded, set the flag and proceed accordingly.

@bdrhn9 has implemented a solution that checks all sponsor wallets at the start. However, the suggested solution would also work well with staggering.

Note that this feature can/should be adapted to RRP in the single-Lambda Airnode.

Similar to api3dao/airseeker#216

An Airseeker that runs thousands of Beacons needs to make a lot of RPC calls to read these individually. It would be better to read these in batches through Multicall.

Currently, airseeker.feature.ts is importing the main() function to run the Airseeker for each test.

This change should be that Airseeker is started as a pm2 service and there is a single sleep() period for each test to wait for one Airseeker cycle to finish after the contracts are re-deployed. (We can experiment with how long the sleep should be, probably 10-15 seconds)

When you run create-airseeker-config it asks

How many RPC provider (per chain) you want to use in the configuration?

If you hit enter, the providers are empty. Since there is no point in having a chain without any providers for an Airseeker it should ask this instead

How many RPC provider (per chain) you want to use in the configuration? (default: 3)

Relates to #233

See transaction 1 and transaction 2.

https://api3workspace.slack.com/archives/C040G0W7RH9/p1692196350796399

Currently we only show datafeedId in the alert description. It'd be ideal if we could also show the dapi name (when available).

When I run E2E tests multiple times they sometimes pass and sometimes fail. This needs to be investigated and fixed.