example normal

endpoint = 'https://ocean.defichain.com/v0/mainnet/stats'

feature = 'data__count__blocks'

nested

endpoint = 'https://ocean.defichain.com/v0/mainnet/poolpairs'

rule = 'data__[displaySymbol]'

feature = 'data__[displaySymbol=dBTC-DFI]__tokenA__reserve

• more flexibility, can serve 1min interval

live data: postgres, redis
backup: blob
redundance on data fetch with redis, if postgres is down

cake rebalance adress
https://defiscan.live/address/dZejceUMav2dhUFXfvzEuXJZMHjwpVeEmi

How and how fast?
-> provide api endpoint that gives adress history of all the tokens

day, dfi, btc, ...

DELETE FROM data_minute
WHERE datetime < NOW() - INTERVAL '10000 hour';

search, filter, click history or value link, click visualisation

python, excel, google-docs, ...?

i have burn total, but also want burned per hour, day... to compare it with emission

in jsonb value: "1.000" value: 1 should also work

https://api.bittrex.com/v3/markets/tickers

needs usage stats

daily active addresses, daily transaction count, daily transaction volume in USD and/or native units.

limit to 10 features

[SQL: insert into data_day (endpoint, feature, datetime, modified, height, count, mean, std, min, q25, median, q75, max, open, close) values('https://api.dfx.swiss/v1/statistic', 'staking__yield__apr', '2023-01-26 00:00:00', '2023-02-02 08:41:36.269925+00:00', Null, 0.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,nan,nan) ON CONFLICT (endpoint, feature, datetime) DO UPDATE SET modified = EXCLUDED.modified, height = EXCLUDED.height, count = EXCLUDED.count, mean = EXCLUDED.mean, std = EXCLUDED.std, min = EXCLUDED.min, q25 = EXCLUDED.q25, median = EXCLUDED.median, q75 = EXCLUDED.q75, max = EXCLUDED.max, open = EXCLUDED.open, close = EXCLUDED.close;]
(Background on this error at: https://sqlalche.me/e/14/f405)
33%|███▎ | 3/9 00:17<00:35, 5.90s/it column "nan" does not exist
LINE 1: ...00', Null, 0.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,nan,nan) O...

how to handle key list key in api response

calc_table.py:84: FutureWarning: Indexing a timezone-aware DatetimeIndex with a timezone-naive datetime is deprecated and will raise KeyError in a future version. Use a timezone-aware object instead.
hour_first = result_df.loc[hour_first_last_idx.values[idx]]

tradingview, history, value (day/hour/minute?)

add defichain-data data to data :)

eb 20 01:33:27 defichain-worker app/advanced-scheduler.5670 https://ocean.defichain.com/v0/mainnet/poolpairs
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 0%| | 0/19 [00:00<?, ?it/s] 0%| | 0/19 [00:07<?, ?it/s]
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 Traceback (most recent call last):
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 File "/app/api_calculate_tables.py", line 100, in
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 value = next((sub for sub in data if sub[search_key] == search_value), None)
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 File "/app/api_calculate_tables.py", line 100, in
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 value = next((sub for sub in data if sub[search_key] == search_value), None)
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 ~~~^^^^^^^^^^^^
Feb 20 01:33:35 defichain-worker app/advanced-scheduler.5670 TypeError: string indices must be integers, not 'str'

now:
/features
/history
/value

split day/hour/minute because minute has other limits and for value its not relevant, for ohlc reading multiple features at once should not be allowed. endpoints need to be specific and easy.

https://www.tradingview.com/chart/?symbol=BITFINEX%3ABTCUSD

DFI:USD + BTC:USD -> DFI:BTC how to add calculated features?

only load last 2 bins for update

i.e. its 12:46 i need to load/update 11-12 bin and 12-13

for minute some extra because not all features are not updated every minute

add some extra or have a once per day safety update with more time?