GithubHelp home page GithubHelp logo

eosio / history-tools Goto Github PK

View Code? Open in Web Editor NEW
64.0 25.0 45.0 2.95 MB

EOSIO History Tools

Home Page: https://eosio.github.io/history-tools/

License: MIT License

CMake 1.19% JavaScript 4.48% C++ 46.98% PLpgSQL 12.19% Shell 1.22% Dockerfile 2.41% C 1.36% TypeScript 4.48% HTML 0.04% CSS 0.36% Python 25.28%

history-tools's Introduction

Description

This release contains the first release of EOSIO History-Tools fill-pg v1.0.0 and is compatible with EOSIO v2.1.0. All other tools included in the prior alpha releases of History-Tools are now deprecated.

fill-pg

fill-pg fills postgresql with data from nodeos's State History Plugin. It provides nearly all data that applications which monitor the chain need. It provides the following:

  • Header information from each block
  • Transaction and action traces, including inline actions and deferred transactions
  • Contract table history, at the block level
  • Tables which track the history of chain state, including
    • Accounts, including permissions and linkauths
    • Account resource limits and usage
    • Contract code
    • Contract ABIs
    • Consensus parameters
    • Activated consensus upgrades

PostgreSQL table schema changes

This release completely rewrites the SHiP protocol to SQL conversion code so that the database tables would directly align with the data structures defined in the SHiP protocol. This also changes the table schema used by previous releases.

Here are the basic rules for the conversion:

  • Nested SHiP struct types with more than one field are mapped to SQL custom types.
  • Nested SHiP vector types are mapped to SQL arrays.
  • SHiP variant types are mapped to a SQL type or table containing the union fields of their constituent types.

Consequently, instead of having their own tables in previous releases, action_trace, action_trace_ram_delta, action_trace_auth_sequence and action_trace_authorization are arrays nested inside transaction_trace table or action_trace type. The SQL UNNEST operator can be used to flatten arrays into tables for query.

The current list of tables created by fill-pg are:

  • account
  • account_metadata
  • block_info
  • code
  • contract_index_double
  • contract_index_long_double
  • contract_index128
  • contract_index256
  • contract_index64
  • contract_row
  • contract_table
  • fill_status
  • generated_transaction
  • global_property
  • key_value
  • permission
  • permission_link
  • protocol_state
  • received_block
  • resource_limits
  • resource_limits_config
  • resource_limits_state
  • resource_usage
  • transaction_trace

Data migration from prior releases

This release upgrades fill-pg to support nodeos v2.1.0. The remaining tools are still disabled and have not been upgraded. The schema used by fill-pg has changed. At present no data migration tool is available, so data may be manually migrated, regenerated by replaying from the desired block number, or exist side by side in two schemas, with older blocks in the v0.3.0 schema and newer blocks in the new schema. Use of the --pg-schema option will facilitate the transition.

Full details of the differences can be found via diff of plain text backups of each schema using pg_dump, not included here for brevity.

Additional details

fill-pg keeps action data and contract table data in its original binary form. Future versions may optionally support converting this to JSON.

To conserve space, fill-pg doesn't store blocks in postgresql. The majority of apps don't need the blocks since:

  • Blocks don't include inline actions and only include some deferred transactions. Most applications need to handle these, so they should examine the traces instead. e.g. many transfers live in the inline actions and deferred transactions that blocks exclude.
  • Most apps don't verify block signatures. If they do, then they should connect directly to nodeos's State History Plugin to get the necessary data. Note that contrary to popular belief, the data returned by the /v1/get_block RPC API is insufficient for signature verification since it uses a lossy JSON conversion.
  • Most apps which currently use the /v1/get_block RPC API (e.g. eosjs) only need a tiny subset of the data within each block; fill-pg stores this data. There are apps which use /v1/get_block incorrectly since their authors didn't realize the blocks miss critical data that their applications need.

fill-pg supports both full history and partial history (trim option). This allows users to make their own tradeoffs. They can choose between supporting queries covering the entire history of the chain, or save space by only covering recent history.

SHiP protocol changes (#10268)

SHiP protocol has been changed to allow a client to request the block_header only instead of the entire block. fill-pg has been updated to utilize this feature when the nodeos it connects to supports it.

Deprecation and Removal Notices

fill-rocksdb, wasm-ql-rocksdb, combo-rocksdb,wasm-ql-pg, and history-tools have been deprecated and disabled as of this v1.0.0 release.

Getting Started

You can use the docker-compose file to see how fill-pg interacts with nodeos and postgresql. For example, if you want to run fill-pg with some snapshot and setup one peer address, create a .env file such as:

SNAPSHOT_FILE=/root/history-tools/[email protected]
PEER_ADDR=peer.main.alohaeos.com:9876

Then execute the command:

docker-compose up

And you will start seeing logs from the 3 containers, showing how they interact between each other.

Further customization can be achieved with other environment variables for example to set an specific branch/commit for nodeos or history-tools, such as:

DOCKER_EOSIO_TAG=develop
DOCKER_HISTORY_TOOLS_TAG=935650a6fb9ca596affe0a3c42e6a1966675061d

You can also modify the provided docker-compose.yaml so that, for example, it takes more p2p peer addresses, for example:

       ...
       - --p2p-peer-address=peer.main.alohaeos.com:9876
       - --p2p-peer-address=p2p.eosflare.io:9876
       - --p2p-peer-address=p2p.eosargentina.io:5222
       - --p2p-peer-address=eos-bp.index.pro:9876
       - --p2p-peer-address=eosbp-0.atticlab.net:9876
       - --p2p-peer-address=mainnet.eosarabia.net:3571
       ...

Contributing

Contributing Guide

Code of Conduct

License

MIT

Important

See LICENSE for copyright and license terms.

All repositories and other materials are provided subject to the terms of this IMPORTANT notice and you must familiarize yourself with its terms. The notice contains important information, limitations and restrictions relating to our software, publications, trademarks, third-party resources, and forward-looking statements. By accessing any of our repositories and other materials, you accept and agree to the terms of the notice.

history-tools's People

Contributors

airswapteam avatar cocho-gore avatar craigbranscom avatar dependabot[bot] avatar heifner avatar huangminghuang avatar jeffreyssmith2nd avatar jgiszczak avatar josephjguerra avatar kesar avatar kj4ezj avatar larryk85 avatar leordev avatar nickjjzhao avatar nksanthosh avatar scottarnette avatar tbfleming avatar victorj8 avatar williamblevins avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

history-tools's Issues

Questions about history-tools, wasm-ql

We have been working using Mongodb (using mongodb plugin) data but since it's going to be replaced, we have decided to switch to state_history solution. However, as my expertise is not on C++ and with a limited knowledge of the wasm-ql. I was wondering if these data is possible to be queried before diving even deeper?

The data that we want to get is basically the stats of the certain application, like volume, number of users, transactions, as well as individual users spending or receiving from certain applications at certain period of time.

In the past with mongodb, we are able to index the data section, hence queries can be done in a very simplistic manner. However, with mongodb going to be deprecated, we have no way but to move on to another supported method.

Again thanks for the help.

missed account name in table of code

if I correctly understood that the wasm for the account is stored in the code table, but in this table there is no account_name field and therefore it will not be possible to associate it with the account table

does not compile with libpqxx 6.4.4

Since libpqxx 6.4.3

The old tablereader and tablewriter headers are no longer included from pqxx/pqxx. They're still there, but you won't automatically include them when you include pqxx/pqxx.

Get compiling error with code on master

history-tools/src/fill_pg_plugin.cpp:35:11: error: no type named 'tablewriter' in namespace 'pqxx'
    pqxx::tablewriter writer;
    ~~~~~~^
1 error generated.

query descending order

What is the proper way to query in descending order: from newest to oldest? I tried to check /v1/history/get_actions to see how that was done and even those results are returned ascending order: from oldest to newest.

Error encountered: terminate called after throwing an instance of 'eosio::chain::block_log_exception'

warn  2019-06-15T13:51:05.502 thread-0  block_log.cpp:214             append               ] 3190002 block_log_append_fail: fail to append block to the block log
Append to index file occuring at wrong position.
    {"position":0,"expected":198746648}
    thread-0  block_log.cpp:202 append
terminate called after throwing an instance of 'eosio::chain::block_log_exception'

I got this error while I attempt to resume the pause state_history_plugin nodeos,

  • Why do I get this error
  • should I replay it?

Double Data In Table Not Unpacking Properly

I built out a wasm-ql query service that currently searches the chain for certain actions. Here is one action table in my smart contract that I am currently trying to get working:

 ACTION create(const name& issuer,
                      const name& rev_partner,
                      const name& category,
                       const name& token_name,
                       const bool& fungible,
                       const bool& burnable,
                       const bool& sellable,
                       const bool& transferable,
                       const double& rev_split,
                       const string& base_uri,
                       const uint32_t& max_issue_days,
                       const asset& max_supply);

Everything returns properly except for the double value. Here is what the data looks like on chain:

"data": {
        "issuer": "some.p.234",
        "rev_partner": "some.p.234",
        "category": "c1",
        "token_name": "t1",
        "fungible": 0,
        "burnable": 1,
        "sellable": 1,
        "transferable": 1,
        "rev_split": "0.14999999999999999",
        "base_uri": "some.url.",
        "max_issue_days": 0,
        "max_supply": "1000 COIN"
},

Here is what my wasm-server file is unpacking into currently:

 struct create_action {
    eosio::name                            issuer               = {};
    eosio::name                            rev_partner          = {};
    eosio::name                            category             = {};
    eosio::name                            token_name           = {};
    bool                                   fungible             = {};
    bool                                   burnable             = {};
    bool                                   sellable             = {};
    bool                                   transferable         = {};
    ????                                 rev_split            = {};
    eosio::shared_memory<std::string_view>  base_uri             = {};
    uint32_t                               max_issue_days       = {};
    eosio::asset                           max_supply           = {};
};

And here is what the struct that the unpacked data is being returned to the client in:

struct tx_create_action {
  tx_action_key                          key              = {};
  eosio::name                            issuer           = {};
  eosio::name                            rev_partner      = {};
  eosio::name                            category         = {};
  eosio::name                            token_name       = {};
  bool                                   fungible         = {};
  bool                                   burnable         = {};
  bool                                   sellable         = {};
  bool                                   transferable     = {};
  ????                                     rev_split        = {};
   eosio::shared_memory<std::string_view>  base_uri         = {};
  uint32_t                               max_issue_days   = {};
  eosio::extended_asset                  max_supply       = {};

  EOSLIB_SERIALIZE(tx_create_action, (key)(issuer)(rev_partner)(category)(token_name)(fungible)(burnable)(sellable)(transferable)(rev_split)(base_uri)(max_issue_days)(max_supply))
};

However, I currently am not unpacking or returning the rev_split to the client because it will not unpack properly.

If I uncomment rev_split and try unpacking it as a double which is what the contract table stores it as, I get the following when trying to compile the client cpp file (the server cpp file compiles fine):

./libraries/eosiolib/wasmql/eosio/schema.hpp:60:39: note: candidate function not viable: no known conversion from 'double *' to 'shared_memory<std::string_view> *' (aka 'shared_memory<basic_string_view<char> > *') for 1st argument
__attribute__((noinline)) inline rope make_json_schema(shared_memory<std::string_view>*) { return "\"type\":\"string\""; }

So given that, I try unpacking it into the structs as a eosio::shared_memory<std::string_view>. If this is set, the code compiles properly, however I get the following error if I try to call the query service for that action:

Error: 500: Internal Server Error: query failed: assert failed

There is no additional error logging returned.

Got error: expected table_delta_v0 got 380572

Hey I'm having this error:

Screen Shot 2019-08-06 at 16 28 14

Database has the schema created properly, but has only one block synced.

Screen Shot 2019-08-06 at 16 28 41

My EOS node is running v1.7.4 and shows the following log:

info  2019-08-06T19:37:48.611 thread-0  state_history_plugin.c:119    start                ] incoming connection
error 2019-08-06T19:37:48.648 thread-0  state_history_plugin.c:278    on_fail              ] async_read: End of file
error 2019-08-06T19:37:48.649 thread-0  state_history_plugin.c:278    on_fail              ] async_write: Operation canceled

Does someone knows why this happens or something that might point me in the right direction?

Thanks a lot!

Add functions for complete API parity with legacy history API

Per https://eosio.github.io/history-tools/wasm-ql.html almost all of the legacy API functions are available via wasm-ql based functions. It looks like only 2 are missing:

get_key_accounts
get_controlled_accounts

The critical one get_key_accounts is used by almost all wallets to discover accounts after a user imports an existing key. Without this node operators will be quite discouraged from cutting over as they likely would need to run an alternative solution just for get_key_accounts calls.

Trying to build the history tool always throw error on ninja step 367

I am using the provided Docker file for ubuntu 18.04 and when I try to build the image, everything goes just fine and on the last step (ninja) I reach
[367/678] Building CXX object CMakeFiles/history-tools.dir/src/wasm_ql_plugin.cpp.o
and this always throw

clang: error: unable to execute command: Killed
clang: error: clang frontend command failed due to signal (use -v to see invocation)

Account Balance Query (Transaction Amounts)

I would like to calculate a point in time balance of EOS for a given account. Can you please provide some guidance on how I can see the amount of eos transferred in a transaction in the Postgres tables. Ideally, I would like to be able to query for all transactions involving a specific account and then sum up the ins and outs as of a arbitrary date and time.

Your help and guidance would be greatly appreciated.

Need help migrating postgres db created with nodeos 1.7.4 to 1.8 branch

@tbfleming I recently upgraded to nodeos 1.8.1 from 1.7.4. Then fill-pg (Alpha 0.1.0) starts to complain with the error:

error 2019-07-29T14:57:49.295 thread-0  fill_pg_plugin.cpp:967        catch_and_close      ] 
ERROR:  column "block_num" does not exist
LINE 1: select block_num, block_id from "chain".received_block where...
               ^
HINT:  Perhaps you meant to reference the column "received_block.block_id".

Is there a way to make fill-pg work with existing schema from nodeos 1.7 branch without starting from scratch again (dropping the current database and recreate) ?

Really appreciate your help and thanks in advance !

incomplete abi in the table accounts

chain: jungle

  1. payload in db:
select encode(abi, 'base64') as abi, name as  account_name, null as wasm from account where name='eosio.token' order by block_num DESC;
                                     abi                                      | account_name | wasm 
------------------------------------------------------------------------------+--------------+------
 DmVvc2lvOjphYmkvMS4xAAgHYWNjb3VudAABB2JhbGFuY2UFYXNzZXQFY2xvc2UAAgVvd25lcgRu+| eosio.token  | 
 YW1lBnN5bWJvbAZzeW1ib2wGY3JlYXRlAAIGaXNzdWVyBG5hbWUObWF4aW11bV9zdXBwbHkFYXNz+|              | 
 ZXQOY3VycmVuY3lfc3RhdHMAAwZzdXBwbHkFYXNzZXQKbWF4X3N1cHBseQVhc3NldAZpc3N1ZXIE+|              | 
 bmFtZQVpc3N1ZQADAnRvBG5hbWUIcXVhbnRpdHkFYXNzZXQEbWVtbwZzdHJpbmcEb3BlbgADBW93+|              | 
 bmVyBG5hbWUGc3ltYm9sBnN5bWJvbAlyYW1fcGF5ZXIEbmFtZQZyZXRpcmUAAghxdWFudGl0eQVh+|              | 
 c3NldARtZW1vBnN0cmluZwh0cmFuc2ZlcgAEBGZyb20EbmFtZQJ0bwRuYW1lCHF1YW50aXR5BWFz+|              | 
 c2V0BG1lbW8Gc3RyaW5nBgAAAAAAhWlEBWNsb3NlAAAAAACobNRFBmNyZWF0ZQAAAAAAAKUxdgVp+|              | 
 c3N1ZQAAAAAAADBVpQRvcGVuAAAAAACo67K6BnJldGlyZQAAAABXLTzNzQh0cmFuc2ZlcgACAAAA+|              | 
 OE9NETIDaTY0AAAHYWNjb3VudAAAAAAAkE3GA2k2NAAADmN1cnJlbmN5X3N0YXRzAAAAAA==     |              | 
 DmVvc2lvOjphYmkvMS4wAQxhY2NvdW50X25hbWUEbmFtZQUIdHJhbnNmZXIABARmcm9tDGFjY291+| eosio.token  | 
 bnRfbmFtZQJ0bwxhY2NvdW50X25hbWUIcXVhbnRpdHkFYXNzZXQEbWVtbwZzdHJpbmcGY3JlYXRl+|              | 
 AAIGaXNzdWVyDGFjY291bnRfbmFtZQ5tYXhpbXVtX3N1cHBseQVhc3NldAVpc3N1ZQADAnRvDGFj+|              | 
 Y291bnRfbmFtZQhxdWFudGl0eQVhc3NldARtZW1vBnN0cmluZwdhY2NvdW50AAEHYmFsYW5jZQVh+|              | 
 c3NldA5jdXJyZW5jeV9zdGF0cwADBnN1cHBseQVhc3NldAptYXhfc3VwcGx5BWFzc2V0Bmlzc3Vl+|              | 
 cgxhY2NvdW50X25hbWUDAAAAVy08zc0IdHJhbnNmZXIAAAAAAAClMXYFaXNzdWUAAAAAAKhs1EUG+|              | 
 Y3JlYXRlAAIAAAA4T00RMgNpNjQBCGN1cnJlbmN5AQZ1aW50NjQHYWNjb3VudAAAAAAAkE3GA2k2+|              | 
 NAEIY3VycmVuY3kBBnVpbnQ2NA5jdXJyZW5jeV9zdGF0cwAAAAA=                         |              | 
(2 rows)
  1. payload from api:
curl -d '{"account_name":"eosio.token"}' -H "Content-Type: application/json" -X POST http://${HOST_HISTORY_API}:/v1/chain/get_raw_code_and_abi
...
"abi":"DmVvc2lvOjphYmkvMS4xAAgHYWNjb3VudAABB2JhbGFuY2UFYXNzZXQFY2xvc2UAAgVvd25lcgRuYW1lBnN5bWJvbAZzeW1ib2wGY3JlYXRlAAIGaXNzdWVyBG5hbWUObWF4aW11bV9zdXBwbHkFYXNzZXQOY3VycmVuY3lfc3RhdHMAAwZzdXBwbHkFYXNzZXQKbWF4X3N1cHBseQVhc3NldAZpc3N1ZXIEbmFtZQVpc3N1ZQADAnRvBG5hbWUIcXVhbnRpdHkFYXNzZXQEbWVtbwZzdHJpbmcEb3BlbgADBW93bmVyBG5hbWUGc3ltYm9sBnN5bWJvbAlyYW1fcGF5ZXIEbmFtZQZyZXRpcmUAAghxdWFudGl0eQVhc3NldARtZW1vBnN0cmluZwh0cmFuc2ZlcgAEBGZyb20EbmFtZQJ0bwRuYW1lCHF1YW50aXR5BWFzc2V0BG1lbW8Gc3RyaW5nBgAAAAAAhWlEBWNsb3NlAAAAAACobNRFBmNyZWF0ZQAAAAAAAKUxdgVpc3N1ZQAAAAAAADBVpQRvcGVuAAAAAACo67K6BnJldGlyZQAAAABXLTzNzQh0cmFuc2ZlcucFIyMgVHJhbnNmZXIgVGVybXMgJiBDb25kaXRpb25zCgpJLCB7e2Zyb219fSwgY2VydGlmeSB0aGUgZm9sbG93aW5nIHRvIGJlIHRydWUgdG8gdGhlIGJlc3Qgb2YgbXkga25vd2xlZGdlOgoKMS4gSSBjZXJ0aWZ5IHRoYXQge3txdWFudGl0eX19IGlzIG5vdCB0aGUgcHJvY2VlZHMgb2YgZnJhdWR1bGVudCBvciB2aW9sZW50IGFjdGl2aXRpZXMuCjIuIEkgY2VydGlmeSB0aGF0LCB0byB0aGUgYmVzdCBvZiBteSBrbm93bGVkZ2UsIHt7dG99fSBpcyBub3Qgc3VwcG9ydGluZyBpbml0aWF0aW9uIG9mIHZpb2xlbmNlIGFnYWluc3Qgb3RoZXJzLgozLiBJIGhhdmUgZGlzY2xvc2VkIGFueSBjb250cmFjdHVhbCB0ZXJtcyAmIGNvbmRpdGlvbnMgd2l0aCByZXNwZWN0IHRvIHt7cXVhbnRpdHl9fSB0byB7e3RvfX0uCgpJIHVuZGVyc3RhbmQgdGhhdCBmdW5kcyB0cmFuc2ZlcnMgYXJlIG5vdCByZXZlcnNpYmxlIGFmdGVyIHRoZSB7e3RyYW5zYWN0aW9uLmRlbGF5fX0gc2Vjb25kcyBvciBvdGhlciBkZWxheSBhcyBjb25maWd1cmVkIGJ5IHt7ZnJvbX19J3MgcGVybWlzc2lvbnMuCgpJZiB0aGlzIGFjdGlvbiBmYWlscyB0byBiZSBpcnJldmVyc2libHkgY29uZmlybWVkIGFmdGVyIHJlY2VpdmluZyBnb29kcyBvciBzZXJ2aWNlcyBmcm9tICd7e3RvfX0nLCBJIGFncmVlIHRvIGVpdGhlciByZXR1cm4gdGhlIGdvb2RzIG9yIHNlcnZpY2VzIG9yIHJlc2VuZCB7e3F1YW50aXR5fX0gaW4gYSB0aW1lbHkgbWFubmVyLgoCAAAAOE9NETIDaTY0AAAHYWNjb3VudAAAAAAAkE3GA2k2NAAADmN1cnJlbmN5X3N0YXRzAAAAAA==="}

none of the abi from the database does not fully coincide with what api gives (2 times less in size than api gives)
+
I overloaded the get_raw_code_and_abi method in the JsonRpc class (eosjs library) in order to parse abi from the database — an error from the database, and with data from http api ( https://github.com/EOSIO/eos/blob/master/plugins/chain_plugin/chain_plugin.cpp#L1678 ) everything is OK

No rocksdb in build. Removed from CMake

It appears in commit 70407e8 rocksdb (combo-rocksdb and fill-rocksdb) were removed from the CMake build. Does the history plugin no longer support rocksdb in favor of postgres? Will there be a new history method for EOSIO_v2?

Please see: Commit_Here

feature request: need add transaction_id to contract_row

eos-jungle=# \d+ contract_row
                                         Table "public.contract_row"
   Column    |         Type          | Collation | Nullable | Default | Storage  | Stats target | Description 
-------------+-----------------------+-----------+----------+---------+----------+--------------+-------------
 block_num   | bigint                |           | not null |         | plain    |              | 
 present     | boolean               |           | not null |         | plain    |              | 
 code        | character varying(13) |           | not null |         | extended |              | 
 scope       | character varying(13) |           | not null |         | extended |              | 
 table       | character varying(13) |           | not null |         | extended |              | 
 primary_key | numeric               |           | not null |         | main     |              | 
 payer       | character varying(13) |           |          |         | extended |              | 
 value       | bytea                 |           |          |         | extended |              | 

How to understand for what transaction action? (now only in which block)

Bug: Unexpected char '113' in "query" happen when trying to test legacy server WASM

Please help, I want to build history api node using legacy server WASM.

These tests work fine.

cd build
curl localhost:8880/v1/chain/get_table_rows -d '{"code":"eosio", "scope":"eosio", "table":"namebids", "show_payer":true, "json":true, "key_type": "name", "index_position": "2", "limit":100}' | json_pp
node ../src/test-client.js

But when I try get block and transaction, there is error.

cleos --print-request -u http://localhost:8880 get block 100                                                                                                           
REQUEST:
---------------------
POST /v1/chain/get_block HTTP/1.0
Host: localhost:8880
content-length: 30
Accept: */*
Connection: close

{
  "block_num_or_id": "100"
}
---------------------
error 2019-11-29T04:59:34.874 thread-0  main.cpp:3941                 main                 ] Failed with error: Parse Error (4)
Unexpected char '113' in "query"
cleos --print-request -u http://localhost:8880 get transaction 'D2DB81DFCB634D5FBCC380E09F9D5D1DB5CBCD8EFD3DD48CDD31024E92FFE810'                                      1 ↵
REQUEST:
---------------------
POST /v1/history/get_transaction HTTP/1.0
Host: localhost:8880
content-length: 78
Accept: */*
Connection: close

{
  "id": "D2DB81DFCB634D5FBCC380E09F9D5D1DB5CBCD8EFD3DD48CDD31024E92FFE810"
}
---------------------
error 2019-11-29T05:03:17.121 thread-0  main.cpp:3941                 main                 ] Failed with error: Parse Error (4)
Unexpected char '113' in "query"

I had tried it using tool like postman and still got error:

query failed: assert failed

wasm-ql console log shows:

error 2019-11-29T05:07:38.530 history-t wasm_ql_http.cpp:272          handle_request       ] query failed: assert failed

can't find table account_metadata

when I run fill-lmdb, it outputs the following error and stops:

info 2019-08-06T10:26:26.727 fill-lmdb lmdb_plugin.cpp:41 plugin_startup ] using database ./chain.lmdb
info 2019-08-06T10:26:26.727 fill-lmdb lmdb_plugin.cpp:44 plugin_startup ] using query config ../src/query-config.json
info 2019-08-06T10:26:26.728 fill-lmdb fill_lmdb_plugin.cpp:97 flm_session ] connect to lmdb
info 2019-08-06T10:26:26.728 fill-lmdb fill_lmdb_plugin.cpp:162 start ] connect to observera:8080
error 2019-08-06T10:26:26.735 fill-lmdb fill_lmdb_plugin.cpp:897 catch_and_close ] can't find table account_metadata
info 2019-08-06T10:26:26.735 fill-lmdb fill_lmdb_plugin.cpp:927 ~flm_session ] fill_lmdb_plugin stopped

Performance improvement for fill-pg

Now with EIDOS and some other 'mining' tokens, we have a lot of action traces inside a single block, so it's important to batch insert/COPY FROM STDIN into action_traces, otherwise there are laggings (a couple hundreds) from time to time.
I did some experiment, and the result looks promising:

diff --git a/src/fill_pg_plugin.cpp b/src/fill_pg_plugin.cpp
index 2ab2141..3fc8370 100644
--- a/src/fill_pg_plugin.cpp
+++ b/src/fill_pg_plugin.cpp
@@ -471,6 +471,10 @@ struct fpg_session : connection_callbacks, std::enable_shared_from_this<fpg_sess
             ilog("large deltas size: ${s}", ("s", uint64_t(result.deltas->end - result.deltas->pos)));
             bulk         = true;
             large_deltas = true;
+        } else if (!bulk && result.traces && result.traces->end - result.traces->pos >= 1024 * 1024) {
+            ilog("large traces size: ${b}", ("b", result.this_block->block_num));
+            bulk         = true;
+            large_deltas = true;
         }

without patch:
image
the Y axis is how many blocks in PG lags behind nodeos (by /v1/chain/get_info)

and with patch:
image

PS:

  1. I didn't submit a pull request coz I'm not familiar with the code base, nor c++, not pgxx etc.
  2. In our case, I also commented out the code regarding inserting into action_trace_authorization and action_trace_auth_sequence coz we don't really need that and it takes a lot of space and extra insertion time. So maybe make that as an option?
  3. You should probably think about table partitioning action_trace (and other related tables). Right now every 100K blocks of action trace takes over 150GB, and it's really impossible to delete from this table (to save space, and we want to keep about 1M blocks of data). We wrote our own script to partition action_trace per 100K blocks and rolling out new partitioned table at the moment.

How to use it?

Any one tell me is there any command i can ran? I only want to use it to trace inline action and get full history from api, but i find nothing about it from document.

how to detect what was orphan?

select irreversible from fill_status UNION ALL select max(block_num) from action_trace;
 irreversible 
--------------
     34480901
     34481230
(2 rows)

the tail (329 blocks) may be overwritten by other data due to orphan.
How to understand that this action belongs to orphan?
Perhaps you need to add an id_block not only to the block_info table?

Legacy API function:get_actions is not support param pos

in file "legacy-server.cpp",just use params.account_name and params.offset ,but there is no "pos".

void get_actions(std::string_view request, const eosio::database_status& /status/) {
auto params = eosio::parse_json<get_actions_params>(request);
auto s = query_database(eosio::query_action_trace_receipt_receiver{
.snapshot_block = std::numeric_limits<uint32_t>::max(),
.first =
{
.receipt_receiver = params.account_name,
.block_num = std::numeric_limits<uint32_t>::min(),
.transaction_id = {},
.action_ordinal = std::numeric_limits<uint32_t>::min(),
},
.last =
{
.receipt_receiver = params.account_name,
.block_num = std::numeric_limits<uint32_t>::max(),
.transaction_id = eosio::checksum256_max(),
.action_ordinal = std::numeric_limits<uint32_t>::max(),
},
.max_results = uint32_t(std::abs(params.offset)),
});

feature request: Add filtering capability

With the now deprecated mongo db plugin, one really appreciated feature was to be able to filter out some actions. It is currently impossible to do it without a custom middleware, we should be able to put action filters in the config.

how to compile and what is the valid env dependence?

I'm using eosio 1.8 and with state-history plugin for my node. I have install gcc 8.3, cmake v3.15, boost v1.69, eosio.cde v1.6.1 and libpqxx v6.4.2 in my centos7 server. From CMakeLists I know the lowest version and which lib I need to install, but when I installed the newest version libs, I only got more and more error info. So I tried to fix those error, at last, I got this lib list, but when I cmake and make, it still return some errors, like this:

[ 86%] Linking CXX executable appbase_example
CMakeFiles/appbase_example.dir/main.cpp.o: In function `boost::program_options::typed_value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char>::xparse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const':
main.cpp:(.text._ZNK5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcE6xparseERNS_3anyERKSt6vectorIS7_SaIS7_EE[_ZNK5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcE6xparseERNS_3anyERKSt6vectorIS7_SaIS7_EE]+0x19): undefined reference to `boost::program_options::validate(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, int)'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `boost::program_options::validation_error::validation_error(boost::program_options::validation_error::kind_t, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)':
main.cpp:(.text._ZN5boost15program_options16validation_errorC2ENS1_6kind_tERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_i[_ZN5boost15program_options16validation_errorC5ENS1_6kind_tERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_i]+0x23): undefined reference to `boost::program_options::validation_error::get_template[abi:cxx11](boost::program_options::validation_error::kind_t)'
main.cpp:(.text._ZN5boost15program_options16validation_errorC2ENS1_6kind_tERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_i[_ZN5boost15program_options16validation_errorC5ENS1_6kind_tERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_i]+0x37): undefined reference to `boost::program_options::error_with_option_name::error_with_option_name(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `boost::program_options::typed_value<unsigned long, char>::name[abi:cxx11]() const':
main.cpp:(.text._ZNK5boost15program_options11typed_valueImcE4nameB5cxx11Ev[_ZNK5boost15program_options11typed_valueImcE4nameB5cxx11Ev]+0x1e): undefined reference to `boost::program_options::arg[abi:cxx11]'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `boost::program_options::typed_value<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, char>::name() const':
main.cpp:(.text._ZNK5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcE4nameEv[_ZNK5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcE4nameEv]+0x1e): undefined reference to `boost::program_options::arg[abi:cxx11]'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `boost::program_options::typed_value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char>::name() const':
main.cpp:(.text._ZNK5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcE4nameEv[_ZNK5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcE4nameEv]+0x1e): undefined reference to `boost::program_options::arg[abi:cxx11]'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `void boost::program_options::validate<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char>(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*, int) [clone .constprop.416]':
main.cpp:(.text._ZN5boost15program_options8validateINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcEEvRNS_3anyERKSt6vectorINS3_IT0_S4_ISB_ESaISB_EEESaISE_EEPSA_IT_SaISJ_EEi.constprop.416[_ZNK5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcE6xparseERNS_3anyERKSA_]+0xb5): undefined reference to `boost::program_options::validate(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, int)'
main.cpp:(.text._ZN5boost15program_options8validateINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcEEvRNS_3anyERKSt6vectorINS3_IT0_S4_ISB_ESaISB_EEESaISE_EEPSA_IT_SaISJ_EEi.constprop.416[_ZNK5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcE6xparseERNS_3anyERKSA_]+0x470): undefined reference to `boost::program_options::invalid_option_value::invalid_option_value(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/appbase_example.dir/main.cpp.o: In function `void boost::program_options::validate<unsigned long, char>(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, unsigned long*, long) [clone .constprop.415]':
main.cpp:(.text._ZN5boost15program_options8validateImcEEvRNS_3anyERKSt6vectorINSt7__cxx1112basic_stringIT0_St11char_traitsIS7_ESaIS7_EEESaISB_EEPT_l.constprop.415[_ZNK5boost15program_options11typed_valueImcE6xparseERNS_3anyERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISB_EE]+0x49e): undefined reference to `boost::program_options::invalid_option_value::invalid_option_value(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost16exception_detail19error_info_injectorINS_15program_options20invalid_option_valueEEE[_ZTVN5boost16exception_detail19error_info_injectorINS_15program_options20invalid_option_valueEEE]+0x30): undefined reference to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options20invalid_option_valueEEEEE[_ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options20invalid_option_valueEEEEE]+0x38): undefined reference to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTCN5boost10wrapexceptINS_15program_options20invalid_option_valueEEE0_NS_16exception_detail10clone_implINS4_19error_info_injectorIS2_EEEE[_ZTVN5boost10wrapexceptINS_15program_options20invalid_option_valueEEE]+0x38): undefined reference to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost10wrapexceptINS_15program_options20invalid_option_valueEEE[_ZTVN5boost10wrapexceptINS_15program_options20invalid_option_valueEEE]+0x38): undefined reference to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost16exception_detail19error_info_injectorINS_15program_options16validation_errorEEE[_ZTVN5boost16exception_detail19error_info_injectorINS_15program_options16validation_errorEEE]+0x30): undefined reference to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options16validation_errorEEEEE[_ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options16validation_errorEEEEE]+0x38): more undefined references to `boost::program_options::error_with_option_name::substitute_placeholders(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const' follow
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost15program_options11typed_valueImcEE[_ZTVN5boost15program_options11typed_valueImcEE]+0x38): undefined reference to `boost::program_options::value_semantic_codecvt_helper<char>::parse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcEE[_ZTVN5boost15program_options11typed_valueINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEcEE]+0x38): undefined reference to `boost::program_options::value_semantic_codecvt_helper<char>::parse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool) const'
CMakeFiles/appbase_example.dir/main.cpp.o:(.rodata._ZTVN5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcEE[_ZTVN5boost15program_options11typed_valueISt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EEcEE]+0x38): undefined reference to `boost::program_options::value_semantic_codecvt_helper<char>::parse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool) const'
../libappbase.a(application.cpp.o): In function `appbase::application::application()':
application.cpp:(.text+0x1440): undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
../libappbase.a(application.cpp.o): In function `appbase::application::set_program_options()':
application.cpp:(.text+0x1a83): undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
application.cpp:(.text+0x1ad1): undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
application.cpp:(.text+0x1bba): undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
application.cpp:(.text+0x1bfb): undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
../libappbase.a(application.cpp.o): In function `appbase::application::print_default_config(std::ostream&)':
application.cpp:(.text+0x30f5): undefined reference to `boost::program_options::option_description::long_name[abi:cxx11]() const'
application.cpp:(.text+0x3295): undefined reference to `boost::program_options::option_description::description[abi:cxx11]() const'
application.cpp:(.text+0x32a8): undefined reference to `boost::program_options::option_description::description[abi:cxx11]() const'
application.cpp:(.text+0x33a8): undefined reference to `boost::program_options::option_description::long_name[abi:cxx11]() const'
application.cpp:(.text+0x34e1): undefined reference to `boost::program_options::option_description::long_name[abi:cxx11]() const'
application.cpp:(.text+0x35db): undefined reference to `boost::program_options::option_description::format_parameter[abi:cxx11]() const'
application.cpp:(.text+0x35f3): undefined reference to `boost::program_options::option_description::long_name[abi:cxx11]() const'
application.cpp:(.text+0x36ca): undefined reference to `boost::program_options::option_description::long_name[abi:cxx11]() const'
../libappbase.a(application.cpp.o): In function `appbase::application::initialize_impl(int, char**, std::vector<appbase::abstract_plugin*, std::allocator<appbase::abstract_plugin*> >)':
application.cpp:(.text+0x3d2f): undefined reference to `boost::program_options::abstract_variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
application.cpp:(.text+0x3ef1): undefined reference to `boost::program_options::abstract_variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
application.cpp:(.text+0x4061): undefined reference to `boost::program_options::abstract_variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
application.cpp:(.text+0x4182): undefined reference to `boost::program_options::abstract_variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'
../libappbase.a(application.cpp.o): In function `boost::program_options::basic_command_line_parser<char>::extra_parser(boost::function1<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>)':
application.cpp:(.text._ZN5boost15program_options25basic_command_line_parserIcE12extra_parserENS_9function1ISt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_ERKSA_EE[_ZN5boost15program_options25basic_command_line_parserIcE12extra_parserENS_9function1ISt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_ERKSA_EE]+0x5d): undefined reference to `boost::program_options::detail::cmdline::set_additional_parser(boost::function1<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>)'
../libappbase.a(application.cpp.o): In function `boost::program_options::basic_command_line_parser<char>::basic_command_line_parser(int, char const* const*)':
application.cpp:(.text._ZN5boost15program_options25basic_command_line_parserIcEC2EiPKPKc[_ZN5boost15program_options25basic_command_line_parserIcEC5EiPKPKc]+0x1d8): undefined reference to `boost::program_options::to_internal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
application.cpp:(.text._ZN5boost15program_options25basic_command_line_parserIcEC2EiPKPKc[_ZN5boost15program_options25basic_command_line_parserIcEC5EiPKPKc]+0x257): undefined reference to `boost::program_options::detail::cmdline::cmdline(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [external/appbase/examples/appbase_example] Error 1
make[1]: *** [external/appbase/examples/CMakeFiles/appbase_example.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 87%] Linking CXX static library libfc.a
[ 87%] Built target fc
make: *** [all] Error 2

It may be my boost installation problem, after all I have installed and uninstalled a lot of difference boost version.
So I'm here looking for help, I think most developers need a doc for build help. Because I only read the wasm-api build in doc, and no CentOS doc.
Thank you for your helping.

how to write a descend query in wasm-ql when I use rocksdb?

I read doc, demo code and src code, but I did't find a way to write a descend query. When people query his transfer records, he wants to see a list start from head block, so I need to provide a descend query. In doc, query parameter 'last' must greater than 'first', and I don't find other parameter can use to order query result. Please give me a help. thank you.

duplicates in the table `account`

chain: jungle

select block_num, present, name, creation_date from account where name='blockiotoken';
 block_num | present |     name     |     creation_date     
-----------+---------+--------------+-----------------------
  10551255 | t       | blockiotoken | 2019-01-25 09:15:27.5
  10549771 | t       | blockiotoken | 2019-01-25 09:15:27.5
(2 rows)

How to understand to which of the accounts the data from the table of contract_row belong (linking by column code)
this is needed to get data from value column by abi

add https server functionality

for regular nodeos we have the https-server-address parameter. Can you please add in https functionality for the wasm ql server wql-listen

No function matches the given name and argument types. You might need to add explicit type casts.

when I execute the example request:

curl localhost:8880/v1/chain/get_table_rows -d '{"code":"eosio", "scope":"eosio", "table":"namebids", "show_payer":true, "json":true, "key_type": "name", "index_position": "2", "limit":100}'

I get the following error using postgres:

LINE 1: select * from "chain".account_range_name(26621714,'eosio','e... ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts.

I also do not see any table called account_range_name

failed fpg-create

for eos release: v1.8.0-rc1 and v1.8.0-rc2

runtime error:

fill_pg_plugin.cpp:854 catch_and_close ] don't know sql type for abi type: code_id

after adding to abi_type_to_sql_type (state_history_pg.hpp) {"code_id", type_for<std::string>},:

fill_pg_plugin.cpp:854 catch_and_close ] don't know sql type for abi type: activated_protocol_feature[]

fill_pg is not reconnecting to nodeos once the connection is broken

Yesterday, I tested eosio-explorer version which used the SHiP+fill_pg instead of mongodb. The eosio-explorer (and hence the fill-pg) is connected to a nodeos instance which is in AWS. When I tried to disconnect from the internet and reconnect, I noticed that the fill_pg is not filling the database any more after my internet connection is restored

Wasm-ql Query Lumps Strings Into Last Structs Key

Here is what my contracts table looks like:

ACTION issuepaid(const name& to,
                         const name& category,
                         const name& token_name,
                         const asset& quantity,
                         const string& relative_uri,
                         const float& amt_paid,
                         const string& currency_paid,
                         const string& memo);

And here is the data on chain:

 "data": {
        "to": "zfqsy2gdthnf",
        "category": "c1",
        "token_name": "t1",
        "quantity": "1 COINS",
        "relative_uri": "",
        "amt_paid": "7.50000000000000000",
        "currency_paid": "USD",
        "memo": "string"
  },

Here is the struct my wasm-ql server is unpacking into:

struct issuepaid_action {
    eosio::name                            to               = {};
    eosio::name                            category         = {};
    eosio::name                            token_name       = {};
    eosio::asset                           quantity         = {};
    eosio::shared_memory<std::string_view> relative_uri     = {};
    eosio::shared_memory<std::string_view> amt_paid         = {};
    eosio::shared_memory<std::string_view> currency_paid    = {};
    eosio::shared_memory<std::string_view> memo             = {};
};

and here is the struct that my server returns the data to the client in:

struct tx_issuepaid_action {
  tx_action_key                           key            = {};
  eosio::name                             to             = {};
  eosio::name                             category       = {};
  eosio::name                             token_name     = {};
  eosio::extended_asset                   quantity       = {};
  eosio::shared_memory<std::string_view>  relative_uri   = {};
  eosio::shared_memory<std::string_view>  amt_paid       = {};
  eosio::shared_memory<std::string_view>  currency_paid  = {};
  eosio::shared_memory<std::string_view>  memo           = {};

  EOSLIB_SERIALIZE(tx_issuepaid_action, (key)(to)(category)(token_name)(quantity)(relative_uri)(amt_paid)(currency_paid)(memo))
};

As you can see, I am unpacking all of the string items as a eosio::shared_memory<std::string_view>. Additionally, I am unpacking the float value amt_paid as a string as well. If I try unpacking as a float the code does not compile.

Here is the response I receive when I initiate the query:

Tx:  { key:
   { receiver: 'ekkub.d.144',
     account: 'ekkub.d.144',
     block: [ 'absolute', 6207062 ],
     transaction_id:
      'C3075EEED56EB019C224EF32F3AD1E5A39B5F13DB4B97A03B318C0085CF9A828',
     action_ordinal: 1 },
  to: 'lonfttyiowk2',
  category: 'blanko',
  token_name: 'altidude.da',
  quantity:
   { contract: 'ekkub.d.144',
     symbol: 'BLANK',
     precision: 0,
     amount: '1' },
  relative_uri: '',
  currency_paid: '',
  amt_paid: '',
  memo:
   '\u0003USD\u0015Bought from gamestore\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000�( \u0001A�=\u0000�( \u0001A�=��\u0012\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000�#�ŌM���[)\u000e?l��e��gƞۀN\u0010P;zA�\u0000\u0001\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000�( \u0001A�=\u0000\u0000\u0000\u0000S8M<\u0000\u0000�Qa\u001f�;\u0001\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000BLANK\u0000\u0000\u0000�( \u0001A�=G#\u0000\u0000\u0000\u0000\u0000\u0000H#\u0000\u0000\u0000\u0000\u0000\u0000I#\u0000\u0000\u0000\u0000\u0000\u0000J#\u0000\u0000\u0000\u0000\u0000\u0000\u0000�( \u0001A�=\u0000�( \u0001A�=��\u0012\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000�#�ŌM���[)\u000e?l��e��gƞۀN\u0010P;zA�\u0000\u0001\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u00

The things to note here:

  • The currency_paid field is empty however, the value that should be there (USD) appears to have bled over into the memo field in the first line \u0003USD
  • The amt_paid value is empty even though it is present in on-chain data
  • The memo appears to be a huge chunk of unicode encoded characters with data that appears to have bled over from the other fields.

I am sure this is an issue with how I am unpacking the data, I am just not sure how to unpack it properly.

schema not created with --fpg-create, with error message "don't know how to proccess global_property"

Hello,

I'm trying to get history from junglenet with fill-pg.
I've built docker image with ubuntu-18.04.dockerfile (commit 4859e33) and running fill-pg shows error message of "don't know how to proccess global_property" and fill-pg stops.

19:34:56 hck@hck:~/workspace/history-tools$ docker run --rm -it -v /var/run/postgresql:/var/run/postgresql history-tools ./fill-pg --fpg-create --fill-connect-to jungle2.eosdac.io:8084
info 2019-11-19T10:34:59.202 fill-pg main.cpp:98 main ] fill-pg version Unknown
info 2019-11-19T10:34:59.203 fill-pg main.cpp:99 main ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info 2019-11-19T10:34:59.203 fill-pg main.cpp:100 main ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info 2019-11-19T10:34:59.203 fill-pg fill_pg_plugin.cpp:94 fpg_session ] connect to postgresql
info 2019-11-19T10:34:59.207 fill-pg state_history_connecti:59 connect ] connect to jungle2.eosdac.io:8084
info 2019-11-19T10:34:59.952 fill-pg fill_pg_plugin.cpp:215 create_tables ] create schema "chain"
error 2019-11-19T10:35:00.158 fill-pg state_history_connecti:157 catch_and_close ] don't know how to proccess global_property
info 2019-11-19T10:35:00.158 fill-pg state_history_connecti:182 close ] closing state-history socket
^Cinfo 2019-11-19T10:35:03.677 fill-pg fill_pg_plugin.cpp:942 plugin_shutdown ] fill_pg_plugin stopped

please help me to figure out what's going wrong
Thank you

Legacy History API incompatible with History Plugin

Looks like legacy-server (https://github.com/EOSIO/history-tools/blob/d8312a722c78f7c7742a728274df18603680534a/wasms/legacy/legacy-server.cpp) don't compatible with API from History API.
It's quite confusing.
And it's more about functionality than data formats.
For example: /v1/history/get_actions just always returns actions from first to abs(offset) action, pos don't used.

Do we can change this to more robust approach or make behavior more similar to History API?
Looks like that enough use some method/filtration, which allow pagination over actions per account? Does it's currently available?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.