Determine best practices of table migration, establish a new table, retire an old table about fio

[lukestokes]

Thanks Casey. From EOSIO: https://developers.eos.io/manuals/eosio.cdt/v1.7/best-practices/data-design-and-migration/

[0xCasey]

~ From EOSIO:

Migration without downtime, but slower

1. Create the new version of your multi index table alongside the old one.
2. Transfer data from the old table to the new one. You may do so as part of your normal access pattern, first checking the new table to see if the entry you seek is present and if not, check the original table, and if it's present, migrate it while adding the data for the new field, then remove it from the original table to save RAM.
3. You must retain both versions of your multi-index table until you have completed this migration. At this point, you may update your contract to remove the original version of your multi-index table.

Migration with downtime, but faster

If you prefer less code complexity and can accept downtime for your application:

1. Deploy a version of your contract solely for migration purposes, and run migration transactions on every row of your table until complete. If the first table is significant, e.g. has a large number of rows, the transaction time limit could be reached while running the migration transactions. To mitigate this implement the migrate function to move a limited number of rows each time it runs;
2. Deploy a new contract using only the latest version of the table, at which point, your migration and downtime are complete.

[ericbutz]

As part of FIP-1 (https://github.com/fioprotocol/fips/blob/master/fip-0001.md) a table migration was needed. The design for this can be found at: https://fioprotocol.atlassian.net/wiki/spaces/FD/pages/49184886/FIO+Request+OBT+Table+Migration

In summary:

Step 1 - Add new tables and migrate (fio.contracts release #1)

Step 1 introduces the new table structure and adds a new “temporary” action call called migrtrx. This action enables a block producer to start migrating existing data over to the new formatted index tables.
This will migrate the three current tables (fioreqctxts, recordobts, fioreqstss) into one new table (fiotrxts). It also creates and uses a temporary table (migrledgers) that is used during the migration process.
The follow example would migrate 5 records from a current table to the new table:
./clio -u http://localhost:8889 push action -j fio.reqobt migrtrx '{"amount":"5","actor":"qbxn5zhw2ypw"}' -p qbxn5zhw2ypw@active
(The maximum amount of records able to be transferred during a single transaction has been set to 10. Any input over this value will be set to max value. After all records have been migrated, the action will return back a success response.)
Notes:
It is preferable if a single BP takes this on and calls migrtrx until all data is migrated.
The complete migration of data has to happen before fio chain is released. An off chain validation should be done to confirm this.
All new FIO Request and Record OBT records will be entered into both the old and new tables.

Step 2 - Update to FIO Request getters (fio chain release)

Step 2 updates the getters (e.g., get_sent_fio_requests) to retrieve data from the new fiotrxts table
Notes:
Users pointing to non-upgraded nodes will be retrieving data from the old tables and users pointing to upgraded nodes will be retrieving data from the new tables.

Step 3 - Clean up data from old tables (fio.contracts release #2)

All fio chain nodes should be upgraded prior to rolling out Contract Step 2.
Step 3 does two things:
Modifies the Request code to stop updating the old tables. After this release, only the new tables will get updated.
Modifies the migrtrx call. It will now remove data from the old tables.
Block producers are able to call migrtrx and remove up to 25 records during a single transaction. Added functionality also allows for the API endpoints to use the new data structure introduced in step 1.
Once the data has been deleted, wallets pointing to nodes that have not upgraded:
New OBT and Requests will be put into the new tables
Get requests and OBT getters will point to the old tables so they will get “no request found” or the OBT data will not be found results.

Step 4 - Removal of temporary actions and removal of old table references

Step 4 includes:
The removal of migrtrx temp action
Removes old table references (fioreqctxts, recordobts, fioreqstss)
Wallets pointing to nodes that have not upgraded:
New OBT and Requests will be put into the new tables.
Get requests and OBT getters will get a 500 “Error 3060003: contract table query exception” error.