GithubHelp home page GithubHelp logo

pyexch's Introduction

pyexch

PyPI version License GitHub GitHub Discussions Donate with Bitcoin Lightning Donate with Bitcoin

EARLY PREVIEW RELEASE of a rudimentary python CLI based rest client for Coinbase

usage: pyexch [-h] [--method <get,post,>] [--url https://...]
              [--params params.json] [--call get_accounts]
              [--keystore ks.json] [--auth exch.auth]

Python Exchange CLI (pyexch)

optional arguments:
  -h, --help            show this help message and exit
  --method <get,post,>  rest http method (get<default>,post,put,delete)
  --url https://...     rest http url to perform actions upon
  --params params.json  json(5) filename holding rest parameters / data
  --call get_accounts   call method in the default client
  --keystore ks.json    json(5) filename where secrets are stored (backup!)
  --auth exch.auth      the auth method to use from keystore.

NOTE: Must name either "--call" or "--url", but not both, and "keystore.json"
is assumed if "--keystore" is not named

Future Plans

  • Add AES encryption, or port samples to CryptoDomeX
  • Add Kraken as a supported Exchange
  • Add Binance as a supported Exchange (from USA ?!?)

Getting Started

This utility is a very thin wrapper around two published Coinbase libraries, though one or both are likely no longer maintained. The first library coinbase-python uses both the v2-api, and the OAuth2-api. The second libary coinbase-advanced-py uses the v3-api. As of April 2nd, 2024 you can access all the V2 and V3 URLs using either the v3-api keys or the OAuth2-api keys. Note that since certain convenience routines like get_transaction are not in the v3-api library, some may still opt to make an OAuth2 token to use these convenience calls. Alternatively you could go directly after the URLs for these services which will work regardless.

Here's a breakdown of what you will need to do to get started:

  1. Make either a V3-API key, an OAuth2-API key, or both.
  2. Read the V2-API docs, aka "Sign In With Coinbase" (SIWC) for URLs/endpoints
  3. Read the V3-API docs, aka "Advanced Trading" (AT) for URLs/endpoints
  4. Read the V2-Library docs for names of convenience calls / routines
  5. Read the V3-Library docs for names of convenience calls / routines
  6. Optionally read the Python 3.12 Tutorial for python questions

Install and Initial Setup

This utility allows you to use a Cryptocurrency Exchange's REST API to perform basic tasks and retrieve current and historical account data. You will need to setup API / OAuth2 access keys in your account to use this utility. The utility supports both GPG and Trezor-CipherKeyValue encryption. You will need either a GPG key pair installed, or a Trezor attached. As a fallback you can store API keys in naked JSON, but that is obviously not recommended.

Install with Debug support

If you want to debug one of the client calls or step into a requests call, you can install from GIT sources. Then you can add breakpoints in source using calls to breakpoint() to get more detailed information.

  1. Get source: git clone https://github.com/brianddk/pyexch.git
  2. Switch directories: cd pyexch
  3. Install develop mode via pip: pip install -e .
  4. Verify install (cli-mode): pyexch --help
  5. Optionally add breakpoint() into one of the *.py files
  6. Optionally step through code in the python debugger (pdb)

Install without GIT

To install the most recent edition directly from GitHub tarball:

pip install https://github.com/brianddk/pyexch/archive/refs/heads/main.tar.gz

You won't get to documentation or templates, but all the code will land and function

Install last release from PIP

  1. Install: pip install pyexch
  2. Verify install: pyexch --help

Alternatively you can run it in module mode (python -m pyexch --help) or run the script directly (python pyexch.py --help).

Building a Keystore

Template keystores are provided in the templates directory. If you decide to use a naked JSON, you can simply modify the null values in the json_ks.json to fill in the key values. If you want to use encryption you will need to modify on of the encryption templates (trezor_ks.json or gnupg_ks.json) and update the unencrypted parameters. These all deal with various encryption settings. Note that for Trezor, zlib is the ONLY supported compression. Since the JSON keystore is self explanatory, I'll focus on building the encrypted keystores.

Building a GnuPG encrypted Keystore:

Start with the GnuPG template gnupg_ks.json, and change the recipient to the key-id of your GnuPG key. This can be the UID (email), or the short or long key hex.

{
  "format": "gnupg",
  "gnupg": {
    "recipient": "TYPE_YOUR_GPG_PUBLIC_KEY_HERE"
  }
}

Once you have populate the recipient, you can update the secrets interactively to keep them off of your filesystem and out of your process viewer. This uses the update_keystore command which takes a JSON template as a parameter. Any null value will trigger prompting on the console. Input is not masked or muted, so you may want to run a clear-screen after to keep reduce the amount of time it is in the console buffer.

  1. Pick a authorization method to use (coinbase.oauth2, coinbase.v2_api, coinbase.v3_api)
  2. Fill in your template data, leave null values for prompting (ex: coinbase_oauth2.json)
  3. Run the update_keystore command to prompt and encrypt secrets

Template:

{
  "format": "json",
  "coinbase": {
    "oauth2": {
      "auth_url": "https://login.coinbase.com/oauth/auth",
      "token_url": "https://login.coinbase.com/oauth2/token",
      "revoke_url": "https://login.coinbase.com/oauth2/revoke",
      "redirect_url": "http://localhost:8000/callback",
      "scope": "wallet:user:read,wallet:accounts:read",
      "id": null,
      "secret": null
    }
  }
}

With both the GnuPG and OAuth2 templates, the update_keystore call will prompt for the null fields, and encrypt the resultant merge with GnuPG

pyexch --keystore gnupg_ks.json --params coinbase_oauth2.json --call update_keystore

If you choose OAuth2, you will need to create / authorize your app to get a grant token

pyexch --keystore gnupg_ks.json --auth coinbase.oauth2 --uri https://api.coinbase.com/oauth/authorize

This will launch your web-browser and web-server to service the request and store the keys in your keystore. Note that since this get takes secret params, the auth_url is treated special and the parameters are built internally for processing to avoid the need for an encrypted params.json file.

You can display the keys that got updated on console using print_keystore call. A redaction template can be included in --params to mask secrets from getting shown on screen. In this workflow, simply seeing that the new oauth store got a auth token and a refresh token is a good indication that the calls worked. If you want to see all the secrets, just omit the --params field in the call to print_keystore

pyexch --keystore gnupg_ks.json --params redactions.json5 --call print_keystore

Note that since OAuth tokens are short lived, you will need to refresh the tokens about every hour. To refresh to token post to the token_url to get a new token. Since this get takes secret params, the token_url is treated special and the parameters are built internally for processing to avoid the need for an encrypted params.json file.

pyexch --keystore gnupg_ks.json --method post --uri https://api.coinbase.com/oauth/token

Note: The --auth choice is cached in the keystore so the last choice used is assumed unless --auth is named again.

Making REST or Client calls

Once you have good API / AUTH tokens stored, you can start making calls to the API's or Clients directly. To determine which URLs are supported look at the API V2 documentation, and API V3 documentation. Note that OAuth2 works on both v2 and v3 URLs.

To learn which calls are supported, look at the V2 Client and V3 Client. All parameters needed for the call commands are taken from the JSON file pointed to in the --params argument.

For example, to exercise the V2-API Show Authorization Information endpoint, you can do the following

pyexch --keystore gnupg_ks.json --url https://api.coinbase.com/v2/user/auth

To call the get_buy_price method from the V2-Client using BTC-USD trading pair (use \" with echo on certain shells)

echo {"currency_pair" : "BTC-USD"} > params.json
pyexch --keystore gnupg_ks.json --params params.json --call get_buy_price

Supported Call Endpoints

These are the supported options for the --call parameter

Internal call endpoints

These are the call endpoints that have internal functions without exchange interaction

  • my_ipv4 - Display your external IPv4 address for API binding
  • my_ipv6 - Display your external IPv6 address for API binding
  • new_uuid - Display a new UUID used for some API calls
  • update_keystore - Used to modify an encrypted keystore (see above)
  • print_keystore - Print the decrypted keystore to the console (with redactions)
  • sort_keystore - Sort the keystore based on template provided in params
  • sort_keyfile - Sort the keyfile based on template provided in params

coinbase.oauth2 call endpoints

These endpoints are supported when using the --auth value of coinbase.oauth2. These are exposed from the coinbase.wallet.client.OAuthClient client object.

  • refresh - Refresh the Oauth2 tokens since they are short lived
  • revoke - Revoke the existing Oauth2 token before expiration

coinbase.oauth2 / coinbase.v2_api call endpoints

These endpoints are supported when using the --auth value of either coinbase.oauth2 or coinbase.v2_api. These are exposed from the coinbase.wallet.client.OAuthClient and coinbase.wallet.client.Client client objects. All of these calls accept named parameters pulled from --params JSON.

coinbase.v3_api call endpoints

These endpoints are supported when using the --auth value of coinbase.v3_api. These are exposed from the coinbase.rest.RESTClient client object. All of these calls accept named parameters pulled from --params JSON.

pyexch's People

Contributors

brianddk avatar

Watchers

 avatar Hytrex avatar

Forkers

hytrex

pyexch's Issues

[api-??-bug] Content encoding does not follow HTTP specifications

The content decoder in Requests is failing since the content encoding appears off. The added gaurd-code should not be required.

pyexch/pyexch/exchange.py

Lines 426 to 440 in 4c89a1e

if type(data) is dict:
return data
if type(data) is Response:
if type(data.content) is str:
data = data.content
else:
try:
return data.json()
except Exception: # as e:
return dict()
if type(data) is str:
try:
return loads(data)
except Exception: # as e:
return dict()

[api-v2-bug] The V2 API requires OAuth2 CB-2FA-Token even with OAuth2 is not used

The /v2/accounts/:account_id/transactions "Send Money" endpoint requires CB-2FA-Token headers when CDP (v3) API keys are used. The documentation implies that these fields are only required for OAuth2 not for API Key authorizations.

This is a serious problem since it requires users to downgrade account security to Authy in order to use the API. It would be reasonable if account security could remain at "Security Key" levels and allow Authy codes to work in parallel on the API calls alone. But this is not the case. Perhaps this is because the API is mis-identifying the authentication method. If this is not a design error, but rather the intended use, there is a coinbase-python-py PR that adds this functionality.

[api-v3-bug] Operations do not appear atomic. Retries should not be so necessary

Normally, most REST operations are rather atomic. When an asset is bought, the balance should update, when sold, the balance should decrement. This is not the case for many endpoints. Order fills and order opens take a finite amount of time to propagate to the underlying wallet.

while abs(balance - target) > 1 and balance < target:
sleep(1)
print(f"Waiting: balance={balance:.2f}, target={target:.2f}")
balance = get_balance(cbv3, account_id)

Whats worse, even orders placed will disappear when read back, requiring retries to verify that placed orders are placed.

if resp["success"]:
for i in range(retry):
try:
resp = cbv3.v3_client.get_order(resp["order_id"])
except Exception:
print("retrying")
sleep(1)
continue
if resp["order"]:
break

[api-v2-bug] Create Address endpoint creates broken Lightning addresses

The Create Address endpoint attempts to support the new BTC-LN addresses, but mixes them with Legacy addresses.

By modifying the parameters to Create Address, an attempt can be made to make a LN-Invoice. The attempt does not generate any error, but neither does it generate a LN-Invoice.

new-ln-address.json

{
  "name": "New LN Address",
  "network": "lightning",
}

Attempt to create an LN-Invoice

btc_acct_id=23456789-abcd-ef01-2345-6789abcdef01
pyexch --param new-ln-addr.json5 --method post --url /v2/accounts/${btc_acct_id}/addresses

The attempt produces no errors, but the (redacted) response JSON is completely incorrect (note deposit_uri)

{
  "id": "89abcdef-0123-4567-89ab-cdef01234567",
  "address": "15T4EFoMRM8nYWSsCupfCHcHUzrHNQBSyt",
  "address_info": {
    "address": "15T4EFoMRM8nYWSsCupfCHcHUzrHNQBSyt"
  },
  "name": "New LN Address",
  "created_at": "2024-05-03T12:11:19Z",
  "updated_at": "2024-05-03T12:11:19Z",
  "network": "lightning",
  "uri_scheme": "lightning",
  "resource": "address",
  "resource_path": "/v2/accounts/23456789-abcd-ef01-2345-6789abcdef01/addresses/89abcdef-0123-4567-89ab-cdef01234567",
  "warnings": [
    {
      "type": "correct_address_warning",
      "title": "Only send Bitcoin (BTC) to this address",
      "details": "Sending any other asset, including Bitcoin Cash (BCH), will result in permanent loss.",
      "image_url": "https://www.coinbase.com/assets/addresses/global-receive-warning-9abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345678.png",
      "options": [
        {
          "text": "I understand",
          "style": "primary",
          "id": "dismiss"
        }
      ]
    }
  ],
  "qr_code_image_url": "https://static-assets.coinbase.com/p2p/l2/asset_network_combinations/v5/btc-lightning.png",
  "address_label": "BTC address (Lightning)",
  "default_receive": true,
  "deposit_uri": "lightning:15T4EFoMRM8nYWSsCupfCHcHUzrHNQBSyt",
  "callback_url": null,
  "share_address_copy": {
    "line1": "15T4EFoMRM8nYWSsCupfCHcHUzrHNQBSyt",
    "line2": "This is a bitcoin network address. Do not send BTC over any other network to this address or your funds may be lost. Do not send Bitcoin Cash (BCH) to this address."
  },
  "receive_subtitle": "BTC",
  "inline_warning": {
    "text": "Only send Bitcoin (BTC) to this address",
    "tooltip": {
      "title": "Bitcoin (BTC)",
      "subtitle": "This is a bitcoin network address. Do not send BTC over any other network to this address or your funds may be lost. Do not send Bitcoin Cash (BCH) to this address."
    }
  }
}

Lightning addresses can be created on the coinbase.com webpage without error. And they can be listed using the List Address endpoint with the following (redacted) data:

{
  "id": "abcdef01-2345-6789-abcd-ef0123456789",
  "address": "lnbc161810n1pnr2taqpp5dcddkl48u4v96gap6e8zkkdx8g5hmuy8rxlsn7l6p30hh7cgr9jssp5968y0gut5cu2uun6rv0yw6n6das8wuspuqk59xtrqpwxlh7xwxkqxqy8ayqnp4qf0ru8dxm7pht536amqu6re6jzsf4akdc8y7x9ze3npkcd2fh8he2rzjq25carzepgd4vqsyn44jrk85ezrpju92xyrk9apw4cdjh6yrwt5jgqqqqzudjq473cqqqqqqqqqqqqqq9qrzjqwghf7zxvfkxq5a6sr65g0gdkv768p83mhsnt0msszapamzx2qvuxqqqqzudjq473cqqqqqqqqqqqqqq9qcqzpgdqq9qyyssqeavfvqcc3yesz7804cutyyncw96f5xwkld7r76fr5ll6m38nz6v54rry5trf62g0uvyhk8wxw8mzsunxsw9tf9svvcuw6cx9lql6aksp5h0hvd",
  "address_info": {
    "address": "lnbc161810n1pnr2taqpp5dcddkl48u4v96gap6e8zkkdx8g5hmuy8rxlsn7l6p30hh7cgr9jssp5968y0gut5cu2uun6rv0yw6n6das8wuspuqk59xtrqpwxlh7xwxkqxqy8ayqnp4qf0ru8dxm7pht536amqu6re6jzsf4akdc8y7x9ze3npkcd2fh8he2rzjq25carzepgd4vqsyn44jrk85ezrpju92xyrk9apw4cdjh6yrwt5jgqqqqzudjq473cqqqqqqqqqqqqqq9qrzjqwghf7zxvfkxq5a6sr65g0gdkv768p83mhsnt0msszapamzx2qvuxqqqqzudjq473cqqqqqqqqqqqqqq9qcqzpgdqq9qyyssqeavfvqcc3yesz7804cutyyncw96f5xwkld7r76fr5ll6m38nz6v54rry5trf62g0uvyhk8wxw8mzsunxsw9tf9svvcuw6cx9lql6aksp5h0hvd"
  },
  "name": null,
  "created_at": "2024-05-03T12:30:33Z",
  "updated_at": "2024-05-03T12:30:33Z",
  "network": "lightning",
  "uri_scheme": "lightning",
  "resource": "address",
  "resource_path": "/v2/accounts/23456789-abcd-ef01-2345-6789abcdef01/addresses/abcdef01-2345-6789-abcd-ef0123456789",
  "warnings": [
    {
      "type": "correct_address_warning",
      "title": "Only send Bitcoin (BTC) to this address",
      "details": "Sending any other asset, including Bitcoin Cash (BCH), will result in permanent loss.",
      "image_url": "https://www.coinbase.com/assets/addresses/global-receive-warning-789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456.png",
      "options": [
        {
          "text": "I understand",
          "style": "primary",
          "id": "dismiss"
        }
      ]
    }
  ],
  "qr_code_image_url": "https://static-assets.coinbase.com/p2p/l2/asset_network_combinations/v5/btc-lightning.png",
  "address_label": "BTC address (Lightning)",
  "default_receive": false,
  "deposit_uri": "lightning:lnbc161810n1pnr2taqpp5dcddkl48u4v96gap6e8zkkdx8g5hmuy8rxlsn7l6p30hh7cgr9jssp5968y0gut5cu2uun6rv0yw6n6das8wuspuqk59xtrqpwxlh7xwxkqxqy8ayqnp4qf0ru8dxm7pht536amqu6re6jzsf4akdc8y7x9ze3npkcd2fh8he2rzjq25carzepgd4vqsyn44jrk85ezrpju92xyrk9apw4cdjh6yrwt5jgqqqqzudjq473cqqqqqqqqqqqqqq9qrzjqwghf7zxvfkxq5a6sr65g0gdkv768p83mhsnt0msszapamzx2qvuxqqqqzudjq473cqqqqqqqqqqqqqq9qcqzpgdqq9qyyssqeavfvqcc3yesz7804cutyyncw96f5xwkld7r76fr5ll6m38nz6v54rry5trf62g0uvyhk8wxw8mzsunxsw9tf9svvcuw6cx9lql6aksp5h0hvd",
  "callback_url": null,
  "share_address_copy": {
    "line1": "lnbc161810n1pnr2taqpp5dcddkl48u4v96gap6e8zkkdx8g5hmuy8rxlsn7l6p30hh7cgr9jssp5968y0gut5cu2uun6rv0yw6n6das8wuspuqk59xtrqpwxlh7xwxkqxqy8ayqnp4qf0ru8dxm7pht536amqu6re6jzsf4akdc8y7x9ze3npkcd2fh8he2rzjq25carzepgd4vqsyn44jrk85ezrpju92xyrk9apw4cdjh6yrwt5jgqqqqzudjq473cqqqqqqqqqqqqqq9qrzjqwghf7zxvfkxq5a6sr65g0gdkv768p83mhsnt0msszapamzx2qvuxqqqqzudjq473cqqqqqqqqqqqqqq9qcqzpgdqq9qyyssqeavfvqcc3yesz7804cutyyncw96f5xwkld7r76fr5ll6m38nz6v54rry5trf62g0uvyhk8wxw8mzsunxsw9tf9svvcuw6cx9lql6aksp5h0hvd",
    "line2": "This is a bitcoin network address. Do not send BTC over any other network to this address or your funds may be lost. Do not send Bitcoin Cash (BCH) to this address."
  },
  "receive_subtitle": "BTC",
  "inline_warning": {
    "text": "Only send Bitcoin (BTC) to this address",
    "tooltip": {
      "title": "Bitcoin (BTC)",
      "subtitle": "This is a bitcoin network address. Do not send BTC over any other network to this address or your funds may be lost. Do not send Bitcoin Cash (BCH) to this address."
    }
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.