pgedge / cli

pgEdge Command Line Interface

Our Distributed Multi-master Postgres runs nicely (on most all flavors of Linux) in vm's, containers, bare metal, or a localhost sandbox.

You can join our Discord Server or learn more about our sponsor. Try our free Cloud Developer Edition to setup a global managed cluster in less than 90 seconds. Also check out our Docker or K8's Helm examples.

Getting started

Node Pre-Reqs

• Use a non-root user from the command line

• Tested with Python 3.9+

  • Python 3.9 on EL8, EL9, SLE-15, & Amazon Linux 2023
  • Python 3.10 on Ubuntu 22.04
  • Python 3.12 on Ubuntu 24.04 & Fedora 40

• optional: config password-less sudo if you want to autostart components with systemctl

• optional config password-less ssh to localhost for using localhost cluster commands

python3 -c "$(curl -fsSL https://pgedge-upstream.s3.amazonaws.com/REPO/install.py)"

Start by installling CLI on a single node (as per above) and then looking next at below examples.

Or try our Tutorials using  Localhost Clusters.

Learn more about our featured extensions

spock - Multi-master w/ Conflict Resolution, Auto DDL and Delta Apply

pgedge/lolor - Large Object LOgical Replication

snowflake - Distribued sequences as unique as snowflakes

Supported Components - All the best such as pg12-17, Postgis, pgVector, Citus...

Get a feel for our powerful CLI commands

   spock - Custom configure world's best logical & multi-master distributed Postgres

   localhost - Create a localhost test cluster

   cluster - Define & control a cluster of nodes

   ace - The Anti-Chaos Engine efficiently assures your distributed data is in sync

   db - Configure and control distributed PostgreSQL DB's

   um - Update Manager commands

   service - Service control commands

CLI Examples

Sandbox with latest Postgres 16, Spock & Snowflake installed into default postgres db

./pgedge install pg16 --start : install spock : install snowflake

Create database db1 owned by denis & setting up Spock & Snowflake into pg16

./pgedge setup -U denis -P secret -d db1 --pg_ver 16

Create a cluster cl1 on localhost with two nodes, then install northwind sample app on cluster

./pgedge localhost cluster-create cl1 2 : cluster app-install cl1 northwind

cli's Issues

Docker Compose error

Hi, I wanted to just try a quick demo with pgEdge to see how it behaves. I found the docker compose setup, but it seems to want to install the spock extension way too early before the database starts up:

...
pgedge-n1_1  |   new: spock.conflict_resolution = 'last_update_wins'
pgedge-n1_1  |   new: spock.save_resolutions = 'on'
pgedge-n1_1  |   new: shared_preload_libraries = 'pg_stat_statements, spock'
pgedge-n1_1  | 
pgedge-n2_1  | pg16 stopping
pgedge-n1_1  | pg16 stopping
pgedge-n2_1  | pg16 starting on port 5432
pgedge-n1_1  | pg16 starting on port 5432
pgedge-n2_1  | 
pgedge-n2_1  | $ pg16/bin/psql -p 5432 -c "CREATE EXTENSION IF NOT EXISTS spock"  demo
pgedge-n2_1  | psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL:  the database system is starting up
pgedge-n1_1  | 
pgedge-n1_1  | $ pg16/bin/psql -p 5432 -c "CREATE EXTENSION IF NOT EXISTS spock"  demo
pgedge-n1_1  | psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL:  the database system is starting up

I've tried to trace through all of the python scripts that are used in the entrypoint, but it's really difficult with the remotely executed scripts.

pgedge help synopsis is confusing.

Issue details:
./pgedge --help displays the help information. However, the argument sequence expected does not seem correct. It shows that the module must follow the command, neither of which are optional:
./pgedge <module> <command> [parameters] [options]

Whereas, it accepts the following command without the module name:
/pgedge install pgedge ...

Expected behavior:
The SYNOPSIS should be updated to match the sequence of arguments accepted by pgedge command, or the synopsis should also specify that the default module is "um", or enforce that module is specified.

[LOLOR] pgdump not working

There are 2 issues observed while taking dump of the database i.e.

• pgdump fail with error if there is any existing native large object present in the database
• pgdump don't stores LOLOR large objects created in the database

The following command was used to take the dump i.e.
pg_dump -d lolor_db -h HostIP

PS: -b option is default

./nodectl --help - Throws a deprecation warning but does nothing else.

Issue details:
./nodectl --help throws the following warning but does not print help information.
⚠ 'nc', 'nodectl', & 'ctl' commands deprecated in favor of 'pgedge'

However, nodectl responds to the other specific commands like:
./nodectl install pgedge -U superuser-name -P superuser-password -d database-name

Expected behaviour:
Help information should also be printed by nodectl along with the warning.

Job queue in diff-tables

Currently, diff-tables expects that #needed-blocks = #(cpus*2). However, for large tables, it might be worthwhile to queue jobs in an MP queue and specify limits on max queued jobs.

Row bounds for diff-tables

Might be useful to add a row bounds feature to diff tables. If the table is huge and the client is not interested in diffing the whole table, it should be possible for them to specify bounds on which the diff should happen.

metrics-check fix

for metrics-check call, data returned from db has timestamp data that can not automatically be put in the return json via mtrc_dict.update({"slots": data}). Below is code changes to consume each row of the results and format the JSON.

https://github.com/pgEdge/nodectl/blob/main/cli/scripts/spock.py#L428

` try:
     con = get_pg_connection(pg_v, db, usr)
     cur = con.cursor()
     cur.execute("SELECT count(*) as resolutions FROM spock.resolutions")
     data = cur.fetchone()
     rsltns = data[0]
     cur.close()
+   mtrc_dict.update({"slots": []})
     cur = con.cursor()
     sql_slots = \
       "SELECT slot_name, to_char(pg_wal_lsn_diff(pg_current_wal_insert_lsn(), confirmed_flush_lsn), \n" + \
      "       '999G999G999G999G999') as confirmed_flush_replication_lag, reply_time, \n" + \
      "       now() - reply_time AS reply_replication_lag \n" + \
      "  FROM pg_replication_slots R \n" + \
       "LEFT OUTER JOIN pg_stat_replication S ON R.slot_name = S.application_name \n" + \
       "ORDER BY 1"
     cur.execute(sql_slots)
-   data = cur.fetchall()
+   for row in cur:
+   mtrc_dict["slots"].append({"slotName":row[0],"flushReplicationLag":row[1],"replyTime":str(row[2]),"replicationLag":str(row[3])})
     cur.close()
-   mtrc_dict.update({"slots": data})`

Ubuntu 22.04 LTS: Pgedge installation fails after it was working for longtime.

I run pgedge setup on ubuntu 22.04 LTS. Used to work in this image for many weeks.
Following the guide https://pgedge.github.io/platform-doc/cli/GETTING-STARTED.html

when i execute ./nodectl install pgedge -U superuser-name -P superuser-password -d database-name

I get the following error.

PYTHONPATH=/home/pgedge/pgedge/hub/scripts:/home/pgedge/pgedge/hub/scripts/lib:/home/pgedge/pgedge/hub/scripts/lib/ubu22-amd:

########### Installing pgedge ###############
  ['pgedge']

File is already downloaded.

Unpacking pgedge-2-4.tar.bz2
#   tar -xf conf/cache/pgedge-2-4.tar.bz2
tar (child): lbzip2: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now