./pgdiff -h1 host1 -p 5432 -d1 db1 -h2 host2 -p2 5432 -d2 db2 INDEX
-- db1: {db1 host1 5432 }
-- db2: {db2 host2 5432 }
-- Run the following SQL againt db2:
2015/03/31 10:48:31 Error running querypq: unexpected error: "setting PGKRBSRVNAME not supported"

In my environment all databases are GSSAPI auth, and the environment variable PGKRBSRVNAME is the key to making that work. There are no passwords for any databases as there are strict policies against them at my firm. Would it be hard to add support for this? I might be able to help, but I don't know any go, though I'm not opposed to learning. If it's as simple as tweaking your code to use gssapi I can manage, but I don't think adding gssapi to the go postgresql driver is a good first project for me ;)

One minor side issue: there's also a misspelling of "against" in the message above. I tested against the binary downloaded on March 31, 2015 against postgresql 9.2.4, and I can guarantee that all the standard client utilities work just fine with this environment var & w/o passwords.
Cheers!

Hi, when trying to execute pgdiff I get the message as stated in the title. What am I missing. How to enable SSL on the server?

I am not sure which part requires SSL. To establish a connection to a postgres server should not require SSL IMHO, maybe you can proof me wrong?

There is a case where a varchar in db1 has no maxlength, but in db2 it has a length of 1024 and nothing will be reported. If you run the comparison the other way pgdiff suggests altering the length of the columns to 1024 with no warning of data loss.

Hi, it's a cool tool you have.

When I run this command

PROMPT> pgdiff COLUMN -U johndoe -H 127.0.0.1 -D test1  -O "sslmode=disable" -S 'public' -P 5432 -W "password" -u johndoe -h 127.0.0.1 -d test1b -o "sslmode=disable" -s 'public' -p 5432 -w "password"
-- schemaType: COLUMN
-- db1: {test1 127.0.0.1 5432 johndoe password public sslmode=disable}
-- db2: {test1b 127.0.0.1 5432 johndoe password public sslmode=disable}
-- Run the following SQL against db2:

then no SQL is outputted.

The databases have a single table with slightly different content, so I would expect some kind of output. I have also tried with TABLE, but no luck.

Here is info about my setup. Hopefully helpful.

Table Content (different)

The table content of the databases are like this:

PROMPT> psql postgresql://johndoe:[email protected]:5432/test1b  --command 'SELECT * FROM people'
 id  | name  
-----+-------
   1 | 1b_1
   5 | 1b_5
 100 | nvidia dgx-2
(3 rows)

PROMPT> psql postgresql://johndoe:[email protected]:5432/test1  --command 'SELECT * FROM people'
 id  | name  
-----+-------
   1 | 1_1
   7 | 1_7
 100 | nvidia dgx-2
(3 rows)

PROMPT>

Table definition (same)

The people table is described like this:

PROMPT> psql postgresql://johndoe:[email protected]:5432/test1b
psql (9.6.5)
Type "help" for help.

test1b=# \d people
    Table "public.people"
 Column |  Type   | Modifiers 
--------+---------+-----------
 id     | integer | 
 name   | text    | 

test1b=# \q
PROMPT> psql postgresql://johndoe:[email protected]:5432/test1
psql (9.6.5)
Type "help" for help.

test1=# \d people
    Table "public.people"
 Column |  Type   | Modifiers 
--------+---------+-----------
 id     | integer | 
 name   | text    | 

test1=# \q
PROMPT>

Database user

The johndoe is a superuser created like this:

CREATE USER johndoe;
ALTER USER johndoe WITH PASSWORD 'password';
ALTER USER johndoe WITH SUPERUSER;
GRANT ALL PRIVILEGES ON DATABASE test1 to johndoe;
GRANT ALL PRIVILEGES ON DATABASE test1b to johndoe;

Environment

PROMPT> psql --version
psql (PostgreSQL) 9.6.5
PROMPT> uname -a
Darwin revision_demoparty2018.lan 17.3.0 Darwin Kernel Version 17.3.0: Thu Nov  9 18:09:22 PST 2017; root:xnu-4570.31.3~1/RELEASE_X86_64 x86_64
PROMPT>

System versions:

OSX Version: 10.13.2 (High Sierra)

$ psql --version
psql (PostgreSQL) 9.5.10

Command

Whilst running the following command:
$ ~/Downloads/pgdiff/pgdiff -U jmurphy -D engine_dev -O "sslmode=disable" -S public \ -u jmurphy -h localhost -d ev2_dev -o "sslmode=disable" -s public \ TABLE

I get the following errors:

Error

...

fatal error: systemstack called from unexpected goroutine
fatal error: systemstack called from unexpected goroutine
fatal error: systemstack called from unexpected goroutine
fatal error: systemstack called from unexpected goroutine
fatal error: systemstack called from unexpected goroutine
fatal error: systemstack called from unexpected goroutine

[1]    63624 bus error  ~/Downloads/pgdiff/pgdiff -U jmurphy -D engine_dev -O "sslmode=disable" -S  -

I want to compare two local database schema's, from two databases called ev2_dev (newer) and engine_dev (older), both of which user jmurphy has read/write access to.

Any idea why I'm getting this error?

I was thinking about using pgdiff as part of my deployment and migration pipeline. The pipeline is already written in go and I was curious if there would be any appetite for making pgdiff into a library. Before I put in the effort to do it I wanted to ask, would you be receptive to making the package importable?

PostgreSQL has support for extensions. Installed extensions can be listed in psql with \dx.

Also useful are the system catalogs pg_extension, pg_available_extensions and pg_available_extension_versions.

Extensions can provide their own objects, eg. some functions might be provided by extension, so pgdiff should know that an extension needs to be installed/upgraded and not list those objects (functions/views/etc.) in regular diff.

Easy extension to test this is pg_stat_statements, which has two functions and a view.

I'm hitting this error as well. Just pulled down the Macos binary, pgdiff-osx-1.0b1.tar

And updated the shell script, and launched.

• ./pgdiff -H localhost -P 5432 -D campus_hbm -h localhost -p 5432 -d campus_ddl ROLE
  fatal error: systemstack called from unexpected goroutine
  fatal error: systemstack called from unexpected goroutine
  <<1800 lines of the above>>

MacOS High Sierra 10.13.3
psql (PostgreSQL) 10.3 (but doubt this is an issue b/c doesn't look like its even getting to the point of talking to Postgres

Any tips greatly appreciated. I know nothing of 'go'.

Ken

Running the following will only select the SCHEMA and not TABLE COLUMN

./pgdiff -U <user> -W <pass> -H <host> -D <db> -O <options> -S <schema> \
-u <user> -w <pass> -h <host> -d <db> -o <options> -s <schema> \
SCHEMA TABLE COLUMN

-- schemaType: SCHEMA
-- db1: {<db> <host> <port> <user> <password> <schema> <options>}
-- db2: {<db> <host> <port> <user> <password> <schema> <options>}
-- Run the following SQL against db2:

Ideally this would actually run all 3 SCHEMA TABLE COLUMN in specified order or combined.

An error occurred while executing pgdiff on macOS 11.4:

./pgdiff
zsh: bad CPU type in executable: ./pgdiff

Hi! In our database 10 different database schemas with many objects. I would like to transfer changes in this schemas to database recipient, but pgdiff create scipt with only objects in public schema. How can I solve this problem?

Hello My Friend,

I'm trying to test the pgdiff solution on a linux machine, but I'm receiving the following error:

2016/08/19 12:11:24 Error running querypq: SSL is not enabled on the server

But looks like the option is correctly set as stated:

-- schemaType: TABLE
-- db1: {dbmon IP1 5432 zabbix sslmode=disable}
-- db2: {dbmon IP2 5432 zabbix sslmode=disable}
-- Run the following SQL against db2:

Can you give me some advice on how can I get rid of this error (without the need to implement SSL on servers)?

Let's say that in DB1 I have this table:

CREATE TABLE public."Test" (
	"TestID" integer
);

And then let's say that in DB2 I have the exact same table. Pgdiff detects a difference. What's worse, it'll generate the following script:

CREATE TABLE public.Test(); and ALTER TABLE public.Test ADD COLUMN TestID integer;

Which is case insensitive and thus different than the original table. If it were to generate a script, the correct one should be:

CREATE TABLE public."Test"(); and ALTER TABLE public."Test" ADD COLUMN "TestID" integer;

installed in postgres home dir trying to compare two schemas same db.

./pgdiff -U postgres -H localhost -D testdb -O "sslmode=disable" -S public -O "sslmode=disable" -u postgres -h localhost -d testdb -s dynapp TABLE

getting permision denied ./pgdiff.sh: Permission denied

ls -l
total 11140
-rwxrwxrwx 1 postgres postgres 6552040 Nov 17 01:42 pgdiff
-rwxrwxrwx 1 postgres postgres 2816 Feb 9 16:06 pgdiff.sh
-rwxrwxrwx 1 postgres postgres 4842856 Nov 17 01:42 pgrun
-rwxrwxrwx 1 postgres postgres 857 Nov 17 01:42 README.md

After running the generated upgrade script we get this error when trying to delete records from a parent table. (cascading deletes to its children)

The problem is the change script is generating these bad triggers.
CREATE CONSTRAINT TRIGGER "RI_ConstraintTrigger_c_16652"

That looks to be a method of creating foreign key cascading which no longer works.

Add this to "trigger.go" in func initTriggerSqlTemplate()
and t.tgname not like 'RI_ConstraintTrigger%'

To support older versions of postgresql it could be a parameter passed in to exclude them.

I need help to understand why 40 views (information_schema._pg_foreign_servers, information_schema._pg_foreign_table_columns, information_schema._pg_foreign_tables ...) are always in difference ?
Does it important ?

Hi, im writing correct host password dbname etc in pipeline like this:
./pgdiff -U postgres -W password-H localhost -P 5432 -D db1-O 'sslmode=disable' -u postgres -w password-h localhost -p 5432 -d db2 -o 'sslmode=disable' TABLE
and have error: Error running querypq: unknown authentication response: 10

Greetings Mr. Carlson

I want to thank you again for the help that you gave me the last week. But now I have another doubt (a more serious one).

• I was trying to compare 2 DB's, column by column with command:

pgdiff COLUMN -U xxx-W xxx-H 10.20.1.17 -D loc002 -O "sslmode = disable" \ -u xxx -w xxx -h 10.20.1.17 -d loc004 -o "sslmode = disable"

• I figured out that when I compare 2 tables, with the same type of columns (as I show in the picture attached) with no data restriction, show the message:
  ALTER TABLE dbname ALTER COLUMN x TYPE character varying(1024)
  And well, in reality there is not a difference between these columns, the just do no have a data restriction.
  Could you please give me an advice of what to do? Because what I need to do is to see if 2 databases are different in terms of columns, and if these are identical i do not need to plot a text.
  Please let me know if I have been clear of what if my issue. I am attaching some pictures about the issue.

Thank you again, kind regards,

Eduardo W.

It would be useful to also find differences between defined ENUM types.
I was running pgdiff between two databases to find differences with ALL and this is something that was not detected.

For example:
CREATE TYPE my_type AS ENUM ('a', 'b');
and
CREATE TYPE my_type AS ENUM ('a', 'b', 'c');

Should detect the additional type 'c' in the second database, or at least that the types are different.

I'm trying to run pgdiff against two servers running PG 8.4. Unfortunately, pgdiff seems to make lots of assumptions that do not hold with PG 8. Example:

$ bin-linux64/pgdiff [...] table
[...]
2015/01/15 16:19:36 Error running querypq: syntax error at or near "COLLATE"

or:

$ bin-linux64/pgdiff [...] index
[...]
2015/01/15 16:20:49 Error running querypq: column con.conindid does not exist

or:

$ bin-linux64/pgdiff [...] role
[...]
2015/01/15 16:21:30 Error running querypq: column r.rolreplication does not exist

Etc, etc. Looks like nothing works.

It would be awesome if pgdiff worked with PG 8 as well as 9. But if it only works with 9, please at least say so in the README! Then I can just skip it and look for another tool.

Hi,

Let me start by saying that I enjoy this project and like the initiative you've taken to create this tool #like

I've been playing around with the tool and I'm running into some walls and hopefully somebody can give me some feedback on the issue below.

Whenever I run the script against 2 databases (Postgres 9.5) it never finds any diffs on the TABLE option. On the other hand it does find diffs on eg. constraints of tables which are present in Database A but not in Database B, but a missing column or missing table isn't hit...

Commando:
./pgdiff -U postgres -W password -H localhost -P 5432 -D test1 -O 'sslmode=disable' -u postgres -w password -h localhost -p 5432 -d test2 -o 'sslmode=disable' TABLE

Output (always the same, even if there are tables present in one database and not in the other):
-- schemaType: TABLE
-- db1: {test1 localhost 5432 postgres password sslmode=disable}
-- db2: {test2 localhost 5432 postgres password sslmode=disable}
-- Run the following SQL against db2:

Is this a known bug that COLUMN and TABLE options are not doing anything?
Am I doing something wrong?

Greets.

In Postgres you can have the same Function with multiple different input Parameters.
Example myFunction(id) and myFunction(id,date)
In this case those functions always shown as different as you just care for the function Name while compare the data.
If you could use also the Field proargnames as the identifier the Problem should be solved. With this you should be able to find the function for the param set corosponding function.

Hi
I am not developer in go.
I essay to compile
Here my procedure

go version
go version go1.9.2 linux/amd64
go get "github.com/joncrlsn/misc"
go get  "github.com/joncrlsn/pgutil"
go get "github.com/lib/pq"
go get  "github.com/ogier/pflag"
go build
./column.go:114:19: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./flags.go:34:132: unknown field 'DbSchema' in struct literal of type pgutil.DbInfo
./flags.go:36:132: unknown field 'DbSchema' in struct literal of type pgutil.DbInfo
./foreignkey.go:127:19: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:116:12: dbInfo1.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:116:32: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:120:42: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:151:13: dbInfo1.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:151:33: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:155:43: dbInfo2.DbSchema undefined (type pgutil.DbInfo has no field or method DbSchema)
./function.go:155:43: too many errors

DbSchema don't exist in the struct DbInfo (pgutil.go)

type DbInfo struct {
    DbName    string
    DbHost    string
    DbPort    int32
    DbUser    string
    DbPass    string
    DbOptions string
}

Bad version of go, pgutil… ?

And Thank you for development of this project 👍

We are getting this error on the generated drop function statements:

ERROR: syntax error at or near "CASCADE"

FOR REAL EXAMPLE:
STATEMENT: DROP FUNCTION default_data.complex_schema_changes_version CASCADE;

It should be written "drop function default_data.complex_schema_changes_version() cascade;
(missing empty parenthesis)

Here is a working example of dropping functions using the "pg_get_function_identity_arguments".
You should be able to easily fix it using this.

with
functions (simple_name, full_name, arguments) as
(
select
(case when ((ns.nspname is null) or (ns.nspname = '')) then '' else ns.nspname || '.' end) || p.proname,
(case when ((ns.nspname is null) or (ns.nspname = '')) then '' else '"' || ns.nspname || '".' end) || '"' || p.proname || '"',
pg_get_function_identity_arguments(p.oid)
from
pg_proc p
left join pg_namespace ns on ns.oid = p.pronamespace
where
-- Exclude system schemas
lower(ns.nspname) not in ('pg_catalog' , 'information_schema')
order by
ns.nspname, p.proname
)
select
simple_name,
format('DROP FUNCTION %s(%s);', f.full_name, f.arguments)
from
functions f

Currently pgdiff outputs wrong migration for exclusion indexes, as it does not detect it's a special case. This is a generalization of unique constraint, a bit more info here -- https://www.postgresql.org/docs/current/static/ddl-constraints.html#DDL-CONSTRAINTS-EXCLUSION

Setup DB1:

create extension btree_gist;
create table foo (a integer, b integer);
alter table foo add constraint foo_eidx exclude using gist (a with =, b with !=);

Setup DB2:

create extension btree_gist;
create table foo (a integer, b integer);

Output for INDEX:

./pgdiff -D db1 -d db2 -U user -u user -W xxx -w xxx  INDEX
-- schemaType: INDEX
-- db1: {db1 localhost 5432 user xxx * sslmode=disable}
-- db2: {db2 localhost 5432 user xxx * sslmode=disable}
-- Run the following SQL against db2:
CREATE INDEX foo_eidx ON foo USING gist (a, b)

Seems this functionality is missing.

Hi,

I have a doubt. I tried to use your program with all the parameters, but it always shows "Not yet handled: "
Could you put a practical example of how to use it. And thanks again!

Trying to run in Mac OS X High Sierra. Postgres 10
Got:
fatal error: systemstack called from unexpected goroutine
./pgdiff.sh: line 45: 12866 Bus error: 10 ./pgdiff -U "$USER1" -W "$PASS1" -H "$HOST1" -P "$PORT1" -D "$NAME1" -O "$OPT1" -u "$USER2" -w "$PASS2" -h "$HOST2" -p "$PORT2" -d "$NAME2" -o "$OPT2" $TYPE > "$sqlFile"

It looks like it's creating the DbInfo object properly but it ignores the configured port when connecting

$ ./pgdiff FUNCTION --user1=db_user --user2=db_user --host1=localhost --host2=localhost --port1=5458 --port2=5450 --dbname1=db --dbname2=db
-- schemaType: FUNCTION
-- db1: {db localhost 5458 db_user  }
-- db2: {db localhost 5450 db_user  }
-- Run the following SQL against db2:
2016/03/01 11:48:30 Error running querydial tcp [::1]:5432: getsockopt: connection refused

It's possible to query for all views along with any views they depend on:

WITH views AS (
  SELECT viewname
       , definition
  FROM pg_views
  WHERE schemaname = 'public'
)

SELECT viewname
     , children
     , definition
FROM views
  LEFT JOIN (
    SELECT vtu.view_name
         , array_agg(vtu.table_name::text)
    FROM information_schema.view_table_usage vtu
    WHERE vtu.view_schema  = 'public'
      AND vtu.table_schema = 'public'
      AND vtu.table_name IN (SELECT viewname FROM views)
    GROUP BY 1
  ) deps(viewname, children) USING (viewname)
;

(this only considers public schema, but so does existing implementation).

Then, instead of ordering by viewname as is done now, a topological sort could be straight-forwardly provide an iteration order for views that would avoid having to make multiple pgdiff passes.

I expended some effort this evening trying to do the topological sort in the query (which would make it a drop-in replacement) but didn't get anywhere fruitful. It seems possible but it's either too late or my SQL-fu is not strong enough.

Instead, the tsort could be done in Go. Or, someone else could possibly step in and amend the query above to do it in Postgres.

This is 7-INDEX.sql generated by pgdiff

CREATE UNIQUE INDEX books_lost_id_pkey ON books_lost USING btree (id)
ALTER TABLE public.books_lost ADD CONSTRAINT books_lost_id_pkey PRIMARY KEY USING INDEX books_lost_id_pkey; 

It fails to run with error

ERROR:  syntax error (approximate position: "ALTER")
STRING 2: ALTER TABLE public.books_lost ADD CONSTRAINT books_lost_id_p...
          ^

And it gone after adding a semicolon to the first string:

CREATE UNIQUE INDEX books_lost_id_pkey ON books_lost USING btree (id);
ALTER TABLE public.books_lost ADD CONSTRAINT books_lost_id_pkey PRIMARY KEY USING INDEX books_lost_id_pkey;

Hi, previously I've been running pgdiff on Postgresql 8 but now we already migrate to Postgresql 13 and soon will be Postgresql 14. However when I try to perform pgdiff, error as below appear. Changing pghba to trust/md5/scram-sha method not working. By any chance this current version of pgdiff not support. (additional option SSLMODE disable)

Error running querypq: unknown authentication response: 10

This project seems nice, but you really should get rid of the binaries from the git repository. It makes it cumbersome and wasteful to work with the repo.

Use releases instead: https://help.github.com/articles/distributing-large-binaries/
(I see you have already used the release feature in Github)

I have 2 DB, reference DB have 10 tables, target one have 9.
Original columns order of books_lost table:

CREATE TABLE books_lost (
    id integer NOT NULL,
    title text NOT NULL,
    author_id integer,
    subject_id integer
);

Order, formed by Pgdiff (sortted by alphabet)
6-COLUMN.sql

ALTER TABLE public.books_lost ADD COLUMN author_id integer;
ALTER TABLE public.books_lost ADD COLUMN id integer NOT NULL;
ALTER TABLE public.books_lost ADD COLUMN subject_id integer;
ALTER TABLE public.books_lost ADD COLUMN title text NOT NULL;

The difference affects error on futher data synchronization between DB's. It would be right to leave original columns order.

For a table with a column named 'default', get following output:

ALTER TABLE public.example_table ADD COLUMN default boolean NOT NULL DEFAULT false;

Resulting in a syntax error. Syntactically correct output would quote the 'default' column identifier as such:

ALTER TABLE public.example_table ADD COLUMN "default" boolean NOT NULL DEFAULT false;

Are there plans to add Views, Functions, and Triggers to pgdiff?