crunchydata / crunchy-containers

Containers for Managing PostgreSQL on Kubernetes by Crunchy Data

Crunchy Container Suite

General

The Crunchy Container Suite provides Docker containers that enable rapid deployment of PostgreSQL, including administration and monitoring tools. Multiple styles of deploying PostgreSQL clusters are supported.

Please view the official Crunchy Data Container Suite documentation:

https://access.crunchydata.com/documentation/crunchy-containers/

If you are interested in contributing, please read the Contribution Guidelines.

Getting Started

Complete build and install documentation is found here. The provided Dockerfiles build the containers on a Ubi 8 base image and use the community PostgreSQL RPMs.

Crunchy provides a commercially supported version of these containers built upon RHEL 7 and the Crunchy supported PostgreSQL. Contact Crunchy for more details at:

https://www.crunchydata.com/contact/

Further descriptions of each of these containers and environment variables that can be used to tune them can be found in the Container Specifications document.

Usage

Various examples are provided in the Examples documentation for running in Docker, Kubernetes, and OpenShift environments.

You will need to set up your environment as per the Installation documentation in order to execute the examples.

crunchy-containers's Issues

crunchy-dba - evaluate pulling PV and PVC creation outside

re-evaluate the PV/PVC creation, where it occurs specifically.

upgrade grafana container

feature request: upgrade to the latest version of grafana for crunchy-grafana

create pgpool with /pgconf example

create a pgpool example where you override the config files with a volume mount

example of using syslog for postgres logs

enable syslog for postgres logs

healthcheck and readiness probes in examples

add healthcheck and readiness probes into examples

execute cannot be used while an asynchronous query is underway

I have created a transaction which has 1000 inserts and executed it via pgpool but before the transaction completed I force terminated the client.
Now, I can able to SELECT that table via pgpool as well as directly via master. But on replica when I execute SELECT query I get the following error.

execute cannot be used while an asynchronous query is underway********** Error **********

I installed my cluster using kubehelm(1 crunchy-master, 3 crunchy-replicas) and pgpool seprately. I don't even added persistent volumes. Its been a while now but I'm still getting the same error.

logs:

$ kubectl logs pgpool
...
2016-10-28 10:49:43: pid 29: LOG:  statement: BEGIN
2016-10-28 10:49:43: pid 29: LOG:  statement: 
2016-10-28 10:49:43: pid 29: LOG:  statement: INSERT INTO "public"."test" ("text") VALUES (E'0')
...
2016-10-28 10:51:15: pid 29: LOG:  statement: 
2016-10-28 10:51:15: pid 29: LOG:  statement: INSERT INTO "public"."test" ("text") VALUES (E'502')
2016-10-28 10:51:15: pid 29: ERROR:  unable to read data from frontend
2016-10-28 10:51:15: pid 29: DETAIL:  socket read failed with an error "Connection reset by peer"
2016-10-28 10:51:30: pid 30: LOG:  new connection received
2016-10-28 10:51:30: pid 30: DETAIL:  connecting host=gke-cluster-1-default-pool-14ed9ed5-r6c7.c.container-1372.internal port=53435
...

Then I also truncated that table and inserted 1000 records in single INSERT via using pgpool. It successfully completed and I can able to SELECT the inserted records via pgpool and master. But still having error on replica.

$ kubectl logs pgpool
...
2016-10-28 10:56:41: pid 30: LOG:  statement: INSERT INTO "public"."test" ("text") VALUES (E'0'),(E'1'),(E'2'),(E'3')
...
(E'9996'),(E'9997'),(E'9998'),(E'9999')
2016-10-28 10:57:27: pid 30: LOG:  statement: 

application not being able to connect via pgpool

While trying to run mattermost on openshift.
It's working successfully if connects directly to master postgresql instance
and when tries connecting via pgpool it is failing and throwing the below error:

[EROR] Failed to retrieve user profiles
team_id=5esim8ypjpbdub75hy693nwfny, err=SqlUserStore.GetProfiles:
store.sql_user.get_profiles.app_error, read tcp
x.x.x.x:49766->x.x.x.x:5432: i/o timeout

Postgres Data will be lost

There's a problem with postgres data that are stored in a volume. if by any reason I terminate the pod, Kubernetes will recreate it, maybe on the same node(everything's ok) or maybe on another node(all data will be lost).

what's your solution for this?

evaluate coredns for applicability

evaluate coredns as a replacement for crunchy-dns current implementation

update pgbadger to 1.0

evaluate coreos operator

evaluate for applicability

create kube 1.3 petset example

as of kube 1.3, petsets were introduced, this can be used to help build out a postgres cluster.

Missing 9.5-1.2.5 crunchy-postgres tag

Please add this build to your docker hub

container for upgrading

evaluate idea for wrapping a database upgrade procedure into a container

pgbackrest support

feature request to support pgbackrest

Incorrect environment variable name in install.asciidoc

The environment variable being set in Step 1 of the installation docs is named incorrectly. Currently it is CCP_IMAGE but is referenced in the next line as CCP_VERSION. This causes the pull-from-dockerhub.sh script to fail.

document running the containers on mesosphere

The list in the doc doesn't match dockerhub

So the list in this doc doesn't match what is in dockerhub

https://hub.docker.com/u/crunchydata/

Is there some way to get the two lists in agreement or better descriptions in DockerHUb so we can see what is deprecated

Thanks!

badger - reduce image

reduce the image by removing the postgresql server dep

add signal trap to pgadmin4 start.sh

similar to the postgres signal trap, add to better shutdown pgadmin4, it uses a sqllite db internally so a clean shutdown here is preferred.

NFS permissions correction

in the docs and examples, make sure NFS permissions are realistic with nfsnobody as the group owner, and not 777 on the perms, but rather 775

example of statefulset using dynamic PVC

this example would show stateful sets using dynamic PVCs

liveliness and readiness probe broken in openshift

hi,
as the subject says, those probes are broken in the examples with image tag centos7-9.5-1.2.5.

it looks like you have to manually pass the username to the commands to make them work.

I've tested it with the latest version of minishift (0.9.0) and openshift (1.3.1) with the test environment you get with oc cluster up.

regards,
fernando

upgrade badger container to latest badger package

update pgadmin4 container

use latest pgadmin4 release

Why have the watch and pgbouncer since they both seem to do failover?

@jmccormick2001 is this a pick and choose type of thing or are these two containers meant to be run at the same time? I only see the watch container in the diagram.

support pg 9.6

create dockerfiles for postgres 9.6

evaluate init container usage

evaluate the applicability of the new 'init container' functionality in kube/ose

create configmap example

cleanup examples for consistency

general catch all issue to cover the cleanup of all examples and example docs, directories should match up better across environments, docs should match up across environments, removal of outdated or unsupported or unnecessary examples across all environments

evaluation deis for applicability

evaluate deis for applicability

add statgo metrics

integration statgo into the metrics collector, and push metrics to prometheus....

https://github.com/akhenakh/statgo

allow checksums in the database

make this configurable, by default it is disabled, allow user to turn it on via env var, initdb section...see if Greg has some docs on the usage of this that can be added to the docs.

refactor automated test suite for Docker examples

crunchy-postgres: unable to create /pgdata/$HOSTNAME when using PVC. CrashLoop.

Running on kubernetes 1.3. Set up pgdata volume as a PVC, which is getting served by an NFS system. Note: if I use emptyDir for the volumes, no issue. (But of course I can lose my data.)

Issue: Receiving a "mkdir: cannot create directory '/pgdata/postgres-deployment-1217853053-9wk6b': Permission denied", which finally causes a CrashLoopBackoff after enough attempts to retry. (Note: container's being set up as part of a deployment with replicas count of 1.)

Have cross-checked examples/kube/kitchensink, as well as other kube examples. Not catching any relevant differences.

Looking in rhel7/9.5/Dockerfile.postgres.rhel7, looks like the chown happens in the Dockerfile itself, which means that it can't actually apply (I think) to directories created by persistent volume mounts. In that same Dockerfile, I see commented out setcap calls against chown like someone was tinkering with this. Also see in notes/notes-priv.txt and notes/todo2 various wrestlings that look like they're related to this issue...

Feature Request: Point in Time Recovery (PITR)

I am interested in brining up production postgres databases in OpenShift (Origin). Part of our requirements include Point In Time Recovery (PITR) but I don't see any this feature in crunchy-postgresql. Is it on the roadmap? I have done some work with pghoard (https://github.com/ohmu/pghoard) to have it work in a docker and plan to do more work to allow it to serve archived WALs for recovery out of a docker (with object store backing). Does this fit architecturally with crunchy-postgresql? Do you have other ideas to provide PITR?

add fsGroup and or supplementalGroups to all relevant kube examples for consistency

fsGroup and or supplementalGroups

to avoid persistent disk issues, the use of fsGroup and/or supplementalGroups is used
in the spec examples....all of the kube/ose examples need to reflect this.

refactor automated test suite for kube

trap the SIGTERM and pass SIGINT to the postmaster

for cleaner postgres shutdown, trap the SIGTERM and pass
SIGINT to the postmaster, handle docker stop/kill cases and docker daemon shutdown case

evaluate kompose applicability

evaluate kompose applicability for kube examples

fix restore example to prompt for the backup archive path

remove the hardcoded backup path in the standalone restore example

change crunchy-watch to run as normal user

need to change the container to run as normal user instead of root to match other containers and allow it to run in locked down OSE/kube environments.

run pgbouncer as non-root

modify the container to run as non-root user

support GCE-Kube deployment

create examples and instructions on how to run the container in a GCE-Kube environment with persistent storage

support pgaudit

add support for pgaudit extension https://github.com/pgaudit/pgaudit

update docs for promgateway container

need to update docs, replace pushgateway with promgateway in containers.asciidoc

Issue with a template

Hey there:
This template
https://github.com/CrunchyData/crunchy-containers/blob/master/examples/openshift/master-slave-dc/master-slave-rc-dc-ict.json

uses an image stream (with latest) but there is no image stream defined anywhere else. It seems almost all the other templates use an actual image with a tag rather than an image stream.

Also, what does ict stand for.

Can we get a readme for some of the examples so, you know, if you want to demo them or something we know which one to use ;)

I am going to pull the 1.2.1 images into my all-in-one for the workshop. Let me know if I need to rebuild my image

replace sleep when replica waits for master

replace the sleep in the postgres script when the replica waits for the master to become ready

create docker 1.12 example

refactor automated test suite for openshift