Dockerise SimplQ to enable self hosting about simplq-backend

[mmendiratta]

@daltonfury42 Yes I can do that! Can you link me to an issue that needs help with that?

[chalx]

@daltonfury42 but I don't think we need another repository for this. We can have each part(backend, frontend) with own Dockerfile and then just make a docker compose to make them work together.

[daltonfury42]

I get that, and that setup would be more straightforward to implement. But a single image would be convenient for someone to run with a single command, plus they would not have to install docker compose as an extra dependency.

I need to think a bit more but let me know if above doesn't seem that much important. We can go the docker-compose route if implementation wise the single image is not very neat. It's mostly inspired by https://docs.sonarqube.org/latest/setup/get-started-2-minutes/ but their frontend could be server rendered.

[chalx]

@daltonfury42 we can have subtree in simplQ-docker of backend and frontend, each one with their own Dockerfile and a bash script from which you can tell, run docker-compose or run single docker.

[daltonfury42]

I never though about subtrees, thanks.

Yes, we could start with this, and see if we can also have a single image later.

One thing I am wondering is, how would the frontend know the backend URL? Will it have to be computed from window.location? 🤔 That will be a tiny change to make.

[chalx]

On frontend we can use dotenv. But how do you deploy frontend in this moment?

[maaverik]

I have a doubt about using a single image. I haven't used docker before, so this might be a silly question, but isn't the point of having docker images so that developers could work on either the frontend or the backend and just run the other without having to care about the setup? So why would someone ever want to run both frontend and backend images at the same time? Can we do development on top of images?

[daltonfury42]

Good question, actually.

Here the idea is to use docker for "ease of deployment". This opens up a lot of possibilities of expansion. If someone is interested in self hosting SimplQ, there would be a single docker image that he could run anywhere (a physical server, K8, some other container orchestration service etc.).

@chalx is suggesting to have seperate images for frontend and backend services, and I have to agree that it's more straightforward and correct.

[chalx]

Also maybe someone will need only the API docker to create something on top of it and he doesn't need frontend for example. And also if someone wants to deploy it he can do it using Kubernetes, grab the jar and js and deploy on a Tomcat, or why not run an docker swarm cluster on this own computer.

[daltonfury42]

On frontend we can use dotenv. But how do you deploy frontend in this moment?

We are using AWS Amplify for frontend deployment. These commands are currently run to build front end. Currently the backend URL is hardcoded in code here.

We could use dotenv, but still it gets picked up at build time. Then we would have to either give the user a script to build the image, or we would have to package the code inside the docker image and build it when the container starts, supplying the right URL. 🤔

[daltonfury42]

We could theoretically substitute the URL some how in the files generated by the build before it's being served, I have to look into this.

We can define this constant in a separate js(on) file and use it, I think.

[chalx]

@daltonfury42 I have made a pull request to address this issue: SimplQ/simplQ-frontend#515

[chalx]

@daltonfury42 I think we can close this

[daltonfury42]

The hard part is over. Thanks for introducing gulp, it was something new for me. I was thinking of next steps. These are the few more things I have in mind.

1. Write one more docker file, for backend.
2. Write a script, and a docker compose file in the simplQ-docker repo that will build the images.
3. Update the Readme in the repo with instructions.
4. Add a link in the footer "Self Host SimplQ" that will take user to the Readme.

For later:
4. Write GitHub Actions that will build and push the images to Docker Hub.

[chalx]

@daltonfury42 I saw that in backed exists a Dockerfile, maybe we can use that one. Also I need another information like, what database is used and have you think to use a version control for database (ex. Flyway)?

[daltonfury42]

We should change the base image from jdk8 to jdk11, other than that we can use the same file. I didn't know that there existed one.

In our current setup, if DB connection parameters are passed as env variables, we use it to connect to the specified DB. Else fallback to an in-memory H2 database. The fallback is like a development mode setup.

In our production DB, I have been manually doing migrations till now and logging the changes here for future reference if required. Setting up Flyway seem like a logical next step.

[chalx]

@daltonfury42 Can you provide me all the SQL scripts? I will take care of this and then I will continue with dockerizing the app.

[chalx]

And also can you do a Gitter to be more easy to communicate?

[daltonfury42]

@daltonfury42 Can you provide me all the SQL scripts? I will take care of this and then I will continue with dockerizing the app.

Here is a schema dump, will this do?

--
-- PostgreSQL database dump
--

-- Dumped from database version 12.3
-- Dumped by pg_dump version 12.5 (Ubuntu 12.5-1.pgdg18.04+1)

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: owner; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.owner (
    id character varying(255) NOT NULL,
    companion_device character varying(255)
);


ALTER TABLE public.owner OWNER TO postgres;

--
-- Name: owner_queues; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.owner_queues (
    owner_id character varying(255) NOT NULL,
    queues_queue_id character varying(255) NOT NULL
);


ALTER TABLE public.owner_queues OWNER TO postgres;

--
-- Name: queue; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.queue (
    queue_id character varying(255) NOT NULL,
    owner_id character varying(255),
    queue_name character varying(255),
    queue_creation_timestamp timestamp without time zone,
    status integer
);


ALTER TABLE public.queue OWNER TO postgres;

--
-- Name: token; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.token (
    token_id character varying(255) NOT NULL,
    contact_number character varying(255),
    name character varying(255),
    notifiable boolean,
    owner_id character varying(255),
    status integer,
    token_creation_timestamp timestamp without time zone,
    queue_queue_id character varying(255),
    token_number integer
);


ALTER TABLE public.token OWNER TO postgres;

--
-- Name: queue queue_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.queue
    ADD CONSTRAINT queue_pkey PRIMARY KEY (queue_id);


--
-- Name: token token_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.token
    ADD CONSTRAINT token_pkey PRIMARY KEY (token_id);


--
-- Name: queue uk_4emkh34msxtqa7blxlcimoqy9; Type: CONSTRAINT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.queue
    ADD CONSTRAINT uk_4emkh34msxtqa7blxlcimoqy9 UNIQUE (queue_name);


--
-- Name: token fk5l67mwstpoy8yuneuva5tjlwe; Type: FK CONSTRAINT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.token
    ADD CONSTRAINT fk5l67mwstpoy8yuneuva5tjlwe FOREIGN KEY (queue_queue_id) REFERENCES public.queue(queue_id);


--
-- Name: SCHEMA public; Type: ACL; Schema: -; Owner: postgres
--

REVOKE ALL ON SCHEMA public FROM rdsadmin;
REVOKE ALL ON SCHEMA public FROM PUBLIC;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO PUBLIC;


--
-- PostgreSQL database dump complete
--

And also can you do a Gitter to be more easy to communicate?

I was trying it out, created a community and have added you to it.

[daltonfury42]

What are the tasks left for this? What I can think of are:

1. Write a script in the repo to build the frontend and backend images.
2. Write a docker compose file, which would have three services: frontend, backend and a postgres db (with migrations applied)
3. Update the readme with instructions.

@mmendiratta has offered to help us with this. @chalx can you guide him?

[daltonfury42]

Here is a sample setup @PavanDongare found: https://github.com/docker/awesome-compose/tree/master/react-java-mysql

[chalx]

@daltonfury42 I have done that but I had no time to pushed it. I will commit these days.

[chalx]

Also I will investigate about github docker registry and push images there and docker-compose will use that images.

[mmendiratta]

I like the idea of Docker Compose, so @chalx you have done the suggestion the @daltonfury42 gave? Otherwise, I can go ahead and create the Dockerfiles, the docker-compose, and a Makefile that can spin up any one of the FE, BE, or DB.

[chalx]

@mmendiratta To make it run I have also changed some code on backend so I will do a pull request with them, then I think we should look on GitHub docker registry and docker-compose will be easy to do. Using this approach will make full application start very easy for everyone(technical and non-technical users). If GitHub docker registry is not feasible then I think we can put images on docker hub. What do you think @daltonfury42 ?

[daltonfury42]

Yes, on the same page. Let us try out the GitHub registry and fallback to Dockerhub if it doesn't meet our needs.

@mmendiratta Let's avoid duplicate efforts, let @chalx take his time and finish this up. There are only two other MVP features that are left to do, one is SMS alerts and then the queue history. If you are interested in helping with SMS, let me know. I am not good at app development, I am still learning, I have a basic android app, you can take it up from me and learn some android also along the way.