civicactions / bowline Goto Github PK

Unclear exactly what the issue is, but when we started out, /var/log/www ownership was as follows:

root@drupal:/var/www/logs# ls -la
total 16476
drwxr-xr-x  5 www-data dialout      170 Aug 24 16:40 .
drwxr-xr-x 28 www-data dialout      952 Aug 24 20:17 ..
-rw-r--r--  1 www-data dialout        2 Aug 24 21:32 .gitignore
-rw-r--r--  1 www-data dialout 16185940 Aug 24 21:39 apache.access.log
-rw-r--r--  1 www-data dialout   667823 Aug 21 23:11 apache.error.log
root@drupal:/var/www/logs# ls -lan 
total 16476
drwxr-xr-x  5 501 20      170 Aug 24 16:40 .
drwxr-xr-x 28 501 20      952 Aug 24 20:17 ..
-rw-r--r--  1 501 20        2 Aug 24 21:32 .gitignore
-rw-r--r--  1 501 20 16185940 Aug 24 21:39 apache.access.log
-rw-r--r--  1 501 20   667823 Aug 21 23:11 apache.error.log

Starting Apache caused an immediate logout at the apache2 -D FOREGROUND line. Apache could start normally in -X (non-forking) mode. As you can see above, the error log was not getting written to, which indicated a possible permissions issue.

We ran chmod -R 777 /var/log/www and Apache then started normally. It is unclear if ownership was the issue (gid 20 had both a "dialout" and "www-data" entry in /etc/group), or permissions (although these look correct), but clearly changing the permissions fixed something.

Perhaps it would be worthwhile to do some more agressive chown'ing to set file permissions for /var/www/*?

On slower moving Drupal sites, I sometimes get 504 gateway timeouts with the reverse proxy. I wonder if there is an Nginx setting we can adjust to allow more time to elapse before returning the 504.

Hi @davenuman,

I noticed you have php-5.3 and php-5.4 branches, but they seem pretty far behind master. I'm wondering if you think there is a way that one could specify to use php 5.3/5.4 in a config file, and have all the code live in the master branch.

I'm working on a project now with a 2 GB SQL DB, and would prefer not to have it backed up before running ./scripts/build import. It would be nice to have a flag like --no-backup to skip doing a DB dump. I know I can configure this on a per project basis, but wondering if you think this is good to have in the main project.

Just a proposal to help keep repo structure clean:

• (a) consolidate everything (bin, lib, .docker) into a single .bowline or .docker directory
• (b) keep bin but move lib into .docker

I saw this when investigating #4, just FYI: https://github.com/docker/fig/releases

Hello,

i'm curious about the name. I'm planning to do a similar project but for wordpress.

Hi @davenuman,

I might be doing something wrong. I'm trying the instructions for a new project in the README, and getting these errors:

$ sudo bash scripts/build init
Error: No such image or container: myproject_web_1

And then:

$ sudo bash scripts/build 
Error: No such image or container: myproject_web_1
# BUILDING CONTAINERS
Building without importing database. (Run as './scripts/build import' if you need to load the db.)
scripts/build: line 56: fig: command not found
scripts/build: line 57: fig: command not found
FATA[0000] Error response from daemon: No such container: composer 
FATA[0000] Error response from daemon: No such container: composer 
docker: "exec" requires a minimum of 2 arguments. See 'docker exec --help'.
scripts/build: line 69: fig: command not found
docker: "inspect" requires a minimum of 1 argument. See 'docker inspect --help'.
docker: "inspect" requires a minimum of 1 argument. See 'docker inspect --help'.
# Fig/docker container status
scripts/build: line 74: fig: command not found

# BUILD COMPLETE
database IP: ; web IP: 
web address: http:///

  If the above link does not work and you are using Boot2Docker,
  run the following command then try the link again.
      sudo route -n add 172.17.0.0/16 192.168.59.103

Any ideas? I have fig 1.0.1 installed locally via pip install fig --user.

Cool Project!
But there is a problem when i try to run compass or compass-watch. Error: compass not found in $PATH.

I installed the devtools with hoist, like recommended.

Rest works fine.

Fig has been replaced by Docker Compose, and it looks like #16 addresses that, so update the project readme.

Thanks again for sharing this project.

If you are using bowline on a system that doesn't support docker natively (i.e. OS X), it's easier to bind the apache and mysql using ports instead of expose in the docker-compose.yml. Otherwise, it seemed to be necessary to modify the iptables to route the container ip through the VM running docker (which is typically boot2docker or vagrant). Without that, it didn't seem possible to open Drupal in a browser.

I saw some lingering notes about using route to setup that routing. In my case, my VM had an IP address of 172.17.8.101. Docker was assigning IPs to containers at 10.1.0.xx. So running this command in OS X terminal added the routing I needed: sudo route -n add 10.1.0.0/24 172.17.8.101. I was then able to access Drupal at the 10.1.0.2 address I saw after running bowline.

Also, the invoke_proxy command doesn't work out of the box, because *.localtest.me (i.e. 127.0.0.1) doesn't point to the VM IP. If vagrant's port forwarding config is used to bind port 80 in the vagrant box to port 80 in the host machine, then it would work. Another option is to use nginx on the host machine to reverse proxy .localtest.me to the vagrant box. Here's an nginx server block that worked for me to do that.

server {
    listen   80;
    server_name .localtest.me;
    location / {
        proxy_pass http://172.17.8.101:80;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
        proxy_set_header X-Real-IP       $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host            $host;
        proxy_set_header        X-Forwarded-Proto $scheme;
        add_header              Front-End-Https   on;
        proxy_redirect     off;
    }
}

The alternative that I found to be better for my workflow is changing the docker-compose-yml to this:

db:
  image: mysql:5.5
  environment:
    - MYSQL_ROOT_PASSWORD=root
    - MYSQL_USER=dbuser
    - MYSQL_PASSWORD=dbpass
    - MYSQL_DATABASE=drupal
    - MYSQL_HOME=/etc
  volumes:
    - .docker/etc/my.cnf:/etc/my.cnf
  hostname: dbhost
  expose:
    - "3306"
  dns:
    - 8.8.8.8
    - 8.8.4.4
web:
  image: davenuman/bowline-web-php:latest
  command: /start.sh
  environment:
    - COMPOSER_HOME=/.composer
  ports:
    - "80:80"
  expose:
    - "80"
    - "9000"
  volumes:
    - .:/var/www
  links:
    - db:dbhost
  hostname: drupal
  dns:
    - 8.8.8.8
    - 8.8.4.4

The only change is inserting the ports section to map port 80 in the web container. Then it's easy to access Drupal via the VM IP.

Thanks again. I don't think there is one way to do this that will match everyone's way of running a docker development environment. I thought it might be helpful to note this option in case others struggle with the network routing.

First off, this project is awesome. Thank you.

I think it isn't obvious from the "existing project" section of the readme that the drupal code needs to be placed in a subfolder called docroot. Those instructions won't work with projects that have the drupal code in the root folder of the git repo.

I used bowline to get a local dev environment from a project hosted on pantheon.io. With Pantheon, the git repo has the drupal files in the root of the project. A git submodule might be the appropriate way to add it. I ended up just using bowline as-is and cloning the pantheon repo into the docroot folder. That seems to work fine.

git clone https://github.com/davenuman/bowline.git
cd bowline
# ... downloaded a db export from Pantheon and renamed it to .snapshot.sql.gz inside the bowline dir
git clone ssh://codeserver.{PANTHEON-ENV}.{PANTHEON-SITE-UUID}@codeserver.{PANTHEON-ENV}.{PANTHEON-SITE-UUID}.drush.in:2222/~/repository.git docroot
rsync -rlvz --size-only --ipv4 --progress -e 'ssh -p 2222' {PANTHEON-ENV}.{PANTHEON-SITE-UUID}@appserver.{PANTHEON-ENV}.{PANTHEON-SITE-UUID}.drush.in:files/ docroot/sites/default/files
. bin/activate
build
settings_init
import
drush st

Hello,

i'm trying to understand how to use the reverse proxy but i didn't quite get it.

Can you help me and explain how to use it ?

When I build my containers, install D8 and do a restart of the containers, they don't come back.

Error message:
Cannot restart container 72cb6d0e6ecabbabf6ea07c57aec8192fefd7109e3b2ef73a02f856138c248b6: Cannot link to a non running container: /bowlinedrupal8_php_1 AS /bowlinedrupal8_web_1/bowlinedrupal8_php_1

It turns out that bowlinedrupal8_php_1 is the bad guy. You can not event start it using

docker start bowlinedrupal8_php_1

This is, what I did

$ build
# STARTING CONTAINERS
Creating bowlinedrupal8_db_1...
Creating bowlinedrupal8_php_1...
Creating bowlinedrupal8_web_1...
 Exit Status: 0
Available commands:  deactivate
activate  bowline  check     destroy  enter  hoist   invoke_proxy  phpcs  settings_init
backup    build    composer  drush    fix-perms  import  phpcbf        run

Containers: 
db   ~  172.17.0.36
php  ~  172.17.0.37
web  ~  172.17.0.38  http://172.17.0.38/
Proxy:
  Not active. (use 'invoke_proxy' to start)

$ (bowline_drupal8) $ settings_init
Docker settings file already initialized.
Hint: If you would like to perform a fresh site install, run this:
  drush si --sites-subdir=default

$ (bowline_drupal8) $ drush si --sites-subdir=default
Running 'drush si --sites-subdir=default' inside container, using uri: http://172.17.0.38/
You are about to DROP all tables in your 'drupal' database. Do you want to continue? (y/n): y
Starting Drupal installation. This takes a while. Consider using the --notify global              [ok]
option.
Installation complete.  User name: admin  User password: 8aWLJ5kLDM                               [ok]
Congratulations, you installed Drupal!                                                        [status]

$ (bowline_drupal8) $ docker-compose restart
Restarting bowlinedrupal8_db_1...
Restarting bowlinedrupal8_php_1...
Restarting bowlinedrupal8_web_1...
Cannot restart container 72cb6d0e6ecabbabf6ea07c57aec8192fefd7109e3b2ef73a02f856138c248b6: Cannot link to a non running container: /bowlinedrupal8_php_1 AS /bowlinedrupal8_web_1/bowlinedrupal8_php_1

Using an Ubuntu 14.04 vagrant box following instructions to install Docker and Bowline. Got Docker installed following instructions at http://docs.docker.com/installation/ubuntulinux/ .

Received an error on trying to run build:

vagrant@ubuntu-14042:~/myproject$ (myproject) $ build
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Docker Compose is required for using Bowline
https://docs.docker.com/compose/install/

I work on various projects that have htdocs, public_html, www, drupalroot, etc. It would be great to have a config variable to specify what the DocumentRoot should be.

Not sure the best way to go about doing that with this project.

Currently only the docker-compose.yml file is supported and if other docker-compose files exist in a project, they are ignored.

See also fd9aee2

I am in ubuntu, using z shell, and the command simply got the install script (the wget section) but was not able to execute the rest.

I am not sure what the issue is. Probaby z shell. Would it be worth it to add extra instruction in case of failure? Something like:
chmod 777 bowline-install.sh ./bowline-install.sh

http://drupalconsole.com/

Create an additional docker container with Drupal focussed developer tools:

• xDebug, Xhprof
• Ruby, Sass, Compass
• phpmyadmin
• others?

Hi, Just cam across your project, interesting.. I also created a Drupal docker image, with lots of flexibility.
Now may aim was a bit different: Allow automated creation of simple websites in minutes.
see https://github.com/Boran/docker-drupal
"Install Ubuntu 14.04/Apache/Mysql with supervisord startup scripts
Install composer and drush
Use included Drupal7, or download Drupal, pull from git or via drush makefile
Install drupal+DB via a standard or custom profile. DB within or outside the container.
Most drupal install settings are environment settings when creating a container from the image."

Its more of a lightweight VM than a single process container.
For CI: created containers for jenkins, and a build container for jenkins (docker-drupal, docker-cibuild on my github account).
Finally a UI for managing docker and specifically containers build with these images (called webfactory, hope to have an operation version on github in the next 10 days)

It might be interesting to cross pollinate or discuss concepts...

It would be great to use pv to show the progress of importing the DB.

Here are some ideas for configuration: http://unix.stackexchange.com/questions/36769/get-a-progress-indicator-when-importing-mysql-databases

@davenuman what do you think?

To be able to modify AT themes in the web container, it looks like we need bundle, nodejs and gruntjs to be part of the build.

http://bundler.io/ for Ruby Gems
https://nodejs.org/ for node package manager (npm) for node modules https://www.npmjs.com/
http://gruntjs.com/ our task runner

Get all that installed then fire off some commands from the theme root...

bundle install
npm install
grunt


Now edit some SCSS files in the UIKit, and remember to turn off CSS aggregation in Drupal.

What would be the procedure to adding these to our containers?

I get an error like:
ERROR: b'Cannot start container 4ba007769bfc573b85db7599882dc435407e7cd792140910f536f88fb1b81e34: [8] System error: not a directory'

I was able to resolve with the following change:

diff --git a/lib/proxy/fig.yml b/lib/proxy/fig.yml
index 1bfda8b..19fac2c 100644
--- a/lib/proxy/fig.yml
+++ b/lib/proxy/fig.yml
@@ -1,10 +1,8 @@
 dnsmasq:
   build: dnsmasq
 nginx:
-  image: nginx
+  build: nginx
   ports:
     - 80:80
-  volumes:
-    - nginx.conf:/etc/nginx/nginx.conf:ro
   links:
     - dnsmasq:dnsmasq
diff --git a/lib/proxy/nginx/Dockerfile b/lib/proxy/nginx/Dockerfile
new file mode 100644
index 0000000..01bad77
--- /dev/null
+++ b/lib/proxy/nginx/Dockerfile
@@ -0,0 +1,2 @@
+FROM nginx
+COPY nginx.conf /etc/nginx/nginx.conf
diff --git a/lib/proxy/nginx.conf b/lib/proxy/nginx/nginx.conf
similarity index 100%
rename from lib/proxy/nginx.conf
rename to lib/proxy/nginx/nginx.conf

It's always obvious to the developer, not so to the external reader. Suggest documenting before you forget what the functions do. I will be happy to "QA" the documentation (as an external reader). Thanks!

I'm currently seeing an issue where the db container shuts down randomly. I'll be in the middle of doing something and then it will go away. I type in bowline and I can see it's not running. If I run build then it starts back up and I can work again. Any ideas on where to start looking?

Well, updating is pretty easy. But there's a few steps I have to do each time:

git rm -f LICENSE.md sandbox.md readme.md
git reset .gitignore
git checkout -- .gitignore

And then, manually verify the composer.json changes (if any).

I don't know what a reasonable workaround for this might look like, but just putting this out there.

Any tips to get XDebug up and running with Eclipse/PhpStorm? I'm not able to get it working with PhpStorm's "zero configuration", which makes me wonder if I'm need to do any configuration in PhpStorm to get this working properly.

Folks will probably have a need for creating custom hoist riggings that don't need to be merged upstream. Could we have a dedicated directory that bowline can search for custom hoists? related: #43

I'm getting the following error when running . bin/activate:

bowline_init:export:19: not valid in this context: db:\nweb
Bowline activated. Enter 'bowline' command to get started.

When running the bowline command I'm getting:

zsh: command not found: bowline

Also, my prompt looks like this:

$ \[\033[01;31m\]()\[\033[00m\] $

The existing file layout simply ended up the way it is during initial development. After review, it would be sensible to clean things up and follow a more expected organization.

Here is a draft new file layout:

bin/activate # Replaces scripts/include.bash. Have it alter PS1 and PATH, and add deactivate alias
bin/build # Just build on it's own
bin/import # Does what "build import" does now
bin/run # Run tests
bin/sync-db # Sync DB
lib/bowline/bowline

This would allow for project specific scripts to be added as bin/foo and such.

The scripts in bin would make use of functions located in lib/bowline/bowline. Code within scripts/build and scripts/run would be re-factored into this file as functions. The lib/bowline part could then be non-project customizable, and hence upgradable.

(Thanks @grugnog for suggesting these changes.)