wearejh / wf2 Goto Github PK

PHP environments for Magento 1, Magento 2 & Wordpress

Rust 96.78% HCL 0.15% Shell 1.11% Dockerfile 0.13% VCL 1.83%

wf2's Introduction

wf2

Express Install

Simply run the following in your terminal

zsh <(curl -L https://raw.githubusercontent.com/WeareJH/wf2/master/express-install.sh) && source ~/.zshrc

Install

wf2 is distributed as a single binary with everything packaged inside - this means you do not need PHP or Composer installed on your machine.

Download the latest version from the releases page
Make the file executable: (assuming it downloaded to the Downloads folder)

chmod +x ~/Downloads/wf2
Move the executable from your Downloads folder to /opt

sudo mv ~/Downloads/wf2 /opt
- If "opt" does not exist run the command below
  
  sudo mkdir /opt
- Then make sure the permissions are correct on the folder
  
  sudo chown -R $(whoami) /opt
Add this to the bottom of your zshrc or bash_profile:

export PATH="$PATH:/opt"
Use the following command to refresh any already open terminals

source ~/.zshrc
Or for bash users

source ~/.bash_profile
Type the following command to check all is installed OK:

wf2
You should see the same output as below (in features):

Help

For help on the commands available, run the following:

wf2 --help

--help is recipe specific. So if you're in a M2 project, you'll only see commands that are relevant to M2 sites.

If you just want to explore what the the wf2 tool can do in each recipe, just use the --recipe command

# See M2 help
wf2 --recipe M2 --help

# See Wp help
wf2 --recipe Wp --help

Troubleshooting

Elasticsearch container restarting

If you encounter the Elasticsearch container crashing and restarting, try increasing the memory available to docker. 12GB seems to be enough but others have had to allocate more. Elasticsearch is resrouce heavy.

Contributing.

Before pushing any code, run the following to ensure you catch problems before they get to CI

bash pre-push.sh

wf2's People

Contributors

Stargazers

Watchers

Forkers

dcabrejas jamelleg shakyshane

wf2's Issues

linux support

add `db-dump`

`db-import` command

`-d` to run `wf2 up` in the background

This will forward to docker-compose up -d

No MySQL client in PHP container

Some n98-magerun2 capabilities are not available inside the container by default - for example:

n98-magerun2 db:console
n98-magerun2 db:dump

The former is not really an issue when it's almost always easier just to open a client directly in the MySQL container or simply use a GUI, but the latter blocked me while trying to procure a stripped dump of a production DB for local use.

This is easily remedied by simply running the following inside PHP container:

apk add --no-cache mysql-client

Which adds the requisite mysql and mysqldump commands. This should ideally be done in the build, of course, so would need to be added to the various Dockerfiles.

[timelog] support upto 30 days (and make it fast)

global caches

Add ElasticSearch

Add ElasticSearch service to the M2 recipe

wf2 timelog missing - to show missing days (days that are 0h)

Add an option of "missing" eg.

wf2 timelog missing

to only show missing days, in discussion we counted "missing" days as ones where nothing has been booked at all.

make commands fall back to docker-compose

to enable stuff like wf2 logs unison

documentation

Bash/ZSH autocomplete

[timelog][feature] - Default range

I think the most common use of the timelog tool would be your current day. Is it worth defaulting the range to 1d ? .. with this wf2 timelog would produce todays output.

`--` on exec commands is annoying

Only needed when your trailing args have -h (because of a bug in Clap)

dns masq

`dc` (docker-compose) command

A straight pass-through to docker-compose

wf2 dc logs unison

# same as: "docker-compose logs unison"

Need extra check with synced files with push?

Might need an extra check to see if user is trying to wf2 push a synced file as it'll just delete the file in the app rather than in the container (pub/static).

 ➜ wf2 --dryrun push app/design/frontend/Graham/default/i18n/en_GB.csv

[0]: File exists check: "/Users/hghazni/Sites/graham-and-green/app/design/frontend/Graham/default/i18n/en_GB.csv"
[1]: Command: "docker exec wf2__graham-and-green__php rm -rf /var/www/app/design/frontend/Graham/default/i18n/en_GB.csv"
[2]: Notify: "- (remote) app/design/frontend/Graham/default/i18n/en_GB.csv"
[3]: Command: "docker exec -u www-data wf2__graham-and-green__php mkdir -p /var/www/app/design/frontend/Graham/default/i18n"
[4]: Command: "docker cp /Users/hghazni/Sites/graham-and-green/app/design/frontend/Graham/default/i18n/en_GB.csv wf2__graham-and-green__php:/var/www/app/design/frontend/Graham/default/i18n"

documentation in general

should use named volumes to prevent possible crossovers

docker-compose will normally create a unique-ish name for volumes, but we should be more deterministic about it to prevent bugs where containers are re-used by mistake

mysql command

Would be nice if we could do something like:

wf2 sql "some query"

And this would simply execute the following command in the MySQL container:

mysql -udocker -pdocker docker -e "some query"

And return the results.

only show limited fields from `Context` when `up` is run

Allowing to force push into the app/code folder

Sometimes the files stop syncing, and the only work around for it now is either sh-ing into the container and do the changes there manually, either restarting wf and/or docker.
Now pushing files when syncing is working is not needed but it would be nice to have an option where you can force push the files when syncing is off.

enforce time logging of yesterday when running wf2 up

Have a check when running

wf2 up

to see if you have logged time for yesterday as a reminder, we could either prompt the user and let them carry on with wf2 up or make it so they can't do it all until they book time.

It was suggested to also combine this check with the check to see if wf2 itself is up to date.

When pulling a subfolder, path is lost

When you pull a subfolder, i.e. wf2 pull var/log, the path is lost and you will end up with a log folder in the root instead of the full path.

Not recognising pv installed when importing db

[timelog][discussion] - Is Flex redundant?

The argument was that if we now have a tool that can extract the hours we have logged from jira, could we not use that instead of flex for flexitime logging, as it is effectively done automatically.

brew builds fail sometimes

Needs a `pull` command

Commands are added here

wf2/wf2/src/cli.yml

Line 22 in ab4cfa3

subcommands:

and then the given sub-command is matched here

wf2/wf2/src/main.rs

Lines 87 to 101 in 0ebf436

 let tasks = match matches.subcommand() { 

 ("up", ..) => recipe.resolve(&ctx, Cmd::Up), 

 ("down", ..) => recipe.resolve(&ctx, Cmd::Down), 

 ("stop", ..) => recipe.resolve(&ctx, Cmd::Stop), 

 ("eject", ..) => recipe.resolve(&ctx, Cmd::Eject), 

 ("exec", Some(sub_matches)) => { 

 let trailing = get_trailing(sub_matches); 

 recipe.resolve(&ctx, Cmd::Exec { trailing }) 

 } 

 ("m", Some(sub_matches)) => { 

 let trailing = get_trailing(sub_matches); 

 recipe.resolve(&ctx, Cmd::Mage { trailing }) 

 } 

 _ => None, 

 };

the rest should be easy to figure out if you CMD+Click through the types

multiple environments

a new 'doctor'

The new version of the doctor command will work a bit like brew doctor, in that it will attempt to detect problems in a current environment.

This issue is to track a list of checks to be added to the M2 recipe

check env.php exists
check auth.json exists
check that composer install has been run
check that ./bin/magento exists - because if it doesn’t, it’s normally because composer install halted/failed
check that all services are actually running
check that any url fields in the DB match those found in the wf2.yml file
check if host entries are there (or that dnsmasq is running)
check if app:config:import needs to be run
check if files exist in the container (to rule out sync problems)

Any other ideas, please comment

Add RabbitMQ

Add RabbitMQ service to the Magento 2 recipe...

--dryrun flag

for dumping a list of tasks that would occur as a result of the current command

WF2 `scripts` RFC

The purpose of this feature is allow the wf2.yaml file to contain a very lightweight, limited & simplistic task list.

There are many times when a project has certain commands or chains of commands that are often run by the developers 'in the know'. The issue is that these 'helper' commands, or 'known fixes' often never make it from the developers head/terminal into any form of documentation.

Even when scripts/commands are documented (for example in a README in the project root) there's nothing stopping those becoming out of date or invalid over time.

The canonical example of where this is a problem comes in the form of Magento 2 performance bundling - where a particular sequence of commands must be ran perfectly in order with exactly the same parameters every time to enable FE/BE to debug problems with bundling locally.

The sequence is something along the lines of

install FE deps
build FE assets
run static-content deploy with correct theme/locale
run the performance bundling

This could/should still be documented in the project readme as an overview - but we can also do better by saving the commands under 'scripts' in the wf2.yaml file.

Goals (in scope)

Provide a YAML format for defining commands
Allow optional names/descriptions for commands for use in documentation/help commands
Allow composition though sequencing
Support aliases
Static analysis of commands to prevent typos (in names, not the actual executable script)
Output the list of available commands in wf2 help (wf2 --help)

None-Goals (out of scope (for now))

Any kind of helpers/adaptors for interop with composer/npm scripts
Running tasks in parallel

Proposed format

To prevent adding complexity to the WF2 tool, the feature set will be limited to the following & takes inspiration from the Circle CI format:

scripts:
   <name>:
      description?: string
      steps: (SimpleCommand|ServiceCommand)[]

SimpleCommand: string

ServiceCommand:
    service?: string
    user?: string
    command: string
    env?: <name>: <value>[]

Simple Commands

These are commands that are run as-is, meaning they have no special meaning/relationship to WF2

# inside wf2.yaml
scripts:
  hello-world:
    description: Just an example :)
    steps:
      - run: echo "hello world"
      - run: wf2 m c:f
# you run: wf2 hello-world

WF2 Service Commands

For cases where you want to execute commands directly into a recipe's service (such as mysql) and provide additional things such as cwd, user etc.

WF2 will verify that the 'service' you specify actually matches something available in the current recipe before executing.

# inside wf2.yaml
scripts:
  hello-world:
    description: Just a service command example :)
    steps:
      - run:
          service: mysql
          user: root
          env:
            SOME_ENV: "123456"
          command: mysqldump something something

# you run: wf2 hello-world

Combining Simple + Service Commands

Internally, WF2 knows nothing about executing shell scripts/commands, all it does is run things in sequence with a Task abstraction. Because of this, we can easily compose Simple Commands with Service Commands:

# inside wf2.yaml
scripts:
  hello-world:
    description: Just a service command example :)
    steps:
      - run: echo "Just about to begin a long process - make a tea"
      - run:
          service: mysql
          user: root
          env:
            SOME_ENV: "123456"
          command: mysqldump something something
      - run: echo "All done!"

# you run: wf2 hello-world

Aliases

To make it easier to compose bigger/more complicated tasks, you can use an alias.

# inside wf2.yaml
scripts:
  my-alias:
    steps:
      - run: echo "starting"
      - hello-world # <--- alias to the task below
      - run: echo "ending!"

  hello-world:
    description: Just a service command example :)
    steps:
      - run:
          service: mysql
          user: root
          env:
            SOME_ENV: "123456"
          command: mysqldump something something

# you run: wf2 hello-world

Nginx configuration

Setting up a project I came across an issue where pages where returning 502 status codes.
After debugging the issue I found this to be the solution to the problem:
https://ma.ttias.be/nginx-proxy-upstream-sent-big-header-reading-response-header-upstream/

This needs to be added to the nginx configuration files.

Ensuring Docker images are up to date

Arguably you'll always want to run with the latest image,

We could either :

run docker-compose pull before up
have it as it's own command
have it done in docter

version upgrades can cause file collisions

An example:

recently we changed how nginx is configured - it nows uses separate files for main & upstream.

but because nginx will slurp up all conf files and merge them, some old files from a previous release where hanging around in the .wf2__recipe__project folder

the solution is simple - clean the folder before writing anything to it

`npm` command

An abstraction over npm to pass through commands to the correct place in the theme

# build all assets
wf2 npm build-all

# watch css files for changes
wf2 npm run watch

# any npm command
wf2 npm run something-else -fff

needs a push command

`composer` command

A straight pass-through for composer commands

wf2 composer install -vvv

`wf2.yaml` config file

whilst the goal is zero config - there's some things that are unavoidable.

domain
php version
dir of theme (for npm commands)

We'll leave this open to be extended, but for now these seem enough

recipe: m2

# Will accept a comma separated list
domain: local.m2

# Will be 7.2 by default
php: 7.1

# Needed for the npm command
theme_dir: app/design/frontend/Trend/default

more emoji

unison config should ignore .sql files in root

Initial Setup

Just jotting down what it took for me to get an M2 instance (Selco) working post WF2 install

Open up two terminal windows

First

wf2 up

Second

wf2 exec composer install // wf2 exec -- composer install --prefer-source
wf2 exec magento-install (I think this is still a thing if not replace with bin/magento install command)

Issues?

docker stop $(docker ps -aq)
docker rm $(docker ps -aq)
docker pull "wearejh/php:7.2-m2"
wf2 exec -- rm -rf generated vendor

Can we make the timelog tool be a standalone tool

FE builds/injecting

exec needs `-r` flag for commands as root

Spaces in absolute path to project are not supported

Anny@crafters-companion-m2$ wf2 up
[wf2 info]: using wf2.yml
No such command: Companion/crafters_companion_m2/.wf2_default/docker_compose.yml

My absolute path being .../Crafters Companion/...

So I guess it just needs to be tweaked to escape spaces. For the meantime, as I suggested on Slack I've just concatenated the problem directory name to work around it 👍

real ssl

Self update

Would be awesome to have a self update command...

Show a diff when app/etc/env.php.dist has changed from app/etc/env.php

When running wf up sometimes a message is displayed mentioning the following:

[wf2 info]: using wf2.yml
[wf2 info]: Your local app/etc/env.php doesn't match app/etc/env.php.dist, override? Y/n

I mostly ignore it because I can't be bothered to find out what changed and worry that I will inadvertently break my setup by unconsciously importing the new file.

So purely for my laziness (and ofc for our other friendly lazy devs) it would be nice to have the option to display what changed and then we can make a decision then and there.

git --no-pager  diff --no-index app/etc/env.php.dist app/etc/env.php

	let tasks = match matches.subcommand() {
	("up", ..) => recipe.resolve(&ctx, Cmd::Up),
	("down", ..) => recipe.resolve(&ctx, Cmd::Down),
	("stop", ..) => recipe.resolve(&ctx, Cmd::Stop),
	("eject", ..) => recipe.resolve(&ctx, Cmd::Eject),
	("exec", Some(sub_matches)) => {
	let trailing = get_trailing(sub_matches);
	recipe.resolve(&ctx, Cmd::Exec { trailing })
	}
	("m", Some(sub_matches)) => {
	let trailing = get_trailing(sub_matches);
	recipe.resolve(&ctx, Cmd::Mage { trailing })
	}
	_ => None,
	};