lionsad / drupal_ti Goto Github PK

http://docs.travis-ci.com/user/migrating-from-legacy/

What are the restrictions? #

Using sudo isn’t possible (right now) #

Our new container infrastructure uses Docker under the hood. This has a lot of benefits like faster boot times and better utilization of resources. But it also comes with some restrictions. At this point, it’s not possible to use any command requiring sudo in your builds.

If you require sudo, for instance to install Ubuntu packages, a workaround is to use precompiled binaries, uploading them to S3 and downloading them as part of your build,then installing them into a non-root directory.

Databases don’t run off a memory disk #

On our legacy infrastructure, both MySQL and PostgreSQL run off a memory disk to increase transaction and query speed. This can impact projects making heavy use of transactions or fixtures.

Is there any way to get drupal_ti working together with BrowserTestBase?

The failure in https://travis-ci.org/d8-contrib-modules/file_encrypt/jobs/135598949 seems to cause issues about it

Commit:
drupalcommerce/commerce@bec25f7

Result:

/home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh: line 13: ./vendor/bin/phpunit: > No such file or directory
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh exited with a failure.

https://travis-ci.org/commerceguys/commerce/jobs/76440492

Probably needs a similar fix as mradcliffe@729b806

Currently, the latest version of Selenium is 2.48.2. However, if I set DRUPAL_TI_BEHAT_SELENIUM to that value, I get:

--2015-10-13 13:02:17--  http://selenium-release.storage.googleapis.com/2.48.2/selenium-server-standalone-2.48.2.0.jar
Resolving selenium-release.storage.googleapis.com (selenium-release.storage.googleapis.com)... 173.194.206.128, 2607:f8b0:400d:c06::80
Connecting to selenium-release.storage.googleapis.com (selenium-release.storage.googleapis.com)|173.194.206.128|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-10-13 13:02:17 ERROR 404: Not Found.

The correct URL is actually:

http://selenium-release.storage.googleapis.com/2.48/selenium-server-standalone-2.48.2.jar

So, it needs to use the first 2-parts of the version number for the directory bit of the URL, but then the full 3-parts for the filename part.

drafty specifies field_collection and entity_translation only as test_dependencies[], which drush does not support.

It also fails on drush/drush:master completely as it specifies drafty_enforce, but drush can't find it - even though this is in drafty.

https://travis-ci.org/mglaman/commerce_demo/jobs/201552451
https://travis-ci.org/drupalcommerce/commerce/jobs/201116282

There's a weird issue.

0.65s$ drupal-ti script
Cannot open file "/home/travis/build/drupalcommerce/drupal-8/drupal/modules/commerce/.php".

0.47s$ drupal-ti script
Cannot open file "/home/travis/build/mglaman/drupal-8/drupal/modules/commerce/demo/.php".
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit-core/script.sh exited with a failure.

drupal-media/url_embed#6 should not be needed for each project.

Support properly via an option within DRUPAL_TI:

# Auto-detect if composer.json with type: 'drupal-module' is present
# Use 0 to disable, use 1 to enable.
DRUPAL_TI_USE_COMPOSER_MANAGER="auto"

Needs to be supported by simpletest and behat runners.

Thanks a lot for publishing this code! The more I read through it, the more useful features I find.

I am running all drupal core simpletests with my module enabled, and I was really confused to see travis failing my build even though all the tests were passing. It turns out that the regex is catching some of the drupal test descriptions:

egrep -i "([1-9]+ fail)|(Fatal error)|([1-9]+ exception)" /tmp/simpletest-result.txt && exit 1

UserLoginTest.php  Drupal\user\Tests\UserLoginTest
Raw "There have been more than 3 failed login attempts for this account."

"[1-9]+ fail" catches "3 failed"

Hello. I'm having a problem running the test. You can see the error and the entire output at
https://travis-ci.org/Tantacom/tantacom-web/jobs/134472696#L631

and my travis.yml file at https://github.com/Tantacom/tantacom-web/pull/5/files#diff-354f30a63fb0907d4ad57269548329e3R24

It looks like drupal is somehow installed two times... one under /home/travis/build/Tantacom/tantacom-web/ and the other /home/travis/build/Tantacom/drupal-8/drupal/

any ideas of what could i be doing wrong?

Inside before_script:

before_script:
  - drupal-ti before_script
  - drupal-ti --include drupal_ti/before/before_script.sh
  - drupal-ti --include drupal_ti/before/runners/behat/before_script.sh

And I have a file like this drupal_ti/before/runners/behat/before_script.sh:

#!/bin/bash

# Add an optional statement to see that this is running in Travis CI.
echo "running drupal_ti/before/runners/behat/before_script.sh"

set -e $DRUPAL_TI_DEBUG

export BEHAT_PARAMS='{"extensions":{"Behat\\MinkExtension":{"base_url":"DRUPAL_TI_WEBSERVER_URL:DRUPAL_TI_WEBSERVER_PORT/"},"Drupal\\DrupalExtension":{"drush":{"root":"DRUPAL_TI_DRUPAL_DIR"},"drupal":{"drupal_root":"DRUPAL_TI_DRUPAL_DIR"}}}}'
BEHAT_PARAMS=`echo $BEHAT_PARAMS | sed -e "s#DRUPAL_TI_WEBSERVER_URL#$DRUPAL_TI_WEBSERVER_URL#" -e "s#DRUPAL_TI_WEBSERVER_PORT#$DRUPAL_TI_WEBSERVER_PORT#" -e "s#DRUPAL_TI_DRUPAL_DIR#$DRUPAL_TI_DRUPAL_DIR#" -e "s#DRUPAL_TI_DRUPAL_DIR#$DRUPAL_TI_DRUPAL_DIR#"`
export BEHAT_PARAMS
echo $BEHAT_PARAMS

But it seems the BEHAT_PARAMS is not considered when the tests are run. And it fails with error:

[InvalidArgumentException]          
Driver "drupal" is not registered.

Any idea why this is happening?

As a workaround I am setting BEHAT_PARAMS under before_script like this:

  # Setting Behat environment
  - DRUPAL_DIR="$TRAVIS_BUILD_DIR/../$DRUPAL_TI_ENVIRONMENT/drupal"
  - BEHAT_PARAMS='{"extensions":{"Behat\\MinkExtension":{"base_url":"DRUPAL_TI_WEBSERVER_URL:DRUPAL_TI_WEBSERVER_PORT/"},"Drupal\\DrupalExtension":{"drush":{"root":"DRUPAL_DIR"},"drupal":{"drupal_root":"DRUPAL_DIR"}}}}'
  - BEHAT_PARAMS=`echo $BEHAT_PARAMS | sed -e s#DRUPAL_TI_WEBSERVER_URL#$DRUPAL_TI_WEBSERVER_URL# -e s#DRUPAL_TI_WEBSERVER_PORT#$DRUPAL_TI_WEBSERVER_PORT# -e s#DRUPAL_DIR#$DRUPAL_DIR# -e s#DRUPAL_DIR#$DRUPAL_DIR#`
  - export BEHAT_PARAMS

Pointing you towards the solution we used to fix the rate limitation failures:
https://www.drupal.org/node/2581389#comment-10492734

It is getting crowded for the environment variables. Ensure that things like

DRUPAL_TI_BEHAT_SCREENSIZE_COLOR 1280x1024x16

etc.

can be specified by runners/behat like defaults/behat.sh:

drupal_ti_default_configuration DRUPAL_TI_BEHAT_SCREENSIZE_COLOR "1280x1024x16"

and commented out in .travis.yml, so that defaults can be specified easily by the runners.

This did not make it into the behat branch:

• Support phantomJS, e.g. like https://github.com/Gizra/negawatt-server/blob/master/.travis.yml

Seeing this in commerce tests:

https://travis-ci.org/joshuataylor/commerce/jobs/81239645

/home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh: line 13: ./vendor/bin/phpunit: No such file or directory
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh exited with a failure.

• Need to copy simpletest setup steps, the functions are re-entrant anyway.
• No need for a web-server though.

1.4.2 was released some time ago, and master contains some fixes that are necessary because of newer Drush releases. @LionsAd asked me to ping him in this issue.

In drupal-7.sh and drupal-8.sh there is a command which run drush in the following way:

~/.composer/vendor/bin/drush.php

so I've the following error:

php -d sendmail_path=/bin/true /home/travis/.composer/vendor/bin/drush.php --yes site-install testing --db-url=mysql://root:@127.0.0.1/drupal
Could not open input file: /home/travis/.composer/vendor/bin/drush.php

Since I'd like to have my drush available globally as I'm using drupal_ti on top of my other CI tests, I've specified the following environment variable: COMPOSER_BIN_DIR=~/bin which automatically link drush binary into ~/bin which is available by default in Travis CI by just running drush (so I don't need to modify my PATH). This way is also described in drush docs.

So I wondering if there is a better way of running drush. It seems this syntax is only used in order to handle mails (php -d sendmail_path=$(which true)).

In my .travis.yml, my approach is a bit different (by installing mailcatcher, so there can be some tests related to received e-mails):

 - gem install mailcatcher && mailcatcher -v
 - echo 'sendmail_path="/usr/bin/env catchmail"' | tee -a "$(php --ini | grep "Loaded Configuration" | awk '{print $4}')"

In this case, we can install site in normal way, which is: drush --yes site-install, so we're not depending on any hardcoded paths and mails are handled by catchmail.

When a module has a dependency on composer_manager, the build fails. It is basically stuck at this point:

The following projects have unmet dependencies:                      [ok]
url_embed requires composer_manager
Would you like to download them? (y/n): y
Install location /home/travis/build/drupal-media/drupal-8/drupal//modules/composer_manager already exists. Do you want to overwrite it? (y/n): 
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

Link to the relevant build in case you want to check: https://travis-ci.org/drupal-media/url_embed/jobs/70604609#L208

Currently, "before_script" performs the following actions:

1. Drupal core dl
2. Drupal site install
3. (Github) Module install

Most 8.x module dependencies do not have releases on drupal.org yet. As such, step 3 fails because module dependencies are not found. There have also been known Drush issues with downloading and discovering dependencies (which is out of your hands).

I would develop drupal-ti before_script to call drupal-ti site-install (steps 1 and 2 noted above) and drupal-ti module_install (steps 3 above).

This enables a user to call drupal-ti before_script if they have no issues with dependencies, or they can call drupal-ti site-install, their own logic, and then drupal-ti module_install if they need more customization.

It'd be really nice if a Drush alias was created and 'drush use'd so that interactions with the Drupal site could be done without the need of navigating the filesystem.

I am using travis to test changes to my Drupal module before merging them back into 8.x-1.x.

However I think that since environments/drupal-8.sh drupal_ti_ensure_module_linked() now gets the module from composer using '*@dev', rather than using the commit of the module that I am actually testing, it always gets a release version (eg alpha-1) from drupal.org.

Why is this better than using drupal_ti_ensure_module_linked() as defined in functions/drupal.sh which uses the commit checked out by travis?

Travis build : https://travis-ci.org/naveenvalecha/autologout/jobs/110087030

Travis yml file : https://github.com/naveenvalecha/autologout/blob/8.x-1.x/.travis.yml

Issue #2551981: Add --directory option to run-tests.sh test discovery.

This is not in Drupal core, but it could potentially cause issues. Just pinging you here so you're aware of the issue.

I don't know if this is due to an recent changes or the release of Drupal 8, but it appears that 'drush dl' commands are working inconsistently between different PHP versions. The obvious reason for that of course is the version of Drush being installed on 5.3 is different to those above.

The issue specifically is when I need to add a test dependency via Drush for a Drupal 7 module, 'drush dl' appears to be getting the Drush 8 release instead of the Drupal 7.

On this, it does look like it's an indicator that the 'drush use' stuff that I implemented previously isn't entirely working correctly, so I will have a look into that as soon as possible.

See the following test log to see the issue in practice: https://travis-ci.org/Decipher/file_aliases/jobs/93308484#L773

A simple workaround is to specify the version of the module in the 'drush dl' command: https://travis-ci.org/Decipher/file_aliases/jobs/93419634#L247

DRUPAL_TI_DB is required even though already present in DRUPAL_TI_DB_URL.

In addition to that travis can create a DB itself, too, now.

Original report

I've got the following error in this build:

$ drupal-ti before_script
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/simpletest/before_script.sh exited with a failure.

My .travis.yml file.

Installed via: composer global require -n --prefer-source lionsad/drupal_ti:1.*

I've been playing with Drupal Ti for a few hours now, and have mostly got to where I want to be, however I am having one issue that I can't really seem to figure out.

My Behat tests using DrupalExtention seem to be able to communicate with the Drupal API. On my local tests it runs fine, but on Travis CI I'm not getting any luck.

Specifically, what I'm doing in Behat is running module_enable() in a @BeforeSuite hook to turn on a test module, but I'm getting the following error on Travis:

PHP Fatal error:  Call to undefined function module_enable() in /home/travis/build/Decipher/test/tests/behat/features/bootstrap/WysiwygFieldsFeatureContext.php on line 14

https://travis-ci.org/Decipher/test/jobs/73371096#L443

You can see my test repo (and it's excessive test commits) at https://github.com/decipher/test

I have a Drupal module that relies on Composer Manager module and the Composer extension for Drush.

Everything goes well till after the Composer extension is installed for Drush. I need to find how I can inject a clear cache on drush after the extension loads.

Here is the travis log output

Download and install the Drush Composer extension? (y/n): y
Project composer (8.x-1.x-dev) downloaded to [success]
/home/travis/.drush/composer.
Project composer contains 0 modules: .
The drush command 'composer' could not be found. [error]

Hello

I'm using drupal_ti to test my Drupal module (https://github.com/Ozmodiar/DIC) using PHPUnit and Simpletest at the same time but when following the installation instructions, something went wrong.

$ composer global require "lionsad/drupal_ti:1.*"
Changed current directory to /home/travis/.composer
./composer.json has been created
Package "lionsad/drupal_ti" listed for update is not installed. Ignoring.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Generating autoload files

$ drupal-ti before_install
/home/travis/build.sh: line 41: drupal-ti: command not found
The command "drupal-ti before_install" failed and exited with 127 during .
Your build has been stopped.

My temporary solution is to add a few extra composer (global) updates, such as:

before_install:
  - composer self-update
  - composer global require "lionsad/drupal_ti:1.*"
  - composer global update
  - drupal-ti before_install

install:
  - drupal-ti install

before_script:
  - composer global update
  - composer update --dev
  - drupal-ti before_script

Is there a better solution to do this?

https://www.drupal.org/node/2489956 suggests

$ cd core
$ export SIMPLETEST_DB=mysql://root:@localhost/dev_d8; ./vendor/bin/phpunit --testsuite=kernel

I am currently using phpunit-core with SIMPLETEST_DB set in .travis.yml like this:

script:
  - drupal-ti --include .travis-phpcs.sh
  - export SIMPLETEST_DB=mysql://root:@127.0.0.1/drupal_travis_db; drupal-ti script

which, honestly, I don't really like but it works for now . :)

I was thinking about two options:

1. Update script.sh in phpunit-core to set environment variable correctly.
2. Create new runner for this kind of tests.

Which approach is correct?

Since a couple of days my build is errored each times, it seems that it's a requirement issue from drush :

Changed current directory to /home/travis/.composer

You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug

./composer.json has been updated

Loading composer repositories with package information

Updating dependencies (including require-dev)

Your requirements could not be resolved to an installable set of packages.

  Problem 1

    - Installation request for drush/drush dev-master -> satisfiable by drush/drush[dev-master].

    - drush/drush dev-master requires codegyre/robo ~1.0.0-beta1 -> satisfiable by codegyre/robo[1.0.0-beta1] but these conflict with your requirements or minimum-stability.

Installation failed, reverting ./composer.json to its original content.

Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/simpletest/install.sh exited with a failure.

The command "drupal-ti install" failed and exited with 1 during .

Your build has been stopped.

Don't have time to dig now, will do tomorrow.

full stack is visible here : https://travis-ci.org/mespronos/mespronos/jobs/129421306

My module requires a library (in vendor) that is installed from the project composer in this way:

{
  "repositories": [
    {
      "type": "package",
      "package": {
        "name": "phpoffice/phpspreadsheet",
        "version": "0.0",
        "dist": {
          "url": "https://github.com/PHPOffice/PhpSpreadsheet/archive/develop.zip",
          "type": "zip"
        }
      }
    }
  ],
  "require": {
    "phpoffice/phpspreadsheet": "~0.0"
  },
  "autoload": {
    "psr-4": {
      "PhpOffice\\PhpSpreadsheet\\": "vendor/phpoffice/phpspreadsheet/src/PhpSpreadsheet"
    }
  }
}

I guess is easy to download and place the code in vendor/phpoffice/phpspreadsheet, but how to fix the autoload?

Hi Fabian,

Hopefully you can help me out with something. In epically large amount of test commits based repo, http://github.com/Decipher/test, I'm trying to get some behat tests for my Wysiwyg Fields module sorted out.

The tests work on my local machine with both selenium/firefox and with phantomjs, but no matter what I do, I can't get them to work on Travis.

The major complexity is that they are CKEditor dialog based tests, which requires timing. I have take two approaches, both which work locally, one using the DrupalExtension 'wait for AJAX' step, and the current with a custom 'wait till :element visible' approach. The first approach just failed outright on Travis, whereas the current just sits there until I kill the test (as it's a loop with no timeout).

If there's anything you can think of I would greatly appreciate it.

Somewhat related to #25, currently the installation profile is hardcoded, but it would be nice if we would set it in the .travis.yml file.

Relatively simple issue, PR coming when I get the time to write it.

Hi, this might be me not entirely understanding the travis test stack properly, but I have a request.

I would like to be able to run all drupal simpletests with my module enabled when I push code to a repo. This is so as to make sure that my module does not break core functionality.

As this takes hours however, I would like to only run tests that my module provides when creating pull requests, so that we don't have to spend hours waiting for tests before merging code.

Drupal_ti uses the DRUPAL_TI_SIMPLETEST_GROUP variable to tell the simpletest runner which tests to limit by, and TRAVIS_PULL_REQUEST tells me whether on not I want the runner to filter tests.

I've attempted to change the variable like this:

if [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then
  echo 'Scheduling all simpletests to be run.'
else
  echo 'Scheduling my_module simpletests to be run.'
  export DRUPAL_TI_SIMPLETEST_GROUP='my_module'
fi

But this doesn't seem to work as by the time the simpltest runner 'script.sh' runs, it has reset DRUPAL_TI_SIMPLETEST_GROUP to whatever is set in the .travis.yml.

Is there any way that I can achieve this without patching the runner script as part of my environment setup?

Document new parameter from PR

Having an example of how to use https://github.com/squizlabs/PHP_CodeSniffer or https://github.com/FriendsOfPHP/PHP-CS-Fixer would be nice for module:

Here's what we had in our .travis.yml file for a d8 module before switching to drupal_ti:

env:
  global:
    PHPCS_VERSION='2.0.*@dev'
    CODER_VERSION='8.2.0-alpha2'
before_install:
  composer global require squizlabs/php_codesniffer:$PHPCS_VERSION
  composer global require drupal/coder:$CODER_VERSION
  ln -s ~/.composer/vendor/drupal/coder/coder_sniffer/Drupal ~/.composer/vendor/squizlabs/php_codesniffer/CodeSniffer/Standards/
script:
  phpcs --report=full --standard=Drupal $TRAVIS_BUILD_DIR/drupal/modules/$MODULE_NAME

Running phpunit tests should not require a Drupal installation as there should be no database state for those tests. It should only need to download Drupal.

Solution should be to decouple download and install tasks from the ensure drupal and ensure module functions.

Unlike unit tests, kernel tests are added to the right test group.

That means they're run twice, once with phpunit and once with run-tests.sh

I'm having some trouble getting Behat to run in a contrib module for D8: stevector/template_mapper#2 My last couple of commits have been switching the api driver between drush and drupal. I'm getting different errors with each.

Do you know of any contrib modules using drupal_ti that I could use as a comparison? I've looked at Panopoly but it seems like a much more advanced use case.

.travis.yml.dist defines DRUPAL_TI_MODULE_NAME, which describes the project name, but not any specific module, since a project can contain multiple modules. In addition, DRUPAL_TI_SIMPLETEST_GROUP can probably contain multiple group names, but that's not really clear.

I've just committed a change to composer_manager that makes it use the root vendor directory instead of the core one: https://www.drupal.org/node/2452781#comment-10240975

Having both vendor/ and core/vendor creates a weird situation for phpunit.
I can run phpunit from the root using

./vendor/bin/phpunit -c core

But I can no longer run phpunit from core/vendor/bin/phpunit, because it ends up including both autoloaders somehow, and crashes with:

PHP Fatal error: Cannot redeclare composerRequireDrupal8() (previously declared in >/home/travis/build/commerceguys/drupal-8/drupal/core/vendor/composer/autoload_real.php:56) in >/home/travis/build/commerceguys/drupal-8/drupal/vendor/composer/autoload_real.php on line 55

This is problematic because simpletest hardcodes the core phpunit, making the build fail.
Not yet sure what the proper fix is, wanted to report it first.

I've created an icon for this project as part of a presentation I'll be giving during a DrupalCamp in Belgium. Feel free to use it!