tripal / tripaltestsuite

Handle drupal bootstrapping, load environment variables, and easily wrap your tests in DB transactions.

Tripal Test Suite

TripalTestSuite is a composer package that handles common test practices such as bootstrapping Drupal before running the tests, creating test file, creating and managing database seeders (files that seed the database with data for use in testing) and much more.

Installation

Within your Drupal module path (e,g sites/all/modules/my_module), run the following.

composer require statonlab/tripal-test-suite --dev

This will install TripalTestSuite along with all of the dependencies.

Usage Documentation

Please visit our online documentation to learn about installation and usage.

License

TripalTestSuite is licensed under GPLv3.

tripaltestsuite's Issues

make:test command needs to trim the namespace

Creating a test in the main tests dir results in the namespace being:

namespace Tests\;

It needs to be:

namespace Tests;

Database Seeders vs. Factories

What is the difference between Database Seeders and Factories? All I could gleam from the README is that both are re-usable ways to add data for tests and that factories can be used in database seeders.

Which raises my next question, when should I use factories vs. database seeders?

PS. For context, I'm adding some simple tests to Tripal BLAST :-D

make:test isn't figuring out the namespace correctly

make:test should remove _, make dirs as necessary, and capitalize words in namespace only but not path

Create Symfony Commands

We need a file tripaltest that runs a symfony console and refactor the current commands to work with tripaltest init

Add docs to make:test

We need to tell the user to not specify tests dir and not to include .php

Database Seeders return the content they created?

Is there a way to get a Database Seeder to return the content it created?

My specific use case: I'm testing updating a blastdb node. So I create a test node, fill in the form with the values for that node, change a few, submit and check they changed. Ideally the test node would be created by a database seeder since it's more complex then adding to a table and will be needed for almost all tests of the blast module.

Currently, I can figure out how to create the node in the database seeder but returning seems to do nothing. I don't want to load the node in the test because then I'm testing both load and update.

My code

Database Seeder

namespace Tests\DatabaseSeeders;
use StatonLab\TripalTestSuite\Database\Seeder;
class BlastDBNodeSeeder extends Seeder {

    /**
     * Seeds the database a test blast database node.
     *
     * @return void
     */
    public function up() {

      // Log in the god user.
      global $user;
      $user = user_load(1);

      $node = new \stdClass();
      if (!isset($node->title)) $node->title = 'Test Blast Database';
      $node->type = 'blastdb';
      node_object_prepare($node);

      $node->language = LANGUAGE_NONE;
      $node->uid = $user->uid;
      $node->status = 1;  // published.
      $node->promote = 0; // not promoted.
      $node->comment = 0; // disabled.

      if (!isset($node->db_name)) $node->db_name = 'Test Blast Database';
      if (!isset($node->db_path)) $node->db_path = '/fake/path/here';
      if (!isset($node->db_dbtype)) $node->db_dbtype = 'nucleotide';
      if (!isset($node->dbxref_linkout_type)) $node->dbxref_linkout_type = 'none';
      if (!isset($node->cvitjs_enabled)) $node->cvitjs_enabled = 0;

      $node = node_submit($node);
      node_save($node);

      // log out the god user.
      $user = drupal_anonymous_user();

      return $node;
    }
}

Test

  /**
   * Update an existing Blast Database Node.
   */
  public function testBlastDBNodeUpdate() {

    // Log in the god user.
    global $user;
    $user = user_load(1);

    // Create the node in the first place.
    $node = DatabaseSeeders\BlastDBNodeSeeder::seed();
    print_r($node);

    // Now use the form to edit it :-)
    // Specifically, we will change the name and type.
    $form_state = array(
      'values' => array(
        'db_name' => 'Test Protein Blast Database',
        'db_path' => $node->db_path,
        'db_dbtype' => 'protein',
        'dbxref_linkout_type' => $node->dbxref_linkout_type,
        'cvitjs_enabled' => $node->cvitjs_enabled,
        'op' => t('Save'),
      ),
    );

    // Execute the node creation form.
    drupal_form_submit('blastdb_node_form', $form_state, $node);

    // Retrieve any errors.
    $errors = form_get_errors();
    print_r($errors);

    // Assert that there must not be any.
    $this->assertEmpty($errors);

    // Check that there is a test blast database.
    $result = db_query('SELECT * FROM {blastdb} WHERE name=:name AND dbtype=:type',
      array(':name' => $form_state['values']['db_name'], ':type' => $form_state['values']['db_dbtype']));
    $this->assertEquals($result->rowCount(), 1);

    // log out the god user.
    $user = drupal_anonymous_user();

  }

Default feature factory fails

There was 1 error:

1) entity_child_cvterm_test::tests_update_all_entities
LengthException: Cannot get 125 elements, only 4 in array

/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/vendor/fzaninotto/faker/src/Faker/Provider/Base.php:177
/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/vendor/fzaninotto/faker/src/Faker/Generator.php:222
/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/vendor/fzaninotto/faker/src/Faker/Generator.php:279
/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/tests/DataFactory.php:72
/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/vendor/statonlab/tripal-test-suite/src/Database/Factory.php:99
/Users/chet/UTK/tripal/sites/all/modules/custom/tripal_cv_xray/tests/ChadoEntityChildCvtermTest.php:68

  'residues' => implode('', $faker->randomElements(['A','C','T','G'], 125))

Add browser tests support

use Facebook Chrome Driver to create browser tests.

Use readthedocs

Uodate docs to use readthedocs

Gist of chado factories

I'll try to create gist with most major chado factories.

When we put into core, we can then roll these factories out to showcase their utility and quickly write some good tests for a targeted piece of chado API.

faker names should be faker words or texts

default seeder has no down method, breaks second run of db:seed

vendor/bin/tripaltest db:seed
Ran Tests\DatabaseSeeders\DevSeedSeeder successfully!

In database.inc line 2227:

  SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "users_name_key"
  DETAIL:  Key (name)=(test user) already exists.


db:seed [<name>]

fix by adding down method or commenting out seed

The publish method fails

When ids are specified we get this error:

PDOException: SQLSTATE[23502]: Not null violation: 7 ERROR:  null value in column "entity_id" violates not-null constraint
DETAIL:  Failing row contains (267343, 66792, XRAY_TEST, 0000001, null).

/Users/Almsaeed/Work/DevSites/Tripal/includes/database/database.inc:2227
/Users/Almsaeed/Work/DevSites/Tripal/includes/database/pgsql/database.inc:106
/Users/Almsaeed/Work/DevSites/Tripal/includes/database/pgsql/query.inc:96
/Users/Almsaeed/Work/DevSites/Tripal/sites/all/modules/tripal_cv_xray/tests/AdminXrayFormTest.php:110
/Users/Almsaeed/Work/DevSites/Tripal/sites/all/modules/tripal_cv_xray/tests/AdminXrayFormTest.php:66

Where should DB transactions wrap: class or function?

https://phpunit.de/manual/current/en/fixtures.html

setUp and tearDown run at each function call.

There is also setUpBeforeClass()

Where would we expect our db transactions to rollback - each function or for the whole class?

Helper function to allow easy command line loading

This is an ideas discussion thread for how to allow more automated data loading with the dev seed project in mind (and perhaps this discussion/utility will end up there instead).

Constraints

• Loaders are never bootstrapped, they must be manually loaded

• I think the easy way to run the loader is to provide a simulated $form and $form_state as required by the Importer. This is because some private methods move the file to temp etc, and the functions used to actually run the loader might be private.

• I'd love to use a drush command, but passing a fake form isn't really possible with Drush -> you'd have to write a seperate drush command for each loader to allow the options to be passed in.

• Could a db:seeder be better? Lets you run it via the command line, assuming you can pass a file path and whatever options needed.

can we use factories in seeders? Nested seeders still run

factory('chado.dbxref')->create(['db_id' => 1]); The returned record is associated to db_id 1 but the additional fake db still gets created and you probably don’t want that

factories not auto cleaned up with db transactions

Problem:

MY DB is flooded with fake CVs and organisms after using factories.

To reproduce:

In my ctest class, I invoke transactions like so:

<?php

use StatonLab\TripalTestSuite\TripalTestCase;
use StatonLab\TripalTestSuite\DBTransaction;

class entity_child_cvterm_test extends TripalTestCase {

  use DBTransaction;

  static $obj;

In my test, I have:

$organism = factory('chado.organism')->create();
     $features =  factory('chado.feature', 100)->create();

I don't specify anything in setUp or breakdown but i dont think I need to?

.env comments and white lines

when reading .env file, trim lines and make sure that:

1. They don't begin with any of the following characters #, //,--
2. They are not empty

Possible new commands

• tripaltest init
• tripaltest make:test testName
• tripaltest make:seeder TableSeeder

MODULE_NAME variable not properly replaced in .travis.yml

travis builds fail out of the box beacuse the variable is messed up.

call to undefined function class_uses_recursive()

Runtime:       PHP 7.1.14
Configuration: /var/www/html/sites/all/modules/tripal_manage_menus/phpunit.xml

EE                                                                  2 / 2 (100%)

Time: 637 ms, Memory: 28.00MB

There were 2 errors:

1) Tests\ExampleTest::testBasicExample
Error: Call to undefined function StatonLab\TripalTestSuite\class_uses_recursive()

/var/www/html/sites/all/modules/tripal_manage_menus/vendor/statonlab/tripal-test-suite/src/TripalTestCase.php:21

2) Tests\MenuTest::testMenuUpdate
Error: Call to undefined function StatonLab\TripalTestSuite\class_uses_recursive()

/var/www/html/sites/all/modules/tripal_manage_menus/vendor/statonlab/tripal-test-suite/src/TripalTestCase.php:21

ERRORS!
Tests: 2, Assertions: 0, Errors: 2.

Got this error from the test created from ./vendor/bin/tripaltest make:test MenuTest. This is the error produced after using ./vendor/bin/phpunit.

upgrade instructions

we need simple upgrade instructions in the README.

search path needs some finesse

PDOException: SQLSTATE[42883]: Undefined function: 7 ERROR:  function create_point(integer, bigint) does not exist
LINE 1: SELECT box (create_point(0, $1), create_point($2,500000000))
                    ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
QUERY:  SELECT box (create_point(0, $1), create_point($2,500000000))
CONTEXT:  SQL function "boxrange" during inlining

currently get this error when running the 'chado.featureloc', factory,

probably need to set search_path to public, chado; which we can do manually or better run the trpal function that sets the searchpath for you.

dbseeders

tripal/tripal#311
A seeder base class.

Yeah add a Seeds folder in the tests folder where we auto detect and load the seeders for use in setup and teardown fns

We can leave it up to tripal core, or to the devseed project, to provide the seeders.
A test class just says which seeders it requires and the db gets seeded prior to each test.

Add to gitignore

Let's load in the file and scan for "vendor/*" and add it in if it doesnt exist.

running out of memory with silent

Silent is only for testing shorter strings. Wrapping several importers causes memory to run out!

PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 12288 bytes) in /Users/chet/UTK/tripal/includes/database/database.inc on line 2227
#0  StatonLab\TripalTestSuite\TripalTestBootstrap::shutdownHandler()

Solution:

• add statement to readme that silent is only for testing smalelr strings
• improve mem management?

add to http://tripal.info/extensions

no analysis factory despite devseedSeeder using it

In the past you've said you want factories only available as gists but im going to again lobby that we (me) provide factories for all base chado tables.

testing importers

namespace tripal_cmap_loader;


class CmapImporter extends TripalImporter {

So im trying to test my importer. I am namespacing my importer like so:

then in my test:

namespace Tests;

use StatonLab\TripalTestSuite\TripalTestCase;
use tripal_cmap_loader;

/**
 * Class ExampleTest
 *
 * Note that test classes must have a suffix of Test.php and the filename
 * must match the class name.
 *
 * @package Tests
 */
class CmapImporterTest extends TripalTestCase
{
    public function testInit()
    {
$this->assertTrue(TRUE);

$importer = new tripal_cmap_loader\CmapImporter();

I keep getting Error: Class 'tripal_cmap_loader\CmapImporter' not found with a variety of namespace combinations.

Release for DOI

All programs will have a DOI for publication. Zenodo integration already added.

faker data can get stranded if seeder or test exits instead of returning and transaction doesnt roll back

idea: what if we made everything that faker creates use a special prop for easy deletion
ie the fake base class inserts into [tablename]prop of term local:faker_inserted_data value = true

possible downside: things inserted with a seeder might want to be persistent.

update README documentation for command line setup

Update README

Once commands are added, we need to update the docs in readme

Feature request: A "a set up" call

the set up would automatically create the

• test directory
• .env file
• travis config

anything else?

php 5 support

We should consider how to support php 5 for the developers who cannot or will not upgrade to php 7

example issue

jlkjlkj

Syntax error

This line defines mRNA as a constant whereas it should be a string

https://github.com/statonlab/TripalTestSuite/blob/e469f7fde2cf69303dc22315b142b46b027c9931/stubs/DevSeedSeeder.php#L227

documentation for uncommenting the default devseed

Create reflect() method

the reflect helper function should return a Reflection object that allows users to access private and protected properties and methods of a given object.

Defenition:
function reflect(String $class_name) : object;

HTTP tests are affected by DBTransaction

Currently, HTTP requests use Guzzle to send an actual request to the site. This is a problem since that means anything in a transaction is not reflected in the DB when sending a request.

So something like this:

   $genus_string = 'a_genius_genus';
    $species_string = 'fake_species';
    $sub_name = substr($genus_string, 0, 1);

    $organism = factory('chado.organism')->create([
      'genus' => $genus_string,
      'species' => $species_string,
    ]);


    $response = $this->get("/admin/tripal/storage/chado/auto_name/organism/$sub_name");
    $response->assertSee("$organism->genus $organism->species");

Would fail although the code is actually correct.

Solution, mock a request by going through the Drupal kernel instead of using guzzle.

Update docs to remove the down method in seeders

Seeders no longer support the down method but the docs were not updated to reflect that.

Update Docs: How to use DatabaseSeeder in Test

I'm currently writing a tripal test that calls a database seeder within it. There is documentation on how to do that here: https://tripaltestsuite.readthedocs.io/en/latest/db-seeders.html#running-seeders

You can also run the seeder manually by using the static seed() method. For example, within a test class, you can run $seeder = UsersTableSeeder::seed() which runs the up() method and returns an initialized seeder object. If you are using the DBTransaction trait, the data will be automatically rolled at the end of each test function.

However, following those instructions as is produces a class not found error:

PHP Fatal error:  Class 'Tests\tripal_chado\fields\UsersTableSeeder' not found in /var/www/sites/all/modules/contrib/tripal/tests/tripal_chado/fields/ChadoFieldGetValuesListTest.php on line 24
#0  StatonLab\TripalTestSuite\TripalTestBootstrap::shutdownHandler()

Instead the instructions should tell us to use the correct namespace as follows: $seeder = \Tests\DatabaseSeeders\UsersTableSeeder::seed()

Create silent() method

The silent helper function should create a buffer to collect all output of a passed Callable.

Definition:
function silent(Callable $callable) : Object;

Object should have the following assertions:

• assertSee(String $string) : PHPUnit::assert(). Checks if a certain string exists in the output.
• assertJsonStructure(Array $array) : PHPUnit::assert(). Checks if the output matches a given json structure.

Example Usage:

$output = silent(function() {
            echo "Some output";
          });
$output->assertSee('Some output');

setup force flag for upgrading

from #61

If you use the default configuration for PHPUNIT and travis, and we mae changes to those files, you wont get them when updating.

We should have a setup flag -f that force-overwrites these files, and hte instructions can let people know if they are ok with the defaults to rerun setup with that force flag.

out of the box does not pass tests

https://travis-ci.org/statonlab/tripal_analysis_kegg

two build issues:

• enabling the module fails (?) for both kegg and ssr.
• No tests in test classes/extra test classes read in (see below)

WW                                                                  2 / 2 (100%)
Time: 31 ms, Memory: 4.00MB
There were 2 warnings:
1) Warning
No tests found in class "StatonLab\TripalTestSuite\TripalTestCase".
2) Warning
No tests found in class "Tests\MyTest".
WARNINGS!
Tests: 2, Assertions: 0, Warnings: 2.
The command "docker exec -it tripal bash -c 'cd /var/www/html/sites/all/modules/custom/tripal_analysis_kegg/ &&composer install && DRUPAL_ROOT="/var/www/html" ./vendor/bin/phpunit'" exited with 1.```

seeders dont have proper access to factories

To produce issue, try to run the below seeder

<?php

namespace Tests\DatabaseSeeders;

use StatonLab\TripalTestSuite\Database\Seeder;

class UsersTableSeeder extends Seeder
{
    /**
     * Seeds the database with users.
     */
    public function up()
    {
        $new_user = [
            'name' => 'test user',
            'pass' => 'secret',
            'mail' => '[email protected]',
            'status' => 1,
            'init' => 'Email',
            'roles' => [
                DRUPAL_AUTHENTICATED_RID => 'authenticated user',
            ],
        ];

      $organism = factory('chado.organism')->create([
        'common_name' => $this->organism_name,
        'genus' => 'Fraxinus',
        'species' => 'excelsior',
        'abbreviation' => 'F. excelsor',
        'comment' => 'The Tripal Dev Seed miniature dataset.',
      ]);

      $this->organism_id = $organism;
      var_dump($organism);



      // The first parameter is sent blank so a new user is created.
        user_save(new \stdClass(), $new_user);
    }
}

resulting error:

vendor/bin/tripaltest db:seed

In Factory.php line 54:

  Attempt to use an undefined factory of type chado.organism.


db:seed [<name>]

Testing Drupal pages: Help Please :-)

Hi, I would like to test a specific Drupal page (Http Requests) but it requires a privileged user to be logged in. How would I go about that?

For example, I want to test the node creation form: [mysite]/node/add/blastdb. This is what I have so far:

  /**
   * Test Creating a BlastDB Node.
   */
  public function testBlastDBNodeCreate() {

    // Go to the node creation page.
    $response = $this->get('/node/add/blastdb');

    // Verify you can see the page.
    $response->assertStatus(200)
             ->assertSee('Create Blast Database');
  }

and this is what I get

./vendor/bin/phpunit
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.7-1
Configuration: ./phpunit.xml

.F                                                                  2 / 2 (100%)

Time: 559 ms, Memory: 42.75MB

There was 1 failure:

1) Tests\BlastDBNodeTest::testBlastDBNodeCreate
Failed to send GET request to http://127.0.0.1/node/add/blastdb. Client error: `GET http://127.0.0.1/node/add/blastdb` resulted in a `404 Not Found` response:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found (truncated...)

./vendor/statonlab/tripal-test-suite/src/Concerns/MakesHTTPRequests.php:131
./vendor/statonlab/tripal-test-suite/src/Concerns/MakesHTTPRequests.php:32
./tests/BlastDBNodeTest.php:40

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

Test if a factory exists

How would I test if a factory exists before I use it in a test?

For example, say I have a dynamic variable containing the chado table, which can be set based on the dataProvider. Since there are not yet factories for all chado tables, I would need to check if the factory exists before creating fake data using it...

if (factory('chado.' . $chado_table)->exists()) {
  factory('chado.' . $chado_table, $num_records)->create();
}

Method request: register hook listener

In order for us to test hooks, we need a way to register a temporary hook that gets called when a hook is triggered.

This is simple to implement since it is just adding an item to the drupal cache then removing it by the end of the test.

test run of seeder

need to run the seeder at #57 on a clean site to verify everything works as expected