Hi,

I see that you are still on Travis CI here, which in my eyes are getting more and more difficult for Open Source projects.
I would be happy to migration your Travis CI to GitHub Actions if you like.

Just let me know.

When a functional test contains a failing test with assertEquals or assertSame, PHPUnit is unable to parse the output.

Shown Error:

ErrorException: Use of undefined constant PATH_typo3conf - assumed 'PATH_typo3conf' in /home/travis/build/derhansen/TYPO3-testing-framework/.Build/vendor/phpunit/phpunit/src/Util/PHP.php:298

How to reproduce:

1. Add the following test to the functional test suite

    /**
     * @test
     */
    public function failingTestWithAssertSame()
    {
        $this->assertSame('test1', 'test2');
    }

2. Push changes and let Travis CI build

See: https://travis-ci.org/derhansen/TYPO3-testing-framework/jobs/222008133

The currently used bootstrap mechanism for the core was deprecated in V9 and removed in V10.

#117

Given:

• Environment variable with upper case for database name like t3_xtb_DTAGBDEVAL108

Expectation:

• Functional tests are executed without (framework) errors

Actual:

• Functional tests fail with

Nimut\TestingFramework\Exception\Exception: Unable to create database with name t3_xtb_DTAGBDEVAL108_ft09fbfa7. This is probably a permission problem. For this instance this could be fixed executing: GRANT ALL ON `t3_xtb_DTAGBDEVAL108_%`.* TO `xxx`@`127.0.0.1`; Original message thrown by database layer: An exception occurred while executing 'CREATE DATABASE t3_xtb_DTAGBDEVAL108_ft09fbfa7':

Can't create database 't3_xtb_dtagbdeval108_ft09fbfa7'; database exists

MySQL enforces lower case for database when the database is created. With the change to Doctrine Api the "Drop Database" command is only executed if the given database name is within the available databases. But the comparison is done with the given database name (coming e.g. from ENV) with the returned list of existing databases. The results in searching for t3_xtb_DTAGBDEVAL108_ft09fbfa7 in a list where t3_xtb_dtagbdeval108_ft09fbfa7 exists. Thus the database is never found and never dropped.

Given processIsolation="true" in the PHPUnit config and a failing assertion the following error occurs:

PHPUnit 5.7.23 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 686 ms, Memory: 6.00MB

There was 1 error:

1) Vendor\Package\Functional\MyTest::doesWhatItShould
PHPUnit_Framework_Exception: PHP Fatal error:  Uncaught Object of class Nimut\TestingFramework\Database\Database could not be converted to string

  thrown in - on line 374

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

With a bit more debugging it seems like PHPUnit tries to serialize the whole testcase in order to return it to the parent process for displaying the test error.

Hi,

I have (nimut) phpunit testing implemented in my project. Recently I noticed that after the first test ran the linked test/fixture folder (by $pathsToLinkInTestInstance) has been deleted. So all the tests afterwards are failing due to missing fixtures.

After some research I found out that it is related to this bugfix. I guess this modification is logical.

But then again is there a way to prevent a teardown of the linked files/folders when not all tests from that class has been executed?

Thanks for the help

Hey,
When i run one test in test class and execute this test with phpunit, this test is running well.
After i add more tests or even one more test into this class and run this class again, an exception is thrown with message RuntimeException: Trying to override applicationContext which has already been defined!
I though, teardown should handle this problem but it doesn't.

Hello,

for one of my functional tests I have added static_info_tables to $this->testExtensionsToLoad. All records of ext_tables_static+adt.sql were added successfully, but after tearDown() of first test, all tables of static_info_tables will be truncated, so for all further tests I have to add the needed record on my own.

Would be helpful, I you could load ext_tables_static+adt.sql for each test case.

Current solution:

// Seems that records of ext_tables_static+adt.sql will be included just once for all tests in this class.
// So, for all tests (except the first one), we have to add the record ourselves.
$country = $this->getDatabaseConnection()->selectSingleRow('*', 'static_countries', 'uid=54');
if ($country === false) {
    $this->importDataSet(__DIR__ . '/../Fixtures/static_countries.xml');
}

Nice greetings

Stefan

In functional tests the getFrontendResponse method has an argument $frontendUserId which is passed though the query string. However, if i'm not overlooking something, that is never used anywhere. I tried using it and the request has no authorisation ($GLOBALS['TSFE']->fe_user->user === null).

Currently the requirements for PHP in composer.json is set to

"php": ">= 5.5, < 7.3"

so it can't be used in PHP 7.3.

TYPO3 9.4 was released yesterday, this package should be updated accordingly.

Hi there,

I am looking for an option to test real http requests like posting to a specific url and checking the database afterwards. Is there any option to make this happen with TYPO3/Extbase or the current testing-framework?

Cheers

Christian

I'm trying to setup a new testing environment for TYPO3 8.7.9 without success.
Could you provide additional information or hyperlinks how to create a valid composer.json which then will include yours a a reqirement.
Which directory should be then referenced by TYPO3_PATH_ROOT?

May I also ask for access to your slack workspace?

Thank you

On our project we are using helhum/typo3-secure-web package together with nimut/testing-framework, and we got an issue when running FunctionalTests together with frontend requests.
As we are using helhum/typo3-secure-web we have a private folder and public folder in the project. And we wrote a functional test to check if frontend request will response with the data we expect (we are using this: https://github.com/Nimut/testing-framework#frontend-requests), TYPO3_PATH_ROOT is set to the private folder, however /private/index.php file is empty, and because of that frontend answers with empty response. So looks like nimut/testing-framework doesn't expect a compatibility with helhum/typo3-secure-web.

Here is PR with possible solution: #111

If you do not add the property to your class, unset will be called in https://github.com/Nimut/testing-framework/blob/master/src/TestingFramework/TestCase/UnitTestCase.php#L59
Which removes the property in our PHP 5.6.33-1+ubuntu16.04.1+deb.sury.org+1 (cli). On all other testes environments the property is jut null after calling unset which is fine as casting to array results in an empty array.

So your proposed solution was to switch code blocks to first iterate over the property before unsetting, as discussed in Slack.

Hey,
It would be very useful to have an access to typoscript defined in setup.ts / setup.txt.
I've tried to get an object with this settings or even mock it but didn't find a way to make it possible.
Can you support this important feature or maybe give a sign how to make it happen?

Since TYPO3 9.2 initialization of backend routes has been moved to a PSR-15 middleware and is thus not done by Bootstrap::init() anymore as called by TestSystem::includeAndStartCoreBootstrap().

Due to this functional tests which rely on initialized routes will fail.

The testing framework should generally initialize the backend routes manually similarly to the v91 compat layer.

Is it possible to run tests against TYPO3 CMS 9.x-dev?

Are more changes needed as just increasing the maximal CMS version in composer.json?

Hi,

I have experienced a problem with nimut/testing-framework and PHP 8.0.14
When I run the PHPUnit command

$ TYPO3_PATH_WEB=$PWD/.Build/Web typo3DatabaseDriver=pdo_sqlite .Build/bin/phpunit -c .Build/vendor/nimut/testing-framework/res/Configuration/FunctionalTests.xml Tests/Functional

I run into an exception, thrown by https://github.com/Nimut/testing-framework/blob/main/src/TestingFramework/TestSystem/AbstractTestSystem.php#L729 as dbname isn't defined.

This is not a problem with PHP 7.4.

If I adjust my command to also include typo3DatabaseName=test it's working like expected.

$ TYPO3_PATH_WEB=$PWD/.Build/Web typo3DatabaseDriver=pdo_sqlite typo3DatabaseName=test .Build/bin/phpunit -c .Build/vendor/nimut/testing-framework/res/Configuration/FunctionalTests.xml Tests/Functional

I haven't found out what the reason for the different behaviour here is. But I would expect it to be something with a default value resolved differently in PHP 8.0.

I'll see if I find the issue and a solution. But for now I wanted to share the information in case others runs into the problem.

Error: Class 'Text_Template' not found

.Build/vendor/nimut/testing-framework/src/TestingFramework/TestCase/AbstractFunctionalTestCase.php:473
Tests/Functional/FrontendTest.php:39

Version: 6.x-dev

This is not a problem when running tests on Travis CI or another throw-away DB service, but for testing using a non-disposible MySQL instance (especially with shared instances).

A) The DB-user used for testing requires the right to create databases and self-grant all rights to it. By this, the user is root-like. This is inacceptible in shared DB instances and a risk to any other data stored in any DB on that instance.

B) For each test using the DB the framework creates a new DB copy with a random suffix:

mysql> show databases;
+----------------------+
| Database             |
+----------------------+
| information_schema   |
| mysql                |
| performance_schema   |
| typo3                |
| typo3_ft1d226aa      |
| typo3_ft53009df      |
| typo3_ft7054112      |
| typo3_ft9def141      |
| typo3_fta832130      |
| typo3_ftb9be75b      |
| typo3_ftc3170a5      |
| typo3_ftc799e14      |
| typo3_ftdcc4980      |
+----------------------+

And it never cleans up afterwards. Using random names makes it very annoying to cleanup manually.

The property $testExtensionsToLoad of FunctionalTestCase can be used to link extensions installed as dependencies into test systems of functional tests.

However, when executing functional tests for an extension itself, there is no way to have it linked and activated here since it doesn't exist as extension in the current project's <web-dir>/typo3conf/ext.

Currently I work around this by manually adding a link to the current project like this:

/**
 * @return void
 */
protected function setUp()
{
    $rootPath = dirname(__FILE__, 3);
    if (!is_link(ORIGINAL_ROOT . 'typo3conf/ext/my_extension')) {
        symlink($rootPath, ORIGINAL_ROOT . 'typo3conf/ext/my_extension');
    }
    parent::setUp();
}

(Assuming __FILE__ exists in Tests/Functional/FooTest.php)

Afterwards typo3conf/ext/my_extension can be added to $testExtensionsToLoad and it will be linked and activated.

Any idea how this could be improved?

… so that I can use the testing framework with PHP 7.2 Thanks! ☕

While trying to run some unit tests I get the following error:

PHPUnit 5.7.15 by Sebastian Bergmann and contributors.

EPHP Fatal error:  Uncaught exception 'RuntimeException' with message 'Could not acquire lock for ClassLoader cache creation.' in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php:761
Stack trace:
#0 /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php(247): TYPO3\CMS\Core\Core\ClassLoader->acquireLock()
#1 /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php(182): TYPO3\CMS\Core\Core\ClassLoader->buildCachedClassLoadingInformation('php_invoker', 'PHP_Invoker')
#2 [internal function]: TYPO3\CMS\Core\Core\ClassLoader->loadClass('PHP_Invoker')
#3 [internal function]: spl_autoload_call('PHP_Invoker')
#4 /.../vendor/phpunit/phpunit/src/Util/Blacklist.php(91): class_exists('PHP_Invoker')
#5 /.../vendor/phpunit/phpunit/src/Util/Blacklist.php(7 in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php on line 761

Fatal error: Uncaught exception 'RuntimeException' with message 'Could not acquire lock for ClassLoader cache creation.' in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php on line 761

RuntimeException: Could not acquire lock for ClassLoader cache creation. in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php on line 761

Call Stack:
    0.0001     253672   1. {main}() /.../vendor/phpunit/phpunit/phpunit:0
    0.0069    1338896   2. PHPUnit_TextUI_Command::main() /.../vendor/phpunit/phpunit/phpunit:52
    0.0069    1339672   3. PHPUnit_TextUI_Command->run() /.../vendor/phpunit/phpunit/src/TextUI/Command.php:118
    0.1401   11372544   4. PHPUnit_TextUI_TestRunner->doRun() /.../vendor/phpunit/phpunit/src/TextUI/Command.php:188
    0.1466   12014136   5. PHPUnit_Framework_TestSuite->run() /.../vendor/phpunit/phpunit/src/TextUI/TestRunner.php:521
    0.1474   12034184   6. PHPUnit_Framework_TestSuite->run() /.../vendor/phpunit/phpunit/src/Framework/TestSuite.php:728
    0.1510   12039528   7. PHPUnit_Framework_TestCase->run() /.../vendor/phpunit/phpunit/src/Framework/TestSuite.php:728
    0.1510   12041664   8. PHPUnit_Framework_TestResult->run() /.../vendor/phpunit/phpunit/src/Framework/TestCase.php:926
   30.4843   18764784   9. PHPUnit_Framework_TestResult->addError() /.../vendor/phpunit/phpunit/src/Framework/TestResult.php:872
   30.4847   18791360  10. PHPUnit_Util_Log_JUnit->addError() /.../vendor/phpunit/phpunit/src/Framework/TestResult.php:264
   30.4847   18791640  11. PHPUnit_Util_Log_JUnit->doAddFault() /.../vendor/phpunit/phpunit/src/Util/Log/JUnit.php:125
   30.4849   18813176  12. PHPUnit_Util_Filter::getFilteredStacktrace() /.../vendor/phpunit/phpunit/src/Util/Log/JUnit.php:451
   30.4852   18833440  13. PHPUnit_Util_Blacklist->isBlacklisted() /.../vendor/phpunit/phpunit/src/Util/Filter.php:69
   30.4852   18833584  14. PHPUnit_Util_Blacklist->initialize() /.../vendor/phpunit/phpunit/src/Util/Blacklist.php:74
   30.4852   18834496  15. class_exists() /.../vendor/phpunit/phpunit/src/Util/Blacklist.php:91
   30.4852   18834776  16. spl_autoload_call() /.../vendor/phpunit/phpunit/src/Util/Blacklist.php:91
   30.4853   18835432  17. TYPO3\CMS\Core\Core\ClassLoader->loadClass() /.../vendor/phpunit/phpunit/src/Util/Blacklist.php:91
   30.4853   18835560  18. TYPO3\CMS\Core\Core\ClassLoader->buildCachedClassLoadingInformation() /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php:182
   30.4853   18835608  19. TYPO3\CMS\Core\Core\ClassLoader->acquireLock() /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/ClassLoader.php:247

There is a lock file in $TYPO3_PATH_WEB/typo3temp/locks after this.

The current stable release 6.x of PHPUnit should be supported too.

Since TYPO3 9 requires php 7.2 it would be nice to have php 7.2 supported in the testing framework as well.

Currently PHPUnit 6 || 7 are supported, it would be good to also support the 8 release.
thanks :)

If a table definition is invalid, causing the table to not get created during setUp() of the functional tests, setUp silently ignores the error and continues, making finding the root of the problem hard.

Steps to reproduce:

1. clone https://github.com/oliverklee/tea/tree/poc/sql-table-creation-failed (from the "sql-table-creation-failed" branch)
2. composer install
3. run the functional tests

Actual result:
The setUp method fails to create
All tests from the TestimonialRepositoryTest fail with the following error message (indicating that the table does not exist in the database):

Table 'typo3_test_ft8aa5eba.tx_tea_domain_model_testimonial' doesn't exist

Expected result:
The tests already fail in the setUp() (at parent::setUp) with a message that the table cannot be created, and what the exact problem with the SQL is. This makes it a lot easier to find the cause of the problem.

I have create a empty test case with typo3 9.x with composer and ddev and iam getting an error:

Maybe the .env is not loaded here with the database config?

There was 1 error:

1) MyExt\MyExt\Controller\FunctionalTest::testCheck
Doctrine\DBAL\Exception\ConnectionException: An exception occurred in driver: No such file or directory

/var/www/html/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:93
/var/www/html/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:169
/var/www/html/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:157
/var/www/html/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/Mysqli/Driver.php:18

Currently only PHPUnit 6.x is supported, it would be good to also support the latest stable 7.x release.

Importing the static data of extensions (e.g., static_info_tables) can be really slow, and hence slowing down the functional tests immensely. I propose the TF either does not import static data at all, or to make this configurable.

Are there any plans on updating the Data Set/Fixture loading functions?

As for now the importDataSet() function of the FunctionalTestCase ist only able to load XML files. I'd like to use php files as well.

The PHPUnit framework already supports PHP, Csv and Yaml sources (see https://phpunit.de/manual/current/en/database.html, sections Array DataSet, CSV DataSet and YAML DataSet).
As I can see, you aren't using the database testing abilities from PHPUnit either. Maybe this part of this framework can be updated too.

I'd like to contribute to this project.
At first I thought about extending the importDataSet() to support PHP files, but this seems obsolete now. I'd rather update the whole database testing part and make it compatible with PHPUnit.

Can I open a PR (it might take a while) and submit changes? Of course there must be concept first.

I'd really like to see these changes in this TYPO3 testing framework.

Currently phpdbg is not supported:

$ php -qrr bin/phpunit
This script supports command line usage only. Please check your command.

This is due to the PHP_SAPI check in the bootstraps for unit and functional tests.

Hi,

thanks for this testing framework. Today i tried to use it in EXT:solr and allmost everything works out of the box.

I stumbled apon one issue with our fixtures. It seems the a database field from an xml fixture is getting lost in some cases. In our case we had the € sign in the fixture, also using the entity(€) did not work.

https://github.com/TYPO3-Solr/ext-solr/pull/1382/files#diff-8f6e673d6b81104401d07d2176003f42L472

For us it is not required to really import exactly this data, but with the core testing framework this seems to work (So the behaviour seems to be different at least).

What is also strange: It works fine on my VM but not on travis-ci.org.

Running a unit test with @runInSeparateProcess fails with an error:

PHPUnit 5.7.25 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 203 ms, Memory: 8.00MB

There was 1 error:

1) Vendor\Package\Tests\Unit\MyTestCase::myTest
PHPUnit_Framework_Exception: PHP Fatal error:  Uncaught RuntimeException: TYPO3_REQUESTTYPE has already been set, cannot be called multiple times in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/Bootstrap.php:714
Stack trace:
#0 /.../vendor/nimut/testing-framework/src/TestingFramework/Bootstrap/Bootstrap.php(46): TYPO3\CMS\Core\Core\Bootstrap->setRequestType(6)
#1 /.../vendor/nimut/testing-framework/src/TestingFramework/Bootstrap/AbstractBootstrap.php(76): Nimut\TestingFramework\Bootstrap\Bootstrap->includeAndStartCoreBootstrap()
#2 /.../vendor/nimut/testing-framework/res/Configuration/UnitTestsBootstrap.php(35): Nimut\TestingFramework\Bootstrap\AbstractBootstrap->bootstrapUnitTestSystem()
#3 /.../vendor/nimut/testing-framework/res/Configuration/UnitTestsBootstrap.php(36): {closure}()
#4 -(463): require_once('/var/www/web/ty...')
#5 {main}
  thrown in /.../vendor/typo3/cms/typo3/sysext/core/Classes/Core/Bootstrap.php on line 714

/.../vendor/phpunit/phpunit/phpunit:52

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Any idea how to fix this?

Since TYPO3 officially supports sqlite databases, this extension should as well.

See TYPO3/testing-framework#79 which implemented this in TYPO3 testing-framework and maybe also TYPO3/testing-framework#121

typo3DatabaseDriver=pdo_sqlite .Build/bin/phpunit -c .Build/vendor/nimut/testing-framework/res/Configuration/FunctionalTests.xml Tests/Functional/

This command fails because of setDBinit configuration. It actually crashes in the initializeTestDatabase method.

When I set the value to an empty string, I still get an error if 2 following tests are executed with the second iteration in the method setUpTestDatabase and the $schemaManager->listDatabases() call. This is not supported in sqlite:

Doctrine\DBAL\DBALException: Operation 'Doctrine\DBAL\Platforms\AbstractPlatform::getListDatabasesSQL' is not supported by platform.

The test of my ViewHelper fails because of the following PHP Warning with PHP 7.2 (seven times the same for each set of data):

7) T3easy\Mycontent\Tests\Unit\ViewHelpers\Format\Json\EncodeViewHelperTest::renderConvertsAValue with data set #6 (array(array('Hello world!', 'Foo'), 'Hello world!', 'Bar'), '{"stringValue2":"Bar"}')
count(): Parameter must be an array or an object that implements Countable

/app/.Build/public/typo3/sysext/fluid/Classes/Core/ViewHelper/AbstractViewHelper.php:119
/app/.Build/public/typo3/sysext/fluid/Classes/Core/ViewHelper/AbstractViewHelper.php:162
/app/.Build/vendor/nimut/testing-framework/src/TestingFramework/TestCase/ViewHelperBaseTestcase.php:164
/app/Tests/Unit/ViewHelpers/Format/Json/EncodeViewHelperTest.php:124

If I run the Unit tests of the Core ViewHelper, vendor/phpunit/phpunit/phpunit -c vendor/typo3/testing-framework/Resources/Core/Build/UnitTests.xml typo3/sysext/fluid/Tests/Unit/ in the same environment, I don't get that PHP warning.

It seems like a problem with the reflection:
https://github.com/TYPO3/TYPO3.CMS/blob/v8.7.28/typo3/sysext/fluid/Classes/Core/ViewHelper/AbstractViewHelper.php#L118-L121

Did I miss to bootstrap something?

My ViewHelper:

<?php
namespace T3easy\Mycontent\ViewHelpers\Format\Json;

use TYPO3\CMS\Fluid\Core\ViewHelper\AbstractViewHelper;
use TYPO3Fluid\Fluid\Core\Rendering\RenderingContextInterface;
use TYPO3Fluid\Fluid\Core\ViewHelper\Traits\CompileWithContentArgumentAndRenderStatic;

class EncodeViewHelper extends AbstractViewHelper
{
    use CompileWithContentArgumentAndRenderStatic;

    const TYPES_PATH = '_types';
    const DEFAULTS_PATH = '_defaults';

    /**
     * @var bool
     */
    protected $escapeChildren = false;

    /**
     * @var boolean
     */
    protected $escapeOutput = false;

    /**
     * Initialize arguments
     */
    public function initializeArguments()
    {
        $this->registerArgument('value', 'mixed', 'The incoming data to convert, or null if VH children should be used');
    }

    /**
     * Applies json_encode() on the specified value.
     *
     * @param array $arguments
     * @param \Closure $renderChildrenClosure
     * @param RenderingContextInterface $renderingContext
     * @return string
     * @see http://www.php.net/manual/en/function.json-encode.php
     */
    public static function renderStatic(array $arguments, \Closure $renderChildrenClosure, RenderingContextInterface $renderingContext)
    {
        $value = self::prepareValue($renderChildrenClosure());

        return json_encode($value, JSON_FORCE_OBJECT);
    }

    /**
     * @param mixed $value
     * @return mixed
     */
    protected static function prepareValue($value)
    {
        $types = [];
        if (is_array($value)) {
            if (array_key_exists(self::TYPES_PATH, $value) && is_array($value[self::TYPES_PATH])) {
                $types = $value[self::TYPES_PATH];
                unset($value[self::TYPES_PATH]);

                foreach ($types as $key => $type) {
                    if (isset($value[$key])) {
                        settype($value[$key], $type);
                    }
                }
            }

            if (array_key_exists(self::DEFAULTS_PATH, $value) && is_array($value[self::DEFAULTS_PATH])) {
                $defaults = $value[self::DEFAULTS_PATH];
                unset($value[self::DEFAULTS_PATH]);
                foreach ($defaults as $key => $default) {
                    if (isset($types[$key])) {
                        settype($defaults[$key], $types[$key]);
                    }
                    if ($value[$key] === $defaults[$key]) {
                        unset($value[$key]);
                    }
                }
            }
        }

        return $value;
    }
}

The Test

<?php
namespace T3easy\Mycontent\Tests\Unit\ViewHelpers\Format\Json;

use T3easy\Mycontent\ViewHelpers\Format\Json\EncodeViewHelper;
use Nimut\TestingFramework\TestCase\ViewHelperBaseTestcase;
use PHPUnit\Framework\MockObject\MockObject;

class EncodeViewHelperTest extends ViewHelperBaseTestcase
{
    /**
     * @var EncodeViewHelper|MockObject
     */
    protected $viewHelper;

    /**
     * @return void
     */
    protected function setUp()
    {
        parent::setUp();
        $this->viewHelper = $this->getMockBuilder(EncodeViewHelper::class)->setMethods(['renderChildren'])->getMock();
        $this->injectDependenciesIntoViewHelper($this->viewHelper);
    }

    /**
     * @return array
     */
    public function valueDataProvider()
    {
        return [
            [
                'value' => [
                    'testString' => 'Hello world!',
                    'testString2' => 'TYPO3 rocks!',
                ],
                'expected' => '{"testString":"Hello world!","testString2":"TYPO3 rocks!"}',
            ],
            [
                'value' => [],
                'expected' => '{}',
            ],
            [
                'value' => [
                    '_types' => [
                        'boolValue' => 'bool',
                    ],
                    'boolValue' => '1',
                ],
                'expected' => '{"boolValue":true}',
            ],
            [
                'value' => [
                    '_defaults' => [
                        'boolValue' => '0',
                    ],
                    '_types' => [
                        'boolValue' => 'bool',
                    ],
                    'boolValue' => '1',
                ],
                'expected' => '{"boolValue":true}',
            ],
            [
                'value' => [
                    '_types' => [
                        'boolValue' => 'bool',
                        'intValue' => 'int',
                    ],
                    '_defaults' => [
                        'boolValue' => '0',
                        'intValue' => '3000',
                    ],
                    'boolValue' => '1',
                    'intValue' => '4000',
                ],
                'expected' => '{"boolValue":true,"intValue":4000}',
            ],
            [
                'value' => [
                    '_types' => [
                        'boolValue' => 'bool',
                        'intValue' => 'int',
                        'equalStringValue' => 'string',
                        'deviatingStringValue' => 'string',
                    ],
                    '_defaults' => [
                        'boolValue' => '0',
                        'intValue' => '3000',
                        'equalStringValue' => 'TYPO3 rocks!',
                        'deviatingStringValue' => 'foo',
                    ],
                    'boolValue' => '1',
                    'intValue' => '3000',
                    'equalStringValue' => 'TYPO3 rocks!',
                    'deviatingStringValue' => 'bar',
                ],
                'expected' => '{"boolValue":true,"deviatingStringValue":"bar"}',
            ],
            [
                'value' => [
                    '_defaults' => [
                        'stringValue1' => 'Hello world!',
                        'stringValue2' => 'Foo',
                    ],
                    'stringValue1' => 'Hello world!',
                    'stringValue2' => 'Bar',
                ],
                'expected' => '{"stringValue2":"Bar"}',
            ],
        ];
    }

    /**
     * @param mixed $value
     * @param string $expected
     * @test
     * @dataProvider valueDataProvider
     */
    public function renderConvertsAValue($value, $expected)
    {
        $this->setArgumentsUnderTest(
            $this->viewHelper,
            [
                'value' => $value,
            ]
        );
        $actualResult = $this->viewHelper->initializeArgumentsAndRender();
        $this->assertEquals($expected, $actualResult);
    }
}

I have found problem with nimut/testing-framework in combination with PostgreSQL.

MySQL and PostgreSQL differs on how the auto_increment is working, but the doctrine/dbal handles this for us.

If I have a fixture with following content simplified (pages).

CREATE TABLE pages (
  uid int(11) unsigned NOT NULL auto_increment,
  title varchar(50) DEFAULT '' NOT NULL,

  PRIMARY KEY (qid),
) ENGINE=InnoDB;

When I then have a test, that insert data into this.
Let's say we have a function adding pages, but only adding titles as uid is auto_increment.

$this->addPage('New page title') will then throw an exception.

Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "pages_pkey"
DETAIL:  Key (uid)=(1) already exists.

This will happen the first 3 times I run the tests, the 4th time, it will add it as the uid 4 isn't set yet.

When looking at the PostgreSQL index, after the fixtures is include.

SELECT nextval('pages_uid_seq');

It will return 1, as no records are inserted yet from PostgreSQL point of view.

I don't know how this should be fixed, but if I run the test with the typo3/testing-framework instead, it's working.

One difference as I see it is that the typo3/testing-framework is using the

$connection = GeneralUtility::makeInstance(ConnectionPool::class)->getConnectionForTable($tableName);
$connection->insert($tableName, $insertArray, $types);

whereas the nimut/testing-framework is using

protected function getDatabaseConnection()
{
    if (null === $this->database) {
        $this->database = new Database();
    }

    return $this->database;
}

Constructor of Database is

public function __construct(Connection $connection = null)
{
    $this->connection = $connection ?? GeneralUtility::makeInstance('TYPO3\\CMS\\Core\\Database\\ConnectionPool')
            ->getConnectionByName(ConnectionPool::DEFAULT_CONNECTION_NAME);
}

$database = $this->getDatabaseConnection();
$database->insertArray($tableName, $insertArray);

So if the diff is "only" the build up of the connection ->getConnectionForTable($tableName) and ->getConnectionByName(ConnectionPool::DEFAULT_CONNECTION_NAME) I don't know.

Hope that it somehow makes sense what I'm writting.
I'll be happy to try to help fix and test this.

I just did an update to TYPO3 10.4.16 which, among others, brings compatibility with Doctrine DBAL 2.13. Thus the latter was also updated to the latest 2.13 version.

Now the first of each functional tests fails to connect to the database. The database exists after the test so maybe something has changed in Doctrine which causes an earlier connection before the FunctionalTestCase setup.

$ composer show doctrine/dbal | grep versions
versions : * 2.13.1
$ mysql -h... -u... -p... -e "DROP DATABASE IF EXISTS typo3_functional_ft3d326a3;"
$ rm -rf .../typo3temp/var/tests/functional-3d326a3/
$ phpunit --configuration phpunit-functional.xml --colors=always --filter SampleFunctionalTest

PHPUnit Pretty Result Printer 0.29.3 by Codedungeon and contributors.
==> Configuration: /.../vendor/codedungeon/phpunit-result-printer/src/phpunit-printer.yml

PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.16
Configuration: /.../phpunit-functional.xml


 ==> SampleFunctionalTest     ⚈  

Time: 4.11 seconds, Memory: 12.00 MB

✖ There was 1 error:

1) Acme\Package\Tests\Functional\SampleFunctionalTest::test
  PHPUnit\Framework\Exception: PHP Warning:  mysqli::real_connect(): (HY000/1049): Unknown database 'typo3_functional_ft3d326a3' in /.../vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/Mysqli/MysqliConnection.php on line 86

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

$ phpunit --configuration phpunit-functional.xml --colors=always --filter SampleFunctionalTest

PHPUnit Pretty Result Printer 0.29.3 by Codedungeon and contributors.
==> Configuration: /.../vendor/codedungeon/phpunit-result-printer/src/phpunit-printer.yml

PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.16
Configuration: /.../phpunit-functional.xml


 ==> SampleFunctionalTest     ✔  

Time: 2.28 seconds, Memory: 12.00 MB

OK (1 test, 6 assertions)

Downgrading to Doctrine 2.11.x fixes the problem:

$ composer show doctrine/dbal | grep versions
versions : * 2.11.3
$ mysql -h... -u... -p... -e "DROP DATABASE IF EXISTS typo3_functional_ft3d326a3;"
$ rm -rf .../typo3temp/var/tests/functional-3d326a3/
$ phpunit --configuration phpunit-functional.xml --colors=always --filter SampleFunctionalTest

PHPUnit Pretty Result Printer 0.29.3 by Codedungeon and contributors.
==> Configuration: /.../vendor/codedungeon/phpunit-result-printer/src/phpunit-printer.yml

PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.16
Configuration: /.../phpunit-functional.xml


 ==> SampleFunctionalTest     ✔  

Time: 2.28 seconds, Memory: 12.00 MB

OK (1 test, 6 assertions)

General Information:

TYPO3 11
typo3-composer-installers: 3

The Problem

When starting an UnitTest with a command like: .Build/bin/phpunit -c .Build/vendor/nimut/testing-framework/res/Configuration/UnitTests.xml Tests/Unit/ everything works as expected until you add an extension that provides an alias for a class. An example would be friendsoftypo3/fontawesome-provider.
If this extension is added this exception is thrown:

Error in bootstrap script: RuntimeException:
Cannot set an alias map as the alias loader is not registered!
#0 <myprojectpath>/.Build/public/typo3/sysext/core/Classes/Core/ClassLoadingInformation.php(142): TYPO3\ClassAliasLoader\ClassAliasMap::addAliasMap()
#1 <myprojectpath>/.Build/vendor/nimut/testing-framework/src/TestingFramework/Bootstrap/AbstractBootstrap.php(262): TYPO3\CMS\Core\Core\ClassLoadingInformation::registerClassLoadingInformation()
#2 <myprojectpath>/.Build/vendor/nimut/testing-framework/src/TestingFramework/Bootstrap/AbstractBootstrap.php(82): Nimut\TestingFramework\Bootstrap\AbstractBootstrap->dumpAutoloadFiles()
#3 <myprojectpath>/.Build/vendor/nimut/testing-framework/res/Configuration/UnitTestsBootstrap.php(35): Nimut\TestingFramework\Bootstrap\AbstractBootstrap->bootstrapUnitTestSystem()
#4 <myprojectpath>/.Build/vendor/nimut/testing-framework/res/Configuration/UnitTestsBootstrap.php(36): PHPUnit\Util\FileLoader::{closure}()
#5 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/Util/FileLoader.php(66): include_once('<myprojectpath>/...')
#6 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/Util/FileLoader.php(49): PHPUnit\Util\FileLoader::load()
#7 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/TextUI/Command.php(565): PHPUnit\Util\FileLoader::checkAndLoad()
#8 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/TextUI/Command.php(345): PHPUnit\TextUI\Command->handleBootstrap()
#9 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/TextUI/Command.php(112): PHPUnit\TextUI\Command->handleArguments()
#10 <myprojectpath>/.Build/vendor/phpunit/phpunit/src/TextUI/Command.php(97): PHPUnit\TextUI\Command->run()
#11 phpvfscomposer://<myprojectpath>/.Build/vendor/phpunit/phpunit/phpunit(97): PHPUnit\TextUI\Command::main()
#12 <myprojectpath>/.Build/bin/phpunit(115): include('phpvfscomposer:...')
#13 {main}

The file in .Build/public/typo3conf/autoload-tests/autoload_classaliasmap.php has now filled arrays so that the ClassAliasMap::$classaliasloader is needed which seems like it is not defined.

For functional tests $testExtensionsToLoad needs provided if you need to access functionality from (other) extensions.

This is not obvious and should be nicely documented. Alternatively, the framework could provide an error message when test execution fail simply because an appropriate definition of the variable is missing (if this is possible).

Cc @comsolit, @robert197, @bittner

I'm currently working on an extension that imports static data using an ext_tables_static+adt.sql file, and that includes a functional repository test that tests that records can be loaded from the corresponding table, and that fields are mapped correctly for the model.

The first time one of those tests is run (when a new test system and a new DB is created), is passes. The second time, it fails because the database is truncated:

    public function setUp(
            // …
    ) {
        $this->registerNtfStreamWrapper();
        $this->setTypo3Context();
        if ($this->recentTestSystemExists()) {
            $this->includeAndStartCoreBootstrap();
            $this->initializeTestDatabase();
            $this->loadExtensionConfiguration();
        } else {
            // …
        }

    protected function initializeTestDatabase()
    {
        $this->bootstrap->initializeTypo3DbGlobal();
        /** @var DatabaseConnection $database */
        $database = $GLOBALS['TYPO3_DB'];
            // …
        foreach ($database->admin_get_tables() as $table) {
            $database->admin_query('TRUNCATE ' . $table['Name'] . ';');
        }
    }

This problem occurs with nimut/testing-framework 2.0.3, but the corresponding code in master does the same things.

May a configuration in the functional test case which tables should be kept would be an approach for this?

The composer.json specifies support for TYPO3 v11. But when trying to run test cases with TYPO3 v11, I get the following error:

 Uncaught TypeError: Argument 2 passed to TYPO3\CMS\Core\Core\Bootstrap::createPackageManager() must be an instance of TYPO3\CMS\Core\Package\Cache\PackageCacheInterface, instance of TYPO3\CMS\Core\Cache\Frontend\PhpFrontend given

CoreBootstrap Line 247 expects PackageCacheInterface instead of FrontendInterface since TYPO3/typo3@6cfc1a1

There is a new helper method Bootstrap::createPackageCache, which converts a FrontendInterface into a PackageCacheInterface interface.

Hi,
could someone explain following things:

• parallels with typo3/testing-framework
• why nimut/testing-framework and not typo3/testing-framework or vice versa or using in parallel
• etc.

Thanks

(will create PR based in on this issue)

Please clarify suport in the future. There is now this information in the README:

Currently TYPO3 CMS 9.5 up to 11.5 are tested and supported.

I think it would be good to be even clearer here, e.g.

There are no plans to support nimut/testing-framework for versions above TYPO3 v11. You are advised to switch to the typo3/testing-framework for TYPO3 v12 and above. The typo3/testing-framework has improved support for testing more than one version of the core. Further information and an introduction can be found in the official TYPO3 documentation:

• Testing Extensions: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/ExtensionTesting.html

More information:

Previously, we referred to Nimut testing-framework in several places in the docs, e.g.:

• Extension Testing https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/ExtensionTesting.html#scope
• composer.json: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ExtensionArchitecture/FileStructure/ComposerJson.html

This is now being replaced in the versions 12.4 and up, see

• issue: TYPO3-Documentation/TYPO3CMS-Reference-CoreApi#2972
• and PR: TYPO3-Documentation/TYPO3CMS-Reference-CoreApi#2977

I have also pushed several PR for updating the "Extension Testing" page. But it still explains testing only one core version (as was previously the case). It does however refer to enetcache, which now tests 2 core versions in the main branch.

The page was originally written by Lolli, I have now just updated some stuff in the newer branches which were no longer up to date (mostly not merged yet).

• PRs: https://github.com/TYPO3-Documentation/TYPO3CMS-Reference-CoreApi/pulls

Unfortunately, there are still some things missing in the docs, but progress is being made.

I assume there will be further PR and Stefan from the core team has already said he would be adding to the documentation at a later date. I assume the topic of testing multiple core versions will also be handled, along with some of the ideosyncrasies.

This is a follow up, based on information I got from Helmut on Slack. If anything I wrote here is not correct or you don't agree with any of my changes, please let me know or contact Slack #typo3-documentation for general documentation related things.

I think a big thank you is also in order for having maintained this package for quite a time, filling a gap in testing support and providing useful tools for TYPO3 extension authors. ❤️ 🌟

Hi.

Are there any plans for adding support for PostgreSQL.

When running my tests for the TYPO3 Crawler with typo3DatabaseDriver=pdo_pgsql I get the error: FATAL: database "typo3test_ft027f169" does not exist if I switch to the typo3/testing-framework the DB is created as expected.

Might be that I jumped to the wrong conclusion here, but looks to me that it's supported.

I have limited knowledge on PostgreSQL, but if I can help in any way getting this added, let me know.

When running functional tests agains TYPO3v7 and e.g. using the DataHandler errors like this can occur:

Error: Call to a member function substr() on null
/home/travis/build/[secure]/typo3-flat-urls/vendor/typo3/cms/typo3/sysext/core/Classes/DataHandling/DataHandler.php:1862
/home/travis/build/[secure]/typo3-flat-urls/vendor/typo3/cms/typo3/sysext/core/Classes/DataHandling/DataHandler.php:1744
/home/travis/build/[secure]/typo3-flat-urls/vendor/typo3/cms/typo3/sysext/core/Classes/DataHandling/DataHandler.php:1703
/home/travis/build/[secure]/typo3-flat-urls/vendor/typo3/cms/typo3/sysext/core/Classes/DataHandling/DataHandler.php:1542
/home/travis/build/[secure]/typo3-flat-urls/vendor/typo3/cms/typo3/sysext/core/Classes/DataHandling/DataHandler.php:1250
/home/travis/build/[secure]/typo3-flat-urls/Tests/Functional/PageTest.php:64

This happens because $GLOBALS['LANG'] is not initialized anywhere. Adding the following to the tests fixes the issue:

$GLOBALS['LANG'] = GeneralUtility::makeInstance(LanguageService::class);
$GLOBALS['LANG']->init(null);

This could be added to FunctionalTestCase::setUpBackendUserFromFixture() since normally LanguageService::init() should be called with $GLOBALS['BE_USER']->uc['lang'].

What do you think?

in my extension i have the following typoscript snippet: https://github.com/helhum/ext_scaffold/blob/bb149b9e6a0ea3aaa3409879065bc4b05c5eafd5/composer.json

Since I require typo3/cms-core only, my .Build folder looks like this:

but there is no Web/typo3/index.php file. This file only exists if I require the full core with typo3/cms

https://github.com/Nimut/testing-framework/blob/master/src/TestingFramework/Bootstrap/AbstractBootstrap.php#L138-L143

See also helhum/ext_scaffold#10