hanneskod / readme-tester Goto Github PK

Now it just screams and fails...

As it is now transforming attributes must be specified in the correct order. For example StartInNamespace must be used before UseClass..

Lets sort transformations to fix this.

interface SortableInteface
{
    public function getPriority(): int;
}

Non-sortable attributes can default to priority 0.

Implement sorting in AttributeSortingPass. Requires ExampleObj::withAttributes() to set the new order.

Required for integration with CI tools..

In verbose mode information on ignored examples should be printed.

Implementing this probably should involve an event triggering mechanism, instead of the iteratior based feedback we have now...

Compiling examples can involve extensive work. When examples are unchanged the compile step could be cached, could make reruns considerably faster.

ExampleProviderInterface could be implemented as Cache\CachingExampleProvider:

• provides cached examples when applicable
• uses a regular ExampleProvider internally when needed
• read from Suite if cache should be used
  • can be implicit false if custom compiler passes are used
  • can be implicit false if stdin (or is this because stdin will simply hash diferently?)

Cache\KeyFactory::createKeyForSuite(Suite $suite): string

• Should depend on the suite configuration in itself, as well as the actual testfile content

Cache\ExampleCache::store(string $key, ExampleStoreInterface $store): void

• has(string $key): bool
• get(string $key): ExampleStoreInterface
• clear(): void

Add extra fields to Suite:

• isCachable() (set to false with --no-cache)
• clearCache() (set to true with --clear-cache) (this is sort of a hack, but will play nicely with CachingExampleProvider..)

As it stands now it simply halts on unknown annotations.

See #18

Hi there,

It's just my curiosity.
Why did you abandon backward compatibility with PHP:^7.2 in the new version 2.0?

Now only files, assertions and failures are reported. This means that nothing is incremented when an example without an expectation is added, as we no longer can expect nothing...

As it is now readme-tester breaks when multiple examples defines the same symbol. For example the following should be possible, but is broken in beta-3:

```php
class Foo {}
```

```php
class Foo {}
```

Thw following will fail as the breaks() function is included twice when the third example includes the second (and both include the example context). Context should only be included once in this situation..

<!-- @exampleContext -->
```php
function breaks() {}
```
<!-- @example example-with-context -->
```php
```
<!-- @include example-with-context -->
```php
```

The purpose of extending is often to bring in variables, and thus extending multiple examples can be useful.

Could use an @include annotation instead of @extends. Since this isn't extention per se include makes more sense and is more php-like.

More: if I implement this using actual php include statements it could solve other issues involving namespaces and maybe yield #19 outdated..

Use @example name-of-test To add a custom name to readme test.

Sometimes you want your readme to consist of one big example that is split up in smaller chunks. You might start with doing some complex setup and then going through different use cases. There should be some mechanism to link examples togheter so that objects created in one example could be reused in another..

Using reactphp/promise..

Right now they are silently ignored.

Let Result contain an array of Outcome objects. (Or Effect??)

An Outcome can be marked using markAsHandled()
And checked using isHandled()

Different kinds of outcomes are defined in classes:

• ErrorOutcome
• ExceptionOutcome
• OutputOutcome
• ReturnOutcome

Unhandled outcomes should trigger failure...

Needs to be able to load an autoloader when examples include classes or similar...

Attributes global to all examples in all input files could be specified in config.

Could for example be used for a lib that always work in the same namespace

global_attributes:
    - ReadmeTester\\StartInNamespace('My\\Lib')

Or from command line

readme-tester --global-attribute "ReadmeTester\\StartInNamespace('My\\Lib')"

Write attributes to examples using a dedicated CompilerPass.

Add docs to global_attributes.md. Also add config documentation in configuration.md.

If name is not set using @example name can be picked from the closest previous header, given that there are no other examples with that name in the entire file.

This issue is intended to track some issues and tools currently not supporting php8. Fixes in src and build needed as these tools are fixed..

• Travis does not build php8
• Box can currently not build using php 8 (includes changes to .travis.yml) (box-project/box#486)
• Phpcs currently does not support all php8 features we use. Avaiting version 3.6 (squizlabs/PHP_CodeSniffer#3167)
• Phpeg is broken with php8 (https://github.com/scato/phpeg)

Merge 2.0 with master once all of these are fixed.

Implementation should be easy...

If not no assert count is made..

If would be cool if we could expect the content of variables at the end of example execution without having to manually return anything from the example. That way we can write cleaner examples but still test as much..

Implementation details could include putting the example code in a function and passing the expected variable as a reference.

Usage

@expectVar $name /regexp/
@expectVarType $name string

...

Now it is simply hidden...

A --strict setting could make tesning fail on unhandled output.

Currently the file extensions looked for when iterating the filesystem are hardcoded in SourceFileIterator.

Should probably be renamed FileIterator and moved to the Console namespace.

And on a similar note it's a bit odd to use README.md source in TesCommand. Better to use . if a default value is needed.

By specifying the name of the example to test

As the dependencies of phpeg conflicts with the lib dependencies.

Should be installed in another manner...

readmetester-phpunit-bridge

This should work

<!--
Some text here..

@example name

some test here
```php
// the example
```
-->

Now each individual file must be specified. Pointing to the docs directory is a given...

Need this switch...

This fails to be reconized as an example due to to many - wenh closing html comment. Fix...

<!--
@example setup

```php
$var = 'asd';
```
--->

Similar to @include but adds lines instead of full examples..

Can be used to step out of php mode before an example. (As a side note the fact that examples are in php mode by default needs to be documented.)

<!-- @prependLine "?>" -->
```php
This is a template test <?php echo $var; ?>...
\```

Or to add assertions.

<!-- @appendLine "assert($a == 1);" -->
```php
@a = 0;
\```

As there is a lot of biolerplate code pre- and apenden to the executed examples it can become difficult do define namespaces in the examples.

README.md can be a tested in itself using html comments to hide testing instructions from readers:

<!-- @expectOutput /regexp/ -->
```php
// This test pass because the correct output is generated
echo "foobar";
```
<!-- @ignore -->
```php
// This test is incomplete and ignored
new UnresolvedClass();
```

This syntax requires an exemplify comment directly followed by a php-colorized code block. The code block is executed in isolation and the defined expectation is validated.

Then some kind of phpunit hook is required to automatically test README.md as normal testing occurs (with the possibility to set what files are to be tested). Can be implemented using a test listener. See PHPUnit_Framework_BaseTestListener.

https://phpunit.de/manual/current/en/extending-phpunit.html

and

https://phpunit.de/manual/current/en/appendixes.configuration.html#appendixes.configuration.test-listeners

The executable would then work as an alternative way of testing examples.

hanneskod/classtools#1 needs to be resolved first

A docblock can be viewed as a markdown "file" and be parsed and tested. Some syntax issues needs to be resoved though.

This is how it would work now:

class Foo
{
    /**
     * <!-- @expectOutput "foobar" -->
     * ```php
     * (new Foo)->bar();
     * ```
     */
    function bar()
    {
        echo "foobar";
    }
}

Does this make sense:

class Foo
{
    /**
     * @expectOutput "foobar"
     * ```php
     * (new Foo)->bar();
     * ```
     */
    function bar()
    {
        echo "foobar";
    }
}

Or a more compact syntax:

class Foo
{
    /**
     * ```php, @expectOutput "foobar"
     * (new Foo)->bar();
     * ```
     */
    function bar()
    {
        echo "foobar";
    }
}

And lastly we want it to be possible to treat all snippets in a file as a whole. So that the following will work..

/**
 * <!-- @exampleContext -->
 * ```php
 * $foo = new Foo;
 * ```
 */
class Foo
{
    /**
     * <!-- @expectOutput "foobar" -->
     * ```php
     * $foo->bar();
     * ```
     */
    function bar()
    {
        echo "foobar";
    }
}