============================================================== FAILURES ===============================================================
____________________________________________________________ test_unittest ____________________________________________________________
capsys = <_pytest.capture.CaptureFixture object at 0x7f60a0efae00>
def test_unittest(capsys: CaptureFixture[str]):
results = run_unittest(capsys, 'unittest')
out = results.out
out.then_find('sybil setup')
out.then_find('fail.rst,line:6,column:1 ... 0\nok')
out.then_find('fail.rst,line:8,column:1 ... 1\nFAIL')
out.then_find('fail.rst,line:10,column:1 ... 2\nERROR')
out.then_find('fail.rst,line:12,column:1 ... 3\nok')
> out.then_find('sybil teardown 4\nsybil setup')
tests/test_functional.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <tests.helpers.Finder object at 0x7f60a0ef8dc0>, substring = 'sybil teardown 4\nsybil setup'
def then_find(self, substring):
> assert substring in self.text[self.index:], self.text[self.index:]
E AssertionError: fail.rst,line:12,column:1 ... 3
E ok
E sybil teardown 4
E
E ======================================================================
E ERROR: /tmp/sybil/tests/functional/pytest/fail.rst,line:10,column:1
E ----------------------------------------------------------------------
E Traceback (most recent call last):
E File "/tmp/sybil/sybil/integration/unittest.py", line 18, in runTest
E self.example.evaluate()
E File "/tmp/sybil/sybil/example.py", line 64, in evaluate
E result = evaluator(self)
E File "/tmp/sybil/tests/functional/unittest/test_unittest.py", line 19, in check
E raise ValueError(message)
E ValueError: X count was 3 instead of 4
E
E ======================================================================
E FAIL: /tmp/sybil/tests/functional/pytest/fail.rst,line:8,column:1
E ----------------------------------------------------------------------
E Traceback (most recent call last):
E File "/tmp/sybil/sybil/integration/unittest.py", line 18, in runTest
E self.example.evaluate()
E File "/tmp/sybil/sybil/example.py", line 66, in evaluate
E raise SybilFailure(self, result)
E sybil.example.SybilFailure: Example at /tmp/sybil/tests/functional/pytest/fail.rst, line 8, column 1 did not evaluate as expected:
E Y count was 3 instead of 2
E
E ----------------------------------------------------------------------
E Ran 8 tests in 0.001s
E
E FAILED (failures=1, errors=1)
E
E assert 'sybil teardown 4\nsybil setup' in 'fail.rst,line:12,column:1 ... 3\nok\nsybil teardown 4\n\n============================================================...------------------------------------------------------------\nRan 8 tests in 0.001s\n\nFAILED (failures=1, errors=1)\n'
tests/helpers.py:67: AssertionError
___________________________________________________________ test_namespace ____________________________________________________________
capsys = <_pytest.capture.CaptureFixture object at 0x7f60987b5ba0>
def test_namespace(capsys):
sybil = Sybil([parse], path='./samples')
documents = [sybil.parse(p) for p in Path(sybil.path).glob('sample*.txt')]
actual = []
for document in documents:
for example in document:
print(split(example.path)[-1], example.line)
example.evaluate()
actual.append((
split(example.path)[-1],
example.line,
document.namespace['parsed'].copy(),
))
out, _ = capsys.readouterr()
> assert out.split('\n') == [
'sample1.txt 1',
'[0]',
'sample1.txt 3',
'[0, 14]',
'sample2.txt 1',
'[0]',
'sample2.txt 3',
'[0, 13]',
''
]
E AssertionError: assert ['sample2.txt 1',\n '[0]',\n 'sample2.txt 3',\n '[0, 13]',\n 'sample1.txt 1',\n '[0]',\n 'sample1.txt 3',\n '[0, 14]',\n ''] == ['sample1.txt 1',\n '[0]',\n 'sample1.txt 3',\n '[0, 14]',\n 'sample2.txt 1',\n '[0]',\n 'sample2.txt 3',\n '[0, 13]',\n '']
E At index 0 diff: 'sample2.txt 1' != 'sample1.txt 1'
E Full diff:
E [
E + 'sample2.txt 1',
E + '[0]',
E + 'sample2.txt 3',
E + '[0, 13]',
E 'sample1.txt 1',
E '[0]',
E 'sample1.txt 3',
E '[0, 14]',
E - 'sample2.txt 1',
E - '[0]',
E - 'sample2.txt 3',
E - '[0, 13]',
E '',
E ]
tests/test_sybil.py:246: AssertionError
======================================================= short test summary info =======================================================
FAILED tests/test_functional.py::test_unittest - AssertionError: fail.rst,line:12,column:1 ... 3
FAILED tests/test_sybil.py::test_namespace - AssertionError: assert ['sample2.txt 1',\n '[0]',\n 'sample2.txt 3',\n '[0, 13]',\n 'sa...
==================================================== 2 failed, 101 passed in 5.25s ====================================================
I don't know about the first failure but the second one seems to be related to filesystem file ordering being unpredictable, i.e.:
$ find -name sample*
./tests/samples
./tests/samples/sample2.txt
./tests/samples/sample1.txt
commit 11496eb5761761b687ad4889b4173d3124caa844 (HEAD, refs/bisect/bad)
Author: Chris Withers <[email protected]>
Date: 2021-10-21 09:35:39 +0200
Fix source filtering.
- add exclude param for symmetry with pattern/patterns
- move all the testing to one place and ensure it's the same for all integration
I've reproduced this with Python 3.10.0 and 3.8.12.