Parse and export directive options, too about sybil

[cjw296]

I think I've got a reasonable amount of this done already on master...skip_improvements
Can you have a look and let me know what you think?

[cjw296]

@sscherfke - last call on this, I'm planning a 6.0.0 release sometime in the next day or few...

[cjw296]

No, but you're right that it's missing and I've implemented in 89d3eee

Could you have a look at master now and see if that meets all your needs?

[sscherfke]

I am happy now (and healthy again). :-)

Thanks a lot for your efforts! <3

[sscherfke]

I skimmed through the changes.

They seem to fix my requirements 1–3 (parse :key: value options from myst/rest directive and add them to the lexemes).

The evaluator stack seems to solve my requirements for 5: I can instantiate a CodeBlockParser for console or toml (or the PythonCodeBlockParser) and push an evaluator that looks for :caption: file.ext and raises a NotEvaluated if it's not there, which would cause the default evaluators (e.g., doctest evaluator for PythonCodeBlock) to kick in. Is that correct?

[cjw296]

So, coming back to this testfixtures supports what you're after but with different spelling. It's a great library, but then I'm biased since I'm its author ;-)

I think master supports what you're after, but I'm not sure I'd use the evaluator stack, just a specialised CodeBlockParser.

Can you have a look at current master and let me know if you still feel anything is missing?

[sscherfke]

I’ll try to test the new changes and give you feedback, but had no time for this yet.

[sscherfke]

Wasn't there directive option parsing for the MyST lexers, too, in the feature branch skip_improvements? In the current master, it's only there for ReST.

It would be cool if that feature was also available for Myst. This is the regex I am currently using:

MYST_START_PATTERN = (
    r"^(?P<prefix>[ \t]*)"
    r"```\{(?P<directive>code-block)}\s*"
    r"(?P<arguments>[\w-]+\b)$\n"
    r"(?P<options>(?:\s*\:[\w-]+\:.*\n)*)?"
)

[cjw296]

@sscherfke - I'll take the thumbs up as "all good now", please let me know if you spot anything else and we can re-open this.

[sscherfke]

It's a thumbs up for "I’ll test it and report back" ;-)

[sscherfke]

Okay, I found two issues:

1. I think the options need to be forwared to the Region so that they are accessible in a parser. It needs to be added here: https://github.com/simplistix/sybil/blob/master/sybil/parsers/abstract/codeblock.py#L56
   Like this (but as an actual class attribute ;-)):

                   r = Region(lexed.start, lexed.end, lexed.lexemes['source'], self.evaluate)
                   r.options = lexed.lexemes.get("options", {})
                   yield r

2. The options in myst blocks must be enclosed with ---, but the MyST docs seem recommend options without ---: https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html – I posted a regex that is able to parse this above.

[sscherfke]

I can provide a PR as base, if you like :)

[cjw296]

2. Both styles are already supported:
   

   sybil/tests/samples/myst-lexing-directives.md

   Lines 21 to 38 in 93eb470

   	```{image} picture.png
   	```
   	```{image} picture.jpeg
   	:height: 100px
   	:width:200 px
   	:scale: 50 %
   	:alt: alternate text
   	:align: right
   	```
   	```{figure} picture.png
   	---
   	scale: 50 %
   	alt: map to buried treasure
   	---

[cjw296]

1. f6d8772 shows how I'd currently do the kind of thing you're after, but I'm now wondering in LexedRegion actually needs to exist...

[sscherfke]

1. I need to access the options in self.evaluate(), so overriding self.__call__() as in that test example does not work for me, I guess: https://gitlab.com/sscherfke/typed-settings/-/blob/main/conftest.py?ref_type=heads#L213
2. You're right. A mistake on my side :)

[cjw296]

I'm not sure doing what you're doing in self.evaluate() is right here; you're essentially picking between two evaluators based on an option in the ReST or MyST directive, so you should code it as such rather than trying to do both in a since evaluate method.

However, as of 59045ff I think you can do either. Let me know what you think.

[sscherfke]

Hi Chris, I got sick last week. Will try your changes asap and provide feedback.

[cjw296]

Very sorry to hear that, hope you're feeling better soon! 6.0.0 is out now with all the changes in it, so please give it a spin and it me know how it goes :-)