Code block indentation about orgajs

[xiaoxinghu]

@thomasheartman Should be fixed in v2. Check this out. Please let me know if there're any more issues. 🤞

[xiaoxinghu]

@thomasheartman sorry man, my bad, I forgot to merge the whitespaces branch, and I realized that and merged it after the .3 release. I just published a 2.0.0-next.4 release. Please have another go.

[gitonthescene]

No worries at all. I appreciate your taking the effort to be explicit and wanted to return the favor. I'm sure this is all something that can be worked out and being explicit helps.

[gitonthescene]

FWIW, you should be able to do this sort of post-processing on the tree itself if this is the output you want. In #47 I propose preserving even more of the original structure in the AST. Incidentally, what output are you talking about? Is this orga-rehype? In the fork pointed at in #47 I have an orga-stringify which compiles the AST back to an org file. It doesn't (yet?) preserve the white space before the keyword in this case, but it does preserve the white space in the body. This doesn't affect it's behavior in org-mode

Incidentally, the initial space does affect behavior in this case:

• Header level one

  • Content.
    #+begin_src elisp
    (message "one")
    #+end_src

    • Content
      #+begin_src elisp
      (message "two")
      #+end_src

  • Content
    #+begin_src elisp
    (message "three")
    #+end_src

In this case, the initial space of the keyword is preserved enough to ensure the same behavior in org-mode.

[thomasheartman]

Hey! Sorry for taking so long to respond. Hectic week 😅

Incidentally, what output are you talking about?

Oh, right. That's a good point. I've only used Orga with Gatsby (gatsby-orga). Maybe this issue belongs in that repo, but I felt that it was more related to Orga itself than to gatsby-orga. Now, though, I'm not so sure.

you should be able to do this sort of post-processing on the tree itself if this is the output you want.

Ah, cool! I'm not sure exactly how I'd do that, though. Is there a guide to get started anywhere? The things mentioned in #47 sound like a great idea, though. If I understand correctly, it would allow a 'lossless'(-ish) conversion between org and AST so you could transform back and forth at will without losing information?

In short, maybe what I want is something that gatsby-orga should handle, rather than something that orga itself should be responsible for.

[gitonthescene]

No worries. Actually #49 has a suggestion to allow configuration of alternative parsers to allow you to attach unified plugins. I don't see one that trims whitespace, but you could write a transformer that leaves most nodes alone, but strips leading whitespace for begin blocks.

One issue with this is that processing the tree very much depends on the shape of the tree and so if the model changes such plugins would have to change.

[thomasheartman]

Oh, that sounds pretty exciting. I must admit I wasn't aware of Unified before you mentioned it previously, and I only now looked it up. From what I understand, it's part of what helps Gatsby work, so writing a transformer like that sounds very much like what I want to do!

From a very cursory glance, the documentation appears to be pretty solid too, so I should be able to sit down and figure it out.

Thank you very much for your input! 🙏

I'll leave the issue open for now in case someone else has anything to contribute and because the issue isn't resolved yet (though there is a potential way forward).

[thomasheartman]

Yeah, this looks great! Amazing work. Looking forward to trying it out later 😄

I'll close this for now, then, and see what I can do with v2. Assuming the Gatsby transformer works with the new version, I should be on my merry way.

Cheers!

[thomasheartman]

@xiaoxinghu:

So I got around to testing it out today with the Gatsby transformer (both orga and gatsby using the latest @next version), and I'm having some difficulty with it.

Mind you, this is using it with gatsby (and the transformer), so I don't know where the issue is, but I'm reporting it here because it seems related to this problem.

The first line is indeed indented correctly, but any lines following this are are indented an number of spaces for whatever reason. This image demonstrates the effect it has. Note that in the first code block, all use statements should start at the same column. In the actual source block, they're all aligned to the same column.

If I move the content of the source block back to the first column in the buffer (which is much earlier than where org places it), the extra-indented lines move to the first column. In fact, it seems that the first line is indented correctly, but all the following lines are indented according to their column in the buffer.

Are you aware of this? I thought it might be my styling, but that doesn't seem to be it. I disabled all styles, and it still happens.

[gitonthescene]

I still think the only real way to handle this is with a filter which explicitly strips the way you prefer. I don't think there's anything canonical in what you're asking so a custom solution is required.

[thomasheartman]

@xiaoxinghu Oh, swell! Yeah, that seems to work perfectly 😄 Thanks!

@gitonthescene I think I see where your coming from, but I'm not sure I agree. Obviously, what's canonical or not is not up to me to decide, but I've tried to explain my thought process below because I think it makes sense.

As for writing a custom transformer: I'd be happy to give that a go, but I haven't had the time to sit down with it just yet. I agree that that's a good way to go, but think I'd have to invest a fair bit of time to understand how to hook everything up. In this case, though, I do think that Orga should handle it, though I admit that it's only a personal expectation. But as it turns out, it seems that the library author agrees, so I'm happy for now.

Here's my reasoning for why I think Orga should strip leading whitespace (at least up until the default starting area that org inserts):

By default, org mode will indent content further for every subheading you insert. Viewed in an emacs buffer, the text will have an increasing number of spaces put in front. However, when turning this into content viewed on the web (or in other formats), I don't expect the text in the output to necessarily be further indented the deeper down the tree we go (unless the output format also does that by default).

Blocks (source, examples, quotes) are also indented based on their parent heading level. And as with text above, I wouldn't expect the initial indentation to be reflected in the output. In particular: if you edit a source block, org will automatically indent it another 2 spaces from the block's heading. This should be expected. Arguably, if you indent something further (say you indent everything in the code block by 8 spaces), then this should be reflected in the output, but I'm not convinced that's a common use case. At least, I've never done that, but I don't know whether my usage is representative or not.

Based on the previous paragraphs, I would expect that code blocks indented the same as their surround text, using the default block indentation that org mode inserts, have no extra indentation in their output.

[gitonthescene]

Thanks for the explanation. Understanding use cases with clarity can only help to make the software better.

As you said, this is about what you expect from the html output is not necessarily what you require from the AST representation. My main concern is changing the AST structure for a targeted output. Ideally, the AST would be flexible enough to be used for more than just producing web pages. I haven't looked at the changes yet, but ideally it would be somewhere between the AST and the web output, which is why I suggested a transformer.

To be clear, there are likely many ways to represent the AST which allow flexibility. My concern is not to have a particular representation but that the representation be flexible.

[thomasheartman]

Yeah, I agree with what you're saying. It is possible that this should have been in the gatsby-specific repo rather than in the main repo. In case I haven't been clear enough on it (it seems that maybe I haven't): I have always been concerned only with the rendered HTML output in this case. I agree that for use cases where you want to exactly replicate the original source file, the indentation should be preserved as it is in the source file.

Sorry for any misunderstanding.