Feature: built-in ply based preprocessor? about pycparser

[refi64]

It'd be even easier, since pycparser's own lexer can (largely) be reused.

[eliben]

I'm aware of the preprocessor in the PLY repo - it's been there for a long time! I actually once asked Dave what is its status, and he told me it's just a prototype. I don't believe it changed much since then.

FWIW, I'm not really into developing a preprocessor of my own at this time, since my main aim is Unix-like systems, where this is a non-problem. There are good options out there even for Windows folks - for example Clang for Windows: http://llvm.org/releases/download.html

... perhaps I should add this to the README file.

[paniq]

The Clang executable is 28 MB in size. That's a little much just for a preprocessor, esp. when we're shipping it with a game that is about the same size ;-). I also looked at mcpp as an alternative but it looks like cpp.py could be a compact solution to our issues. I'll experiment and report back.

[paniq]

Yeah. It looks like cpp.py has an issue with including files correctly and resolving macros within the included files. I'm closing the issue since the original topic is covered.

[eliben]

I'm curious about you usage scenario, can you provide more details? Shipping pycparser in a game... that sounds interesting 😄

[paniq]

I'm using pycparser in combination with cffi to parse and generate struct definitions (with metadata) for soil, a transparent serialization library that I developed for our game Nowhere. It's being used for both editing and savegame data, and it will ship on Windows, OSX and Linux.

The engine provides introspection and editing UI for stored C types once it has their type declarations, which is where pycparser comes in. People modding the game have access to engine internals.

[eliben]

Thanks for sharing, that definitely sounds interesting.

FWIW, CFFI uses a regex-based hack to get rid of all preprocessor-y stuff. Often this is sufficient to just parse struct definitions and function declarations.

[paniq]

Actually, it doesn't; it has the same "clean your crap first" policy as pycparser, so I used to use cpp.exe for that; as it's a problem shipping GPL code with our game, and we can't count on cpp.exe being available everywhere, we now resorted to mcpp, which appears to work well everywhere and is BSD licensed.