Proposal: script converting and compressing HTML pages automatically about esp32-cam-webserver

[easytarget]

Karel; I just realised I simply closed this without providing any explanation. Apologies for that, it was rude of me.
I basically think that developing such a script is a good fun exercise in scripting etc
.
But is it also re-inventing the wheel.. there are built-in tools in Linux that can do this already, I could generate a compressed and encoded header file very simply like this:
gzip -kf index.html && xxd -i index.html.gz > index.html.h
xxd is part of the vim toolkit, it generates a c-header file from the source file defining index_html_gz[] with the contents in hex form and index_html_gz_len with the size.
Adding that to a Makefile in a traditional development environment is easy, adding it to the Arduino IDE is hard.

Since this project has lots of free memory (note that the HTML is in PROGMEM, not RAM) it's not an issue here, and since the page is only loaded once the data saved by compression is insignificant vs the bandwidth needed for a video stream.

[karlitos]

Hello Owen, thank you for your detailed response and I can fully understand your reasoning. Your explanation actually provides the EXACT solution for my original problem - I spend some time searching for what xxd -i -u is providing out-of-the-box and I never found a single mention of xxd along the way 😃

I think this would be a valuable addition to my original stackoverflow post and I of course I will be mentioning you as the original author of this solution.

EDIT: Just out of curiosity - why does gzip -kfc index.html | xxd -i -u provide different output ?

[easytarget]

I cant remember where I heard of xxd either, just one of these random things I learned somewhere..

The reason why the simpler command you show produces different output is because when you run xxd -i without an input file (stdin) it only generates the hex content, and not the whole structure. When xxd has a input filename I uses that to generate full definition instead of just the contents.

See the effect for yourself like this:

$ echo "hello" > hello.txt
$ cat hello.txt | xxd -i
  0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x0a
$ xxd -i hello.txt 
unsigned char hello_txt[] = {
  0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x0a
};
unsigned int hello_txt_len = 6;

[RobotHacker]

I had run across another option for encoding the html through Cyberchef here. From that I made the following recipes:
decode

encode

[easytarget]

Nice, A good solution for windows users I guess.

If I was making a commercial device with maybe dozens of complex html pages, javascript, graphics etc this would make sense. .. but If I were doing that I would also be using something more complex as my development environment, and I'd simply script something up with bash/xxd and put it as a pre-run script in my buildfiles.

For this project the advantages of keeping the html trivial to edit far, far outweigh any advantages zipping and hex encoding it.