Begin to host documentation site about heavylight

[lewisfogden]

Thoughts on docs:

Getting started

• installing
• show my single policy version (i.e. easy) - equivalent of print("Hello World")
• introducing tables Table

Workflow

• design structures (e.g. using basis, data etc)
• start to end production design (read dataframe > generate results > store in dataframe)

More advanced / performance:

• using numpy for high performance
• cache optimised models
• intifying data and tables (e.g. 'M' -> 0, 'F' -> 1) consistently to avoid string lookup bottlenecks
• storing and summarising results
• enhancing numpy using numba
• running models on the GPU

Utilities

• table generator
• [ ]

[MatthewCaseres]

Sure. I'll sign up for

• NumPy
• GPU
• Cache optimizing

and if I can finish those I'll come back for more work

[lewisfogden]

We've got quite a wide variety of users to cater for - I'm pitching it towards getting people out of building unmaintainable models in Excel (i.e. python noobs, don't know what a class is, types are etc), and you've pretty much built the death star 😁, I think we need to do a little bit of working out the journey and documentation, and working out how to define the overall API.

I'm going to work on the basic - idea in Excel -> delivery using heavylight, and document that a bit more.

[MatthewCaseres]

You are going to know the typical user a lot better than I would, take the docs in whatever direction is best.

I estimate I can make the death star 5-10 times faster, might take a step back to do some more research oriented stuff.

[MatthewCaseres]

I think you will need to change your settings to host from gh-pages branch

[lewisfogden]

Thanks - updated. I've started writing docs as well - using mkdocs + mkdocs-jupyter (to render notebooks). I'll add them in and see how they render. (Note I'm a bit busy next few days so will be random)

[lewisfogden]

Site looks great, thank you! I've moved the example you had (with super.__init__()) to a hidden markdown page as we've not implemented the summary type function in the main code - this is probably a bit advanced for users, and I wonder whether an agg function in the main init would be better (still need to get rid of do_run)

[MatthewCaseres]

yeah it was just an example, having the non cache evicting model introduced first in the documentation makes a lot of sense.

optimized model runs require a storage function to view intermediate aggregated results, but without memory optimization this storage function is not needed in the constructor. So I think no agg in constructor is ideal.

I argue in issue #6 to expose a well formatted dictionary and not worry about formatting user results. I think if the user has a documentation page on how to work with user results, maybe this is good enough.

[MatthewCaseres]

Closing this as docs site is up and running. Specific enhancements can be filed under their own tickets.

[lewisfogden]

Re-opening it as a place to discuss docs (sorry!) - I've got checklist above

Also - My older site for the previous iteration (heavymodel) is here: https://digitalactuary.co.uk/ there is some content we could move over (some bits I don't want to keep) - I might repoint the url at heavylight docs.

[lewisfogden]

Added tables documentation: https://lewisfogden.github.io/heavylight/getting_started/heavylight_tables/

(I've been slightly cheeking in pushing directly to main, worst case I break the documentation and need to unwind the push).

100m model points in 16 seconds kind of feels fast enough for now! I'm going to explore use cases more with what we have before deciding where the gaps are.

Last bit of documentation for now is the 'make_example' command, which I think I might tweak a lot, and would be worth adding a LightModel example as well.