Will we be able to add applications coded in C/asm? about epsilon

[Ecco]

This is actually a distribution issue since you can already add C/asm apps to your own device: just add one in Epsilon's sources. Now obviously we're not going to merge every single app in Epsilon, so we definitely need a way to distribute 3rd-party apps.

[boricj]

@Zezombye I'm worried about the technical aspects of adding native apps at run-time and also the why of such an endeavor.

Dynamic linking would probably be the best solution, but there will be a hefty penalty in relocation/symbol data overhead and it would be a compatibility minefield without a stable ABI to work on (this is C++ and the firmware is in its early days so there's bound to be plenty of changes). Also, that's far from trivial to implement.

Syscalls would solve ABI issues, but you can only pass C objects around. Depending on the amount of functionality you want to expose (just the bare minimum to get games going, or do you want access to kandinsky, escher and poincare too?) that could be quite a lot of glue to maintain.

Any technical solution that would add a non-trivial maintenance burden on the NumWorks team will need a compelling real-world use-case justification before they are going to mainline it. I can see how run-time add-ins in MicroPython bytecode would be OK (easier for students to meddle with than C++, safety of an interpreted language should prevent random crashes, reasonably simple to implement, reuses lots of code already in the firmware, can't tamper with exam mode), but I have yet to hear a truly compelling argument for native run-time add-ins (also, a native add-in that nukes exam mode without even flashing the calculator would lower the bar to cheating so low it's not even funny).

If you just want to add native apps to your calc without having to set up the tool-chain, then how about an online tool on the NumWorks website that lets you pick what things on a centralized hub you want and let them build the firmware for you? If you want to develop some basic apps without the tool-chain then how about grabbing a dumb text editor and do some MicroPython apps (students with no prior programming experience are less likely to be scared away too)?

Once again: why would you need native run-time add-ins and why the NumWorks team should agree to take on non-trivial maintenance burden?

[boricj]

I don't think adding native apps at run-time would be a good idea since we can just freely recompile the firmware from source and do whatever we want. What we need is a modular way of managing apps at compile-time (going as far as being able to unselect official apps for additional Flash space), perhaps even some sort of third-party hub so as not to hunt down community apps scattered across the Internet. Let people build their own calculator, in some sort.

If people really want the ability to add apps at run-time I'd go for packaged apps in MicroPython bytecode, but even then I'm not sure if this would be a good idea.

[boricj]

Native run-time add-ins are, I think, easy to do if it is possible to copy-paste a folder into the OS source code and recompile it without changing anything else, because at this point it means you can add apps pretty easily (though I don't know anything about c++ so I could be wrong).

That's not adding native add-ins at run-time, but at compile-time. I've already done a proof-of-concept of that (#73). Run-time add-ins would be the ability to add new apps without flashing a new firmware, supposing we have data transfer and permanent storage management. Like I said, I don't think this capability would make sense except, possibly, for apps coded in Python.

How would run time apps take more memory than compiled apps? The code is the same, it should take the same place.

You can't just drop an array of bytes in memory and jump into it expecting things will work out. That app inserted at run-time would require a way to interface with the underlying firmware (dynamic linking, syscalls, well-known jump offsets...) that the NumWorks firmware currently does not offer. At compile-time it's the cross-linker that does this interfacing work ahead of execution for us, so it comes "for free".

Wouldn't you have to have that file system anyway if you want to transfer python scripts?

Not necessarily if you only store them in RAM, because the Python app already has to manage its memory. To take advantage of left-over space in Flash however, we'd need a way to manage data at the ion layer.

[Zezombye]

Adding apps at runtime should be easy actually, Casio does it fine (for TI I don't know). When you go to the menu (or when you add something to storage memory) the system scans for application files (which could be .app or .nwk for example) and automatically add them to the menu.

Having to recompile the whole OS would be harder, I currently have spent 5 hours trying to build it, still not done yet. It would be a lot easier if you could just transfer a .nwk file to the calculator, and it appears automatically in the menu.

[Zezombye]

Dynamic linking would probably be the best solution, but there will be a hefty penalty in relocation/symbol data overhead and it would be a compatibility minefield without a stable ABI to work on (this is C++ and the firmware is in its early days so there's bound to be plenty of changes). Also, that's far from trivial to implement.

I don't get what you mean by "hefty penalty in relocation/symbol data"; the system just has to query the files, then their icons, then add them to main menu. As for the compatibility, it's up to the app developers to assure it. Obviously right now the apps would crash quickly, but in 1 year/6 months the firmware should be stable enough.

I can see how run-time add-ins in MicroPython bytecode would be OK (easier for students to meddle with than C++, safety of an interpreted language should prevent random crashes, reasonably simple to implement, reuses lots of code already in the firmware, can't tamper with exam mode), but I have yet to hear a truly compelling argument for native run-time add-ins

Native run-time addins are faster than Python bytecode, which is better for real-time games. Also, I think it gives more flexibility.

(also, a native add-in that nukes exam mode without even flashing the calculator would lower the bar to cheating so low it's not even funny).
Obviously the exam mode would have to be protected, but IMO it wouldn't lower the bar to cheating at all. Transferring an addin should take the same level of tech knowledge as flashing a compromised (compiled) OS.

how about an online tool on the NumWorks website that lets you pick what things on a centralized hub you want and let them build the firmware for you?

Why not, it would hold the same purpose as run-time addins (adding apps without having to compile the OS by ourselves). However, if such a tool exists, it means adding apps is possible automatically - and then why not do that at runtime?

If you want to develop some basic apps without the tool-chain then how about grabbing a dumb text editor and do some MicroPython apps (students with no prior programming experience are less likely to be scared away too)?

Basic apps should not be addins, for that the MicroPython interpreter is enough (no need to convert to bytecode).

Once again: why would you need native run-time add-ins and why the NumWorks team should agree to take on non-trivial maintenance burden?

Native run-time add-ins are, I think, easy to do if it is possible to copy-paste a folder into the OS source code and recompile it without changing anything else, because at this point it means you can add apps pretty easily (though I don't know anything about c++ so I could be wrong).

It would be better for high school students, because compiling the OS requires way much more tech knowledge than just transferring an addin. Your solution of a web-based compiler where you could just upload your apps and you get an OS is currently ok. However:

• If Numworks implements calc-to-calc transfer, a student couldn't simply give apps to another student who doesn't have access to a computer.
• If a SD card is supported, there would be no way to hold more apps by swapping them from the SD card to the flash.

[mateoconlechuga]

I'm okay with just choosing applications at compile time; I don't really see a need for supporting run time applications when memory is the limit. I would much prefer python scripts and REPL for programming wise. The only thing I can think of that might be useful is transferring python programs to and from other calculators.

As for C and ASM support, yes, just compile the apps directly into the binary. I see no need now for such modularity.

[Zezombye]

I don't really see a need for supporting run time applications when memory is the limit

How would run time apps take more memory than compiled apps? The code is the same, it should take the same place.

[RedGl0w]

Fixed with epsilon 16 too @artaxxx

[debrouxl]

Well, ""fixed"". While the previously Epsilon 16 betas made it possible to add some native code applications without recompiling the OS, there are severe restrictions on said applications. TI-68k and TI-Z80 FlashApps from 20 years ago offered far fewer restrictions, they were persistent across reboots... IOW, they were usable :)
The scheme used by TI requires only slight adaptation for the modern world with calculators supporting exam mode crap: disable (some) FlashApps in exam mode (unless the given standardized test doesn't mandate it), as opposed to delete them outright. In this day and age, it shouldn't be too hard to make a bulletproof FlashApp metadata parser...

[mateoconlechuga]

In addition to this; because of the limited flash space I think it may be beneficial to allow a more modular approach to applications. Rather than having them embedded in firmware; it would be neat to be able to add/delete them on the fly. Anywho, just an idea.

[rhaamo]

Adding apps at runtine would be hard without SD card support or a builtin secondary non-volatile flash storage.

About modularity on compile-time, what's actually in place isn't a lot, there is hardcoded number of apps, manually adding hooks etc.

For what @boricj is saying about a "hub", that would means some refactoring to just have to do something like "include myapp/Makefile" in "apps/Makefile" and it would be "magically" added.

[adriweb]

AFAIK, the numworks doesn't have any notion of a "filesystem" or "files", so there's nothing to be scanned.
It's a completely different approach. In the current state of things, everything is just a big block of code/data/... with basically no modularity on the user side.

And yes, building on windows is apparently a pain, but then again, for things non-windows, any sane developer would use linux (or mac, but less so), much less trouble :D
@jacobly0 managed to do it though.

[Zezombye]

Yes, obviously there needs to be a file system included, along with a memory manager.

Most high schoolers use windows and aren't power users, so we need to take that in account and design a solution that is user-friendly (for example, a software for file transfer on the numworks, no need to compile or install any dependencies).

[blackketter]

+1 for loadable python apps.

[mateoconlechuga]

No. You basically have to create an entire filesystem for removing, archiving, deleting, protecting, and programing transfer support, and continue to maintain that for the entire life. Not only does that eat up a ton of available flash memory within the firmware, there's no need for it when it can just be compiled in with ease.

[Zezombye]

Wouldn't you have to have that file system anyway if you want to transfer python scripts?

[mateoconlechuga]

No? Not how they are currently stored.

[RedGl0w]

Nowadays the issue should maybe be renamed to "will we be able to have runtime app modularity?"

[artaxxx]

Thanks @RedGl0w !