googlechromelabs / llaminator

Llaminator

Llaminator is an installable, fully offline-enabled PWA that helps you conveniently store and easily access images directly from your browser, desktop, or home screen.

Features

• Small: barely any storage space or network traffic required.
• Convenient: installable and accessible directly from your desktop or home screen.
• Offline: after the initial visit, Llaminator works without any mobile or internet connection.
• Easy: no complicated configuration or confusing interface. Just open Llaminator and view your image.

Installing and building Llaminator

Node v16 or v17 (with NPM v7 or v8, respectively) are used by Llaminator for dependency management, and have to be available on the host operating system. The recommended way to install these is to first install NVM, then use NVM to install the latest NPM+Node by running:

nvm install node

Then check out this repository and initialize your environment by running:

npm install

Running Llaminator

Llaminator is currently made of entirely static code, so there's no complicated server setup necessary. You can use any http server to host the files.

For example, you could run the following and then access Llaminator at http://localhost:4629:

npm run-script serve

Exporting Llaminator as a static site

To compile the typescript, resolve node dependencies, etc, and dump the output into dist/, you can run:

npm run-script build

Note: this probably won't be part of our typical development workflow, so expect it to break from time to time.

Coding Style

The source of truth is npm run-script lint, which is configured by .eslintrc.json. To automatically fix as many lint errors as possible, rather than just printing them out, you can instead run npm run-script fix.

This project expects LF line endings, not CRLF (and this is enforced by eslint). That means that Windows users may want to configure their local .git/config as follows:

[core]
  autocrlf = input

(See https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration for documentation about this option).

If you choose to do the above, remember to also configure your text editor to read and write using LF rather than CRLF. For convenience, this project contains a .editorconfig file capable of setting this up in many popular editors, though some may need plugins. See https://editorconfig.org/ for more details and plugin installation instructions.

Contributing

See CONTRIBUTING for more.

License

See LICENSE for more.

llaminator's Issues

CI - Run tests on CI

With #60, we build the project on CI. It would be nice to test it too.

[Cross-device sync] Firebase sign-in

This will allow proper authentication, with users being able to upload/download only from their own personal bucket (perhaps in a follow-up issue). Apart from being more generally useful, this also creates some semblance of privacy (although the storage bucket owner still has full access).

Typescript-ify the tests

This change adds a test - this test should be converted to TypeScript.

code style/linting rules and enforcement

I imagine there are plenty of node-based linters we can hook up to our build process. Is there a way to enforce it from github-side for pull requests?

@beverloo @andreban

What's our bundle size / performance budget?

Lit is great, and definitely worth it. We've used material-web on top of that, but I'm actually really disappointed about how massively it impacts our bundle size. We're still <90 KiB uncompressed, but it's probably way overkill for our use-case. To illustrate, ~15 KiB of that are for the ripple effects, which are cool, but we don't need to support every edge-case that's significant for a generic library.

Personally I'm not too worried about bundle size, and, to be fair, we've pulled in most significant parts of MWC already, perhaps except for <mwc-input> & friends. Should we set a target size? I know @glennhartmann had concerns here.

Would like some other opinions 😃

llama-item doesn't update properly

When adding a first image, everything works fine.

But adding a new image results in a blank screen rather than showing the new image. Everything looks find on page refresh, though.

[Cross-device sync] Save downloaded firebase items locally so they're available offline

Similar to #102 - simple and straightforward, no sign-in, auth, etc. Just download images from the storage bucket.

User interface for storing new content

Writing this as an issue so that we can get together some of our thoughts. How exciting to work on another great example of engineering driven design 😄

It seems to me that we have a few critical user journeys to address (#2):

1. Storing (new) content in Llaminator.
2. Displaying existing content in Llaminator.
3. Managing existing content in Llaminator. (Delete/Edit/Share)

I'd like to start looking at the first one, storing new content in the app. From a high level, it seems to me that we're looking at something such as the following.

Entry points:

1. Add button within the application.
2. Share action from elsewhere on the system.

I'd like to consider having a floating action button when content is being added from within the application, which opens a file selection activity. This will need some analogous UI on non-mobile platforms, likely just a button.

We'll use the Web Share Target API to enable the second entry point.

Behaviour
When content is being added, we should first display it to the user to confirm that they would like to store this, and allow them to enter an optional name and description for the content. Such information is useful in the UI, for a search function, and can come back when the content is being shared to external sources again. Furthermore, this allows the user to cancel the operation in case they misclicked.

When the user confirms that the content should be added, we store both the file and the metadata in our own cache.

Currently we only store the file contents, and we store that in IndexedDB. We should consider what our options are here, and trade it off against e.g. using the Cache API.

CI - consider using a dependency cache

The setup-node action allows using a dependency cache. Try using that and seeing if it speeds up the CI.

[Cross-device sync] Move Firebase config out of .ts file

Probably into a JSON file with a build-time step to incorporate the JSON contents into the code rather than reading it at runtime

Consider cross-device syncing.

It would be really nice if there were a way for users to access their images across devices. This would also mean that we have persistence if the user's local storage is cleared.

Data storage options for file content and metadata

For each file stored in Llaminator, we'll have (a) the file's contents, and (b) metadata about the file, e.g. its name and description. We need to decide on how this storage layer is going to work.

Requirements

• Storage in our control (i.e. not the browser cache) in which we can get, set and delete arbitrary data.
• The storage should strictly live on the user's device, and not be synchronized.
• We should be able to read metadata without reading file contents.

Improved developer experience over IndexedDB would also be grand, where a library such as IDB would get us quite far already.

generate Trusted Web Activity app

Using https://github.com/GoogleChromeLabs/bubblewrap

example: glennhartmann@c5adbc6

[Cross-device sync] Look into Periodic Background Sync

Instead of syncing on every page load :)

add web share target support

External contributor - first impression

These are the notes I took, at the beginning of the year 2022, documenting my first impressions of getting the project up and running, for someone (like myself) who has no knowledge of the project, and perhaps wants to contribute. Consider it a stress test on the installation procedure.

Maybe none of the stuff mentioned below would be classified as bugs, maybe some would. But this documents the process I went through and what I discovered along the way. Maybe it provides some insight.

First thing I discovered was I didn't have npn installed
sudo apt install npm fixed that.

I then downloaded the zip from github and placed the code into my code directory.

~/code$ node -v
v10.19.0
~/code$ npm -v
6.14.4

So far so good, I guess. Let's try...

~/code/llaminator-main$ npm install

However, the first message is a conflict message. Not the start I was hoping for. Can we git rid of that?

npm WARN conflict A git conflict was detected in package-lock.json. Attempting to auto-resolve.
npm WARN conflict To make this happen automatically on git rebase/merge, consider using the npm-merge-driver:
npm WARN conflict $ npx npm-merge-driver install -g
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!

Next, according to the instructions is to serve the site from an http server, by doing...

~/code/llaminator-main$ npm run-script serve

> [email protected] serve /home/finnur/code/llaminator-main
> webpack serve --open --mode=development

sh: 1: webpack: not found
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! [email protected] serve: `webpack serve --open --mode=development`
npm ERR! spawn ENOENT
npm ERR! 
npm ERR! Failed at the [email protected] serve script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/finnur/.npm/_logs/2022-01-05T17_16_53_308Z-debug.log

Should webpack be listed as a pre-requisite in the instructions?

~/code/llaminator-main$ npm install webpack succeeds. Let's try again...

~/code/llaminator-main$ npm run-script serve

> [email protected] serve /home/finnur/code/llaminator-main
> webpack serve --open --mode=development

[webpack-cli] /home/finnur/code/llaminator-main/node_modules/webpack-dev-server/lib/servers/WebsocketServer.js:10
  static heartbeatInterval = 1000;
                           ^
SyntaxError: Unexpected token =
    at Module._compile (internal/modules/cjs/loader.js:723:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Server.getServerTransport (/home/finnur/code/llaminator-main/node_modules/webpack-dev-server/lib/Server.js:1191:28)
    at Server.createWebSocketServer (/home/finnur/code/llaminator-main/node_modules/webpack-dev-server/lib/Server.js:1757:38)
    at Server.start (/home/finnur/code/llaminator-main/node_modules/webpack-dev-server/lib/Server.js:2305:12)
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] serve: `webpack serve --open --mode=development`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] serve script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/finnur/.npm/_logs/2022-01-05T17_32_58_219Z-debug.log

I'm guessing this is not expected? :)

Set up GitHub actions

To run tests on PRs and report binary size

CI - consider uploading the index-sizes artifact.

Our CI archives the npm-logs if something fails, see if there's any value in archiving the index-sizes artifact as well.

possible feature: annotating (aka "drawing on") images

Should support mouse, touch, stylus (ie, use pointer events)

[Cross-device sync] Implement anonymous upload to Firebase storage

Step 1 of sync capability: anonymous upload - no sign-in, no permissions, no security, and no actual "sync" (just upload, no download)

new upload button rendering

I noticed 2 issues:

1. the upload button doesn't change at all on mouse-hover to provide the "clickable" affordance
2. on desktop, the window always thinks it's slightly too small:

`npm run-script build` output doesn't work

See #22

The error message says that index.ts's "MIME type ('video/mp2t') is not executable."

I'm guessing that's because we're trying to include a ".ts" file as a script, and .ts is being misinterpreted.

make a logo and icons

figure out how to add pdf support

@EiraGe

decide where to host app (and add documentation)

set up TypeScript and build system

[Cross-device sync] Move Firebase stuff out of the storage class

It should be in its own module on top of storage

discussion: overall project, filesystem, build and dev environment, etc. thread

I've noticed that the following PRs have been trying to deal with just getting a build system actually fully working with typescript and service workers (0 new actual functionality in the app):

• #13
• #14
• #17
• #18
• #22
• #26
• #29
• #32
• #33
• another one I'm currently working on to fix Workbox as part of #19

What in the world is going on with this ecosystem?

I think this is a (roughly) exhaustive description of our ideal state:

• dev server that can auto-reload when there are changes to the filesystem
• a way to "export" a copy of the post-build codebase, suitable for hosting elsewhere (including static file servers)
• typescript for both main site code and service worker code, enforcing strict type checks
• some amount of code-sharing ability between the main site code and service worker code (WebShareTarget, as a concrete example, is a place where code sharing would make a lot of sense)
• ability to use 3rd party packages (ideally via npm) in both main site and service worker
• (feel free to add more to this list)

Here's a summary of issues we've encountered along the way (I will probably forget some):

• Workbox requires either bundling, Service Worker modules, or an importScripts() from a CDN
  • Service Worker modules have compatibility issues
    • Uncaught SyntaxError: Cannot use import statement outside a module is such an unhelpful error message at sw installation time
  • bundling is generally the ideal option (failing Service Worker modules) for both Workbox and other npm packages, but @web/dev-server can't auto-reload bundles
  • the CDN doesn't come with types built-in by default, and @types/workbox-sw isn't working for me
• @web/dev-server serves files with their original extensions (like *.ts) rather than their "compiled" extensions (*.js). But when doing a static build, browsers (or maybe just Chrome?) can't seem to figure out that ".ts" isn't 'video/mp2t' (#24)
• different servers have different root directories - maybe there's a way to make all the paths properly relative, but so far we've ended up with a hack as part of the build process
• I'm not entirely convinced that @web/dev-server (and maybe even the rollup config) is actually using our tsconfig.json. Maybe it's ESBuild's fault? No idea.
  • we also may need(?) to mess around with our filesystem and configs like https://github.com/jakearchibald/typescript-worker-example to get the tsconfig.json to work properly for both "normal" code and service worker
• package-lock.json
• it's not entirely clear which type of storage we should be using (#20, #21)
• at first it seemed like declare var self: ServiceWorkerGlobalScope was required to make self work properly in typescript in the service worker. Now, for some reason, it's telling me I can't redeclare self (currently its type is WorkerGlobalScope & typeof globalThis as set by the webworker lib)
• code sharing between main code and sw is difficult and complicated. sw, generally, doesn't support modules, which is what we should be using for the main code. sw modules have compatibility issues. Some sort of bundling build system may(?) be able to mitigate this (see note about @web/dev-server not playing nice with that) but then it's still tricky to figure out which parts are sharable - for example, what type is self? Could change depending on the context. "main" code and sw code are generally built with different libs.

adding, deleting, then re-adding the same image doesn't work

Thank you, also for the demo page!

There is one new issue that I'm observing (but we can address separately): add foo.jpg, delete it, then add foo.jpg again. It refuses to add the image, with no warnings/messages/anything. Interestingly, adding bar.jpg after the delete works fine.

Originally posted by @beverloo in #80 (review)

ServiceWorker is broken

See #17

use persistent storage

See https://web.dev/storage-for-the-web/

style guidelines discussion thread

#64 landed, so we have some default style rules. The question is, do we like them?

Leaving for open discussion for the moment.

saving the image results in it being interpreted as a text file

The blob URL doesn't have a filename or type information, so it saves as a "text file"

video.mp4

[Cross-device sync] Upload/download to "private" per-user storage

Once #102, #103, and #104 are done, we can finally make user-specific storage/sync. The "private" is in quotation marks because the storage bucket owner still has full access.

`false` instead of nothing when platform doesn't support webshare

the word false is written in place of the share button. It shouldn't be.

set up puppeteer for testing

see also: #14 (comment)

decide on and document user journeys

images are cropped

Since #52, I think.

[Cross-device sync] Add optional end-to-end encryption

Following up on #105 - since the storage bucket owner has full access to all users' synced data, I'd like a fully end-to-end encrypted mode. This should be opt-in because the user needs to understand that if they lose their encryption key (aka, forget their password) then they cannot recover their synced data. (It also means they need to provide a password in the first place - I don't think we can reuse their account credentials for this, but maybe there's a way).

make layout as flexible and responsive as possible

We need to support

• desktop
  • including dynamic viewport size changes
• mobile (portrait and landscape)
• tablets
• foldables