ark-builders / ark-android Goto Github PK
View Code? Open in Web Editor NEWReusable components for the ARK project
Home Page: https://www.ark-builders.dev
License: MIT License
Reusable components for the ARK project
Home Page: https://www.ark-builders.dev
License: MIT License
Context:
User may sync or copy manually an ARK-enabled folder from another device. The synced folder should magically appear in "Roots" tab of the filepicker.
User may:
.ark-roots
file in the filesystemHow to solve this? Scanning the whole filesystem every time doesn't look reasonable. Is it ok to do it in background?
Context:
When the user walks through folders using classical mode, they can pin a folder to the "Roots" tab.
If this folder isn't contained in any root folder:
.ark
subfolder must be created in it.If the folder is already contained in an other root:
Tag metric is a function which enables us to order tags. Technically, it can be alphabetical ordering but it isn't really useful conceptually. More convenient metric is popularity of a tag, which stands for amount of resources labeled with the tag.
With linear selector we can use only 1 metric. But with cloud selector we can use up to 3 metrics (see #7).
Possible metrics to implement:
It also should be possible to invert ordering by any metric.
Similar to how filesystem can have parent and child folders, tags should be capable of linking as well.
Each tag will be able to have multiple parents and multiple children. This change makes it possible to construct graphs of tags, where labeling a resource with, e.g. "Pushkin", can automatically label the resource with "Literature" and/or "Russian" tags. This dependency graph is defined only by user, so it is up to them to create graph which makes sense to them.
This dependency graph notion also affects [tag selector]
which must also select all children tags of any selected tag. E.g. if a user selects "Literature", they are also presented by resources labeled as "Pushkin", "Shakespeare", "Remarque" etc. But not in focus mode (see #17).
Graph structure of tags must be persisted and replicated, it can be stored in e.g. .ark/graph
.
Sometimes it is convenient to pin a resource, so it would be shown on top of other resources. Score is more general approach, which allows to prioritize one resource over another and also to deprioritize resources, so resources with negative score would be shown in the bottom of current selection.
Scoring requires separate storage similar to tags storage, but much simpler — file .ark/scores
should contain mapping from Resource ID into integer number (can be positive or negative). Default value should be 0, so if a resource is not mentioned in the file, that implicitly means it's score is 0.
For modifying score of a resource, "boost" buttons should be drawn on top of its preview in GalleryFragment. Just "+" and "-".
The app should have switch in resources grid, enabling/disabling usage of score during resources sorting.
An ARK-enabled folder is either:
.ark
subfolder.ark/favorites
fileSince we use content-addressing, we have special file called index
. It is stored in .ark
subfolder of each root folder. Favorite folders are children of root folders and don't have their own index, they use their root's index
file.
Normally, roots are listed in .ark-roots
file in the disk root. If it doesn't exist then we should scan the whole filesystem (Navigator does this on first launch). Each root folder contains list of its favorite folders in .ark/favorites
file.
The "Roots" tab lists all ARK-enabled folder: both roots, and favorites. It was implemented in scope of this issue:
arklib-android
When a user presses [back] button, the current input is saved. This is fine when we have unfinished input, because it is lost (as expected). But, if we deleted several tags and press [back] button — we don't undo deletion of tags.
Right now, we handle [done] as saving current state into tags storage. After [done] event we leave input field active. We should close the dialog after [done].
If we enter new tag and trigger [done], it becomes "chip". If we press [backspace] this new chip must be deleted. Chips should only be view of text input.
EditTagsDialogPresenter
should store raw input and EditTagsDialogFragment
only draw this input as chips.
Right now, the input stored only in the fragment. We also should store initial state in the presenter. We should not spawn coroutines and write to the storage before [done] is triggered. If [back] is triggered, we should discard all changes (this means no writes to storage was done during work with the dialog).
Similar to ARK-Builders/arklib#10 but for geographical locations. This will be used in ARK Globe and ARK Navigator.
Is it possible to render map previews?
For usage in tag selectors, it would be convenient to have tag stats: kind of metrics, which we can't count just by looking at resources in corresponding collections. Stats can be, e.g. amount of times tag was used or how recently tag was used.
These stats can be kept in Room, but this way stats from one device will not transport to another. Sometimes it might be preferred to treat devices differently though (e.g. a phone and a laptop could be used for different workflows). In terms of performance, we will not have penalty if we will write stats into tags storage (we write to it every time we tag a resource anyway). We can allow a user to set her preferences regarding to persist stats or not.
Statistics to consider:
Here, 1 and 3 are query stats and 2 and 4 are label stats, but I am not sure that this separation of stats is really useful. It is based on the fact that we have query tag selector and label tag selector, they are similar but not the same and we can sort tags in them basing on different stats. On other hand, we can simplify 4 stats into just 2.
About 3 and 4: "how recently" doesn't necessarily mean "time". We can implement these stats using timestamps, but we need to sort tags by them, so might be more practical to just keep stack and put a tag on top of it when it is used.
We now incorporate user-defined metadata known as properties, which are essentially key-value pairs. For instance, a book resource could have a property like "title": "Moby Dick"
. However, property values might differ when sourced from various devices, leading to potential conflicts. In such cases, users must have the option to manually select the correct value to resolve these conflicts. Internally, we represent a property by a key coupled with a list of values. So it may be simply "title": ["Moby Dick"]
in conflict-free scenarios, or "title": ["Moby Dick", "Moby-Dick; or, The Whale"]
when an alternative value exists.
A view for representing such fields would be handy:
When the user selects a value from versions list, all elements from the corresponding Set
must be erased except the selected one.
Sometimes, it would be convenient to filter resources not only by tags but also by their type. In our app, we replace file notion with resource and type with kind. Kinds are more generic: e.g. both JPG
and PNG
are images and a user rarely cares about exact types of images. Usage of kind notion also hints that, technically, it might be determined not only by file extension.
Tag selector in resource grid should provide user with auto-generated virtual tags. For the beginning, these virtual tags must correspond to resource kinds. Virtual tags must be highlighted differently. They are different from normal tags in that it's not possible to label a resource with them manually, and they are not stored. They are similar to tags that every tag can filter selection and can be composed with other tags. This implies that virtual tags should also be sorted the same way as other tags in the selector.
From UI perspective, button in tag selector also should be created to turn on/off both generation and displaying of virtual tags.
Virtual tags must be also provided for unlabeled selection, i.e. when no tags are selected. If the user didn't turn virtual tags off, of course.
For cloud selector (see #7) in 2D mode, we can do tricky scrolling to fit only part of the cloud on the screen.
Let's say that we use distance from center to represent some metric (freshness or coupling from #13 ). When we have too much of tags, it is difficult to fit them all on the screen, so we would show the most central tags and place the rest beyond the screen. Now, imagine that we want to show the border of the cloud. We can just zoom-out everything in the middle of it, releasing screen space for the cloud border.
The middle of tags cloud can be shown by zooming-out everything what is closer to the center, and putting everything what is further from the center beyond the screen. Let's present a user with a slider allowing her to choose how much tags to zoom-out and by moving the slider it becomes possible to overview the whole cloud.
When maximized, tags selector should reveal query input box.
Query itself consists of tags and should be just a quicker way to produce the same result as tags selector.
Maybe just motion speed thresholds should be adjusted, but it won't hurt to overhaul the whole gesture handling.
This issue is continuation of #21
Videos of how does it work right now:
If arklib
fails to fetch image while generating preview for a link, it creates empty file instead.
Would be great to allow pinning of tags in the selector.
Pinned tags should be stored as user data per each root.
Move RootPickerDialogFragment here from Navigator
The functionality of the RootPickerDialogFragment and the ArkFilePickerViewModel.pinFile() are the same. In fact, we can add root/favorite from two different places. We need to reuse the general logic and prohibit adding root if there is already a root inside.
We also need to prohibit adding device root as ark root.
The most important metadata values would be date of creation, location, device. Dimensions could be scanned as well, but not so useful. Date of creation would allow to sort images by true date and not date of modification. EXIF should be used.
When tags selector is maximized, tabs with alternative selectors must appear. One of alternative selectors is cloud selector, and other is graph selector. Both should switch to default linear one upon dragging selector down.
Graph selector represents a graph with tags as its vertices. When two tags A and B intersect (there are resources labeled with both of them) then we have edge between A and B.
A user should be able to sort tags in tags selector using various metrics (see #13).
In order to achieve this, metric switch needs to be implemented.
For the start, it can be just toggle between popularity and freshness metrics.
There is a "tags selector" in resources grid. It should be possible to maximize it by dragging its edge up.
In maximized state, a user still can select tags but resources grid is not visible, hence not necessary to update previews. But the user should receive feedback about his actions, so number of resources selected should be displayed.
Port it from https://github.com/ARK-Builders/arklib-android
The arklib-android
repo contains cache-generating pipeline which should be decoupled from the core since it's optional.
It has 3 main parts:
The codebase needs some refactoring, too.
For the start, we just need simple function extracting text content from documents.
We need these to complete first:
App get crashed when we do fast scroll and click on tab.
This repo will collect individual modules, each module should be possible to use on its own as a dependency.
filepicker-android
and filepicker-ios
(in future)Approximate module structure should be something like this:
ark-components
|- data-link
|- data-link-android
|- data-link-ios
|- data-pdf
|- data-pdf-android
|- data-pdf-ios
|- data-images
|- data-images-android
|- data-images-ios
|- filepicker-android
|- tag-selector-android
|- tag-selector-tauri
|- score-widget-android
|- score-widget-tauri
Here, data-link
and data-images
are examples of future "data components". Such components will bring particular resource kinds, i.e. will allow handling files of certain type. They can denote a single file type like .link
, or a group of file types like images:
It's not completely clear to me if we should be cautious about dependencies between components.
E.g. component B depends on component A:
Can A and B be used in an app at the same time? Problems with versions could arise.. If usage of connected components poses some limitations, they must be clarified in README, e.g. if the user should inject only B.
Would be more efficient to define GitHub Actions workflows in such a way that B and A run one after another, not separately since building B would mean building A again.
Our score widget isn't that pretty. There is a tiny lib which does the same thing but looks nice: https://github.com/DanielMartinus/Stepper-Touch We can just import it and implement correct handlers.
Bonus feature: increment/decrement in the Stepper-Touch lib are triggered by swipes to left or right. If we could mix additional "swipe and hold" logic into it, that would be awesome. So, when the user wants to increment more than by 1, we could either simply start incrementing further until the user releases the swipe, or we could display additional score slider:
Right now, every time we change tags, we count their popularity by resources in TagsSelectorPresenter
.
We can collect this statistics in StatsStorage
and use it right away
But before that, we need to make sure that statistics are collected correctly.
For example, deleting a resource without our app. In this case, we need to reduce popularity of tags of this resource.
private fun sortByPopularity(
tagsOfSelectedResources: List<TagItem>,
tagsOfUnselectedResources: List<TagItem>,
tagItemsByResources: Map<ResourceId, Set<TagItem>>
) {
val tagsOfSelectedResPopularity = Popularity
.calculate(tagsOfSelectedResources)
val tagsOfUnselectedResPopularity = Popularity
.calculate(tagsOfUnselectedResources)
val tagsPopularity = Popularity
.calculate(tagItemsByResources.values.flatten())
code:LINK
field is missing in kind json file for links
See #17 for definition and examples of Focus mode.
This mode should be useful to explore loosely-connected resources and mark them with more tags.
Tags selector must be hidden when it is empty. E.g. at the moment, when a user presses "show untagged only" button, she sees empty selector.
Let's call tags that we have right now as quick tags.
It would be nice to implement more powerful concept of dedicated tags.
Quick tags:
Dedicated tags:
[tag menu]
[dedicated tag editor]
dialogBoth quick and dedicated tags will have id which is a word without spaces and special characters. Id must be recognized as the tag in all text inputs ([tag selector]
, [tags editor]
dialog in Gallery). These text inputs should also support finding the tag by typing only a part of the title (see #20) or typing it's parent tag.
White spaces must become a kind of separator in [tags editor]
dialog (Gallery), the same as comma. Special characters should highlight unfinished input as invalid.
Port it from https://github.com/ARK-Builders/arklib-android
The app uses "tags selector" to specify tags, i.e. it outputs some set of tags.
But there are multiple ways to interpret a selection into subset of resources.
Of the following 1, 2 and 5 seem to be most natural and modes 3, 4 might be useful sometimes.
Normal mode (intersection)
Every resource which is marked with all of specified tags is displayed.
I.e. resources which contain extra tags are also displayed.
Focus mode
This mode is stricter: only those resources which are marked with all
of specified tags and none of non-specified tags are displayed.
I.e. resources which contain extra tags are filtered-out.
Extra mode
This mode is derived from 1 and 2: we apply Normal mode
and filter-out all resources from Focus mode.
I.e. resources which are marked with all of specified tags
and at least one of non-specified tags are displayed.
We can also, on the opposite, define Focus mode via Extra mode.
Inversion mode
This mode is inverted Normal mode: we take the whole
collection and filter resources which are marked with all of specified tags out.
I.e. we display only those resources which are not marked with any of specified tags.
Union mode
We display all resources marked with at least one of specified tags.
To illustrate these modes, let's take some collection of resources:
Examples of queries on this collection:
Empty query (beginning of user's interaction)
Normal mode outputs 1, 2, 3, 4, 5, 6, 7, 8
Focus mode outputs 8
Extra mode outputs 1, 2, 3, 4, 5, 6, 7
Inversion mode outputs nothing
Union mode outputs nothing
Single tag query, e.g. {a}
Normal mode outputs 1, 4, 6, 7
Focus mode outputs 1
Extra mode outputs 4, 6, 7
Inversion mode outputs 2, 3, 5, 8
Union mode outputs 1, 4, 6, 7
Two tags query, e.g. {a, b}
Normal mode outputs 4, 7
Focus mode outputs 4
Extra mode outputs 7
Inversion mode outputs 1, 2, 3, 5, 6, 8
Union mode outputs 1, 2, 4, 5, 6, 7
Three tags query, e.g. {a, b, c}
Normal mode outputs 7
Focus mode outputs 7
Extra mode outputs nothing
Inversion mode outputs 1, 2, 3, 4, 5, 6, 8
Union mode outputs 1, 2, 3, 4, 5, 6, 7
Should be possible to set a tag as favorite.
Favorite tags must be represented in linear tags selector before any other tags.
Favorite tags must be ordered by popularity, the same as other groups of tags.
We have the scoring widget, consisting of 3 elements:
[inc_score]
button (arrow up)[curr_score]
label (star with a number inside)[dec_score]
button (arrow down)The problem is that when a user has a lot of scored resource in her collection, in order to "pin" some resource it is necessary to tap multiple times on inc_score
button.
Let's add this behaviour:
User has opened a collection of resources.
Some resources have scores set.
Let's denote the highest score of the collection as MAX
.
User is looking at some particular resource.
Let's denote it's score as CURR
.
User taps [inc_score]
and holds it.
Slider [set_score]
appears.
It has values from CURR
to MAX + 1
in it.
Default value is MAX
.
User has several options now:
a. Release the button: MAX
score is set to the resource.
b. Slide one up and release: MAX + 1
score is set.
c. Slide to anything below and release: any chosen score is set.
(only values from CURR
to MAX + 1
are available)
This happens when clicking on Navigate data
in Navigator
app with some PDF files.
Related: ARK-Builders/ARK-Navigator#412
Cmdline: dev.arkbuilders.navigator
pid: 11803, tid: 12465, name: DefaultDispatch >>> dev.arkbuilders.navigator <<<
uid: 10716
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
x0 0000000000000000 x1 00000000000030b1 x2 0000000000000006 x3 00000079040392d0
x4 00000000ebad808a x5 00000000ebad808a x6 00000000ebad808a x7 3f8000003f800000
x8 00000000000000f0 x9 8246067d94e6f48d x10 0000000000000000 x11 ffffff80fffffbdf
x12 0000000000000001 x13 0000000000000070 x14 0000000071931070 x15 0000007904039138
x16 0000007bbeb8d050 x17 0000007bbeb69eb0 x18 fffffffffffffff8 x19 0000000000002e1b
x20 00000000000030b1 x21 00000000ffffffff x22 000000789d838850 x23 0000007904039580
x24 000000789ceab8a0 x25 000000789d8f9a60 x26 000000790403b21c x27 000000790403b210
x28 000000790403b110 x29 0000007904039350
lr 0000007bbeb1cba0 sp 00000079040392b0 pc 0000007bbeb1cbcc pst 0000000000000000
backtrace:
#00 pc 000000000004fbcc /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: ba489d4985c0cf173209da67405662f9)
ARK-Builders/arklib#1 pc 0000000000bf3674 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (std::sys::unix::abort_internal::h4cbe40d43106b45b+4)
ARK-Builders/arklib#2 pc 0000000000bf0e40 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (rust_panic+76)
ARK-Builders/arklib#3 pc 0000000000bf0c64 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (std::panicking::rust_panic_with_hook::h9c80bfe4d552cb32+660)
ARK-Builders/arklib#4 pc 0000000000bf09b0 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::haece24ab2b06546c+196)
ARK-Builders/arklib#5 pc 0000000000bef6a4 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (std::sys_common::backtrace::__rust_end_short_backtrace::h5255d8dea90e7932+4)
ARK-Builders/arklib#6 pc 0000000000bf0700 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (rust_begin_unwind+108)
ARK-Builders/arklib#7 pc 0000000000c0faac /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (core::panicking::panic_fmt::h61a82d7f25335953+44)
ARK-Builders/arklib#8 pc 0000000000c0fe5c /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (core::result::unwrap_failed::h0973c0cfe8c3b00d+88)
ARK-Builders/arklib#9 pc 00000000004461dc /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (core::result::Result$LT$T$C$E$GT$::unwrap::hd97d7155f1a185bd+152)
ARK-Builders/arklib#10 pc 0000000000448f60 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (arklib::pdf::render_preview_page::he4a0f3d9b25ab6ef+1860)
ARK-Builders/arklib#11 pc 0000000000435dd4 /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!libarklib.so (Java_dev_arkbuilders_arklib_LibKt_pdfPreviewGenerateNative+1768)
ARK-Builders/arklib#12 pc 0000000000461554 /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+148) (BuildId: 12e00d030bcfeb51f978c01791e0cd24)
#13 pc 0000000000209398 /apex/com.android.art/lib64/libart.so (nterp_helper+152) (BuildId: 12e00d030bcfeb51f978c01791e0cd24)
ARK-Builders/arklib#14 pc 00000000002468bc [anon:dalvik-classes16.dex extracted in memory from /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!classes16.dex]
ARK-Builders/arklib#15 pc 0000000000209334 /apex/com.android.art/lib64/libart.so (nterp_helper+52) (BuildId: 12e00d030bcfeb51f978c01791e0cd24)
ARK-Builders/arklib#16 pc 000000000025120e [anon:dalvik-classes16.dex extracted in memory from /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!classes16.dex]
ARK-Builders/arklib#17 pc 000000000020a254 /apex/com.android.art/lib64/libart.so (nterp_helper+3924) (BuildId: 12e00d030bcfeb51f978c01791e0cd24)
ARK-Builders/arklib#18 pc 0000000000251092 [anon:dalvik-classes16.dex extracted in memory from /data/app/~~UopwbnBLZsui9cjV_wT07g==/dev.arkbuilders.navigator-HVeIdiZtVWBKhm-dfxJI2A==/base.apk!classes16.dex]
ARK-Builders/arklib#19 pc 00000000020bd4b8 /memfd:jit-cache (deleted)
It would be nice to make render
subcommand capable of running in batch mode. At this moment, it can render preview for a single resource. If we could render previews for all resources in the folder, we could:
When maximized, tags selector should be capable of switching into alternative mode visually representing tags as a cloud. This mode will be called cloud selector and normal mode will be called linear selector.
Both modes should operate on the same internal state, only visual representation should differ. Cloud selector allows to use more than 1 metric for ordering tags. The same as linear selector, cloud selector must hide or gray-out tags which are not actual with the current query.
In cloud selector, tags can have different sizes to represent some metric.
Cloud tags selector itself can come in 2 flavors:
Would be nice to add additional selector in tags editing dialog, used for putting tags into input field.
This new selector should be located under text input field and consist of the most frequent tags across the whole storage.
When a user types text into the field, quick tags must be filtered by prefix. This filter must be reset upon entering the tags separator (comma).
Text inputs working with tags, like [tag selector]
in Resource Grid and [tags editor]
in Gallery, should be able to work with not only full id, but also with parts of title in case of dedicated tags (#257). Tags with parents (#255) also should be possible to be found by typing their parent id or title.
Any resource should have "score" modifiable by user, which can be used in sortings.
This score must be implemented as separate table .ark/scores
and bind resource id to an integer number.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.