Comments (15)
@davidar this is so, so, so cool! I just read a book on IPFS! \o/
Yeah please do.
@davidar knows this, but note to everyone else: please be careful to make sure books we rehost are ok to be rehosted by us (i.e. look at the license/copyright)
We should talk to archive folks about using IPFS to replicate all the archive's books sooner than the rest of the stuff.
cc @brewsterkahle in case he checks github.
from archives.
so great!
yes, we can help with selecting materials.
-brewster
On 9/13/15 4:03 PM, Juan Benet wrote:
@davidar https://github.com/davidar this is so, so, so cool! I just
read a book on IPFS! \o/Yeah please do.
@davidar https://github.com/davidar knows this, but note to everyone
else: please be careful to make sure books we rehost are ok to be
rehosted by us (i.e. look at the license/copyright)We should talk to archive folks about using IPFS to replicate all the
archive's books sooner than the rest of the stuff.cc @brewsterkahle https://github.com/brewsterkahle in case he checks
github.β
Reply to this email directly or view it on GitHub
#14 (comment).
from archives.
Wow, @brewsterkahle commenting on one of my issues :D
The problem I ran into was browsers still not supporting JPEG2000 for some inexplicable reason, so I guess the best strategy would be to batch convert everything to regular JPEG first? I also tried some javascript JPEG2000 decoders, but they seemed way too slow (especially on mobile).
+1 for getting all the scanned books into IPFS, that would be amazing.
from archives.
Yes, the archive does server-side reformatting of pages (turns out jpg
is not very good for scanned book pages).
This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.
Marcel van der Peijl [email protected] did a search engine for my
blog:
http://gateway.ipfs.io/ipfs/QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr/
in the case of the book reader, I think we would have to download the
jp2's and convert a few of them ahead of time. the book reader
currently does something similar by predownloading a set of pages.
without that lookahead, the performance is klunky.
-brewster
On 9/14/15 1:42 AM, David A Roberts wrote:
Wow, @brewsterkahle https://github.com/brewsterkahle commenting on one
of my issues :DThe problem I ran into was browsers still not supporting JPEG2000 for
some inexplicable reason, so I guess the best strategy would be to batch
convert everything to regular JPEG first? I also tried some javascript
JPEG2000 decoders, but they seemed way too slow (especially on mobile).+1 for getting all the scanned books into IPFS, that would be amazing.
β
Reply to this email directly or view it on GitHub
#14 (comment).
from archives.
There is https://archive.org/details/gutenberg, which can be readily mirrored with https://www.gutenberg.org/wiki/Gutenberg:Mirroring_How-To (s/rsync/ipfs/).
from archives.
@rht I already mirrored a small subset of Gutenberg a little while ago (check the irc logs), so mirroring the rest is definitely on my radar. Part of the reason I've been holding back on that so far is that I think it would be a really cool demonstration of the search engine #8 when it materialises.
from archives.
@brewsterkahle I guess a combination of:
- low quality jpeg versions (thumbnails)
- aggressive preloading and rendering of high quality jpeg2k
where the user is shown 1 while waiting for 2 to load, might work quite well in practice.
@jbenet I'm still concerned about mobile performance, but there may be other things that can be done to help with that. For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries). JavaScript is OK for this on a powerful computer, but not so much on mobile.
from archives.
This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.
I'm also quite interested in this, but on a larger scale, see #8 and ipld/go-ipld-deprecated#8
from archives.
For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries)
@davidar i think we can do everything in javascript. https://github.com/ipfs/node-ipfs is not that far from landing. (the one stopgap is chrome/ff extensions with go-ipfs bundled to resolve ipfs links)
JavaScript is OK for this on a powerful computer, but not so much on mobile.
Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(
from archives.
Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(
Thanks, that would be great.
from archives.
BPG could be another option: the JS decoder for BPG seems noticeably faster than the JP2 ones, and it also seems to have slightly better image quality for similar files sizes (demo).
from archives.
If there's anything I can do to help, I am π―% in.
from archives.
@RichardLitt I'm trying to experiment with BPG (here) but I keep getting Cannot enlarge memory arrays
errors. I've tried recompiling bpgdec.js
with emscripten's -s ALLOW_MEMORY_GROWTH=1
flag, but that's also erroring (I suspect my LLVM package might be too old). If you were able to get this working, I'd really appreciate it :)
Alternatively, if you wanted to get the bookreader working with a JS JPEG2000 decoder, that would also be great.
from archives.
I've created a new issue (ipfs/apps#2) to discuss getting BookReader working on IPFS.
I'll leave this issue open to discuss actually getting scanned books mirrored to IPFS (in a format compatible with the BookReader).
from archives.
re: #14 (comment)
You can use the Archive's IIIF API to fetch Archive.org book pages on the fly in any format, e.g.:
https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.jpg
https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.png etc.
The syntax is iiif/:id$page#
from archives.
Related Issues (20)
- Datasets hosted by cloud providers
- mirrors-instructions have outdated / wrong instructions HOT 1
- KinoΓ―t IPFS-Pack (KinoKabaret Caen) HOT 1
- Software Heritage HOT 1
- Chellenge: 22.5 Trillion Digits of Pi, and Prime Numbers up to one Trillion HOT 2
- Old Internet Files
- Making sure archives are fetchable HOT 11
- Deprecate archives.ipfs.io? HOT 13
- High-impact scientific papers
- Waves in plasmas HOT 1
- Geocities HOT 3
- Congressional Research Service reports (scientific + climate papers) HOT 1
- Godot downloads
- Talks from Our Networks 2018
- It is possible to replace LOCKSS with IPFS/Archives? HOT 1
- LibriVox, free public domain audiobooks HOT 1
- Twitter Datasets on IPFS
- basic works of aristotle
- 10.1016/j.tsf.2007.04.165
- greek literature
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from archives.