Comments (4)
Small points that needs to be confirmed :
- we support only the
all
command offcc2zim
(individual commandsfetch
,prebuild
,zim
) makes no sense in Zimfarm - we need a new platform in addition to the new offliner, in order to limit the number of tasks for fcc2zim per worker
- we limit to 1 fcc2zim task per worker for now (this is not managed in this repo but better ask the question now)
- output flag is not standard: "outpath" instead of "output"
- there is no support for the JSON stat file in fcc2zim (for now)
from zimfarm.
* we support only the `all` command of `fcc2zim` (individual commands `fetch`, `prebuild`, `zim`) makes no sense in Zimfarm
Yes ; ZF doesn't care about scraper internals. We want to be able to supply the list of courses, the metadata and get a ZIM.
* we need a new platform in addition to the new offliner, in order to limit the number of tasks for fcc2zim per worker * we limit to 1 fcc2zim task per worker for now (this is not managed in this repo but better ask the question now)
No. Those are created only if we encounter issues.
* output flag is not standard: "outpath" instead of "output"
Right.
* there is no support for the JSON stat file in fcc2zim (for now)
Right.
from zimfarm.
This task is not possible to complete, work on openzim/freecodecamp#11 needs to be finished (at least first point on CLI parameters) for this task to make sense (otherwise no one but few devs will be able to create recipes for this scraper)
from zimfarm.
I pushed some WIP code on the branch add_fcc
(https://github.com/openzim/zimfarm/tree/add_fcc). Probably mostly ready except Flags / CLI parameters
from zimfarm.
Related Issues (20)
- Clone doesn't set disabled anymore HOT 2
- Jobs that are doing should also list requester (as in the Todo tab)
- Add warning when editor is changing a recipe warehouse path HOT 2
- HTTP 500 error linked to invalid ipv6 while removing secrets
- Again \u0000 cannot be converted to text
- Add capability to recreate a ZIM with updated metadata
- Add concept of scraper release channels HOT 1
- Simplify warehouse list HOT 4
- Zimfarm worker: add support for multiple outgoing IPs HOT 4
- Seems impossible to specify a tag HOT 5
- Fix sotoki scraper configuration
- Watcher is not detecting updated files when SE is fixing issues HOT 3
- Upload logs and artifacts even when task is cancelled
- Name resolution errors in StackExchange watcher
- Configuration of Zimfarm local instance HOT 1
- Installing monitoring of a task is failing HOT 2
- Workers: report real disk size (and CPU / RAM) HOT 6
- StackExchanger watcher regularly restarts after connection reset by peer errors HOT 1
- Is the worker incorrectly checking the space already consumed by the Zimfarm? HOT 8
- Collect zimfarm usage statistics
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zimfarm.