GithubHelp home page GithubHelp logo

sinkaroid / jandapress Goto Github PK

View Code? Open in Web Editor NEW
51.0 9.0 17.0 1016 KB

RESTful and experimental API for nhentai and other doujinshi

Home Page: https://janda.merahputih.moe

License: MIT License

TypeScript 99.84% Dockerfile 0.16%
hentaifox nhentai pururin hentai2read 3hentai asmhentai simply-hentai

jandapress's Introduction

jandapress

RESTful and experimental API for the doujinboards

Jandapress was named JCE (Janda Cheerio Express) and definitely depends on them.
The motivation of this project is to bring you an actionable data related doujin with gather in mind.

PlaygroundContributingReport Issues


The problem

You enjoy consume doujin sites to build web applications. There are a lot sites that have effort especially pururin, simply-hentai and etc, not official api available nor public resource that can be used for everyone. Instead making lot of abstraction and enumerating them manually, You can rely on jandapress to make less of pain. The current state is FREE to use, meant all anonymous usage is allowed no aunthentication required and CORS was enabled.

The solution

Features

  • Gather the most doujin sites
  • Objects taken that are consistent structure, almost
  • Objects taken is re-appended to make extendable
  • All in one: get, search, and random methods
  • In the future we may implement JWT authentication
  • Pure scraping, except nh sigh..

Jandapress vs. the whole doujin sites

Features availability that Jandapress has

Site Status Get Search Random
nhentai Nhentai
pururin Pururin
hentaifox Hentaifox
hentai2read Hentai2read
simply-hentai Simply-hentai
asmhentai Asmhentai
3hentai Asmhentai
nhentai.to Nhentaito

Prerequisites

NOTE: NodeJS 16.x or higher

To handle several requests from each web, You will also need Redis for persistent caching, free tier is available on Redis Labs, You can also choose another provider as we using keyv Key-value storage with support for multiple backends. All data must be stored in <Buffer> here.

Installation

Rename .env.schema to .env and fill the value with your own

# railway, fly.dev, heroku, vercel or any free service, NHENTAI_IP_ORIGIN should be true
RAILWAY = sinkaroid

# default port
PORT = 3000

# backend storage, default is redis, if not set it will consume memory storage
REDIS_URL = redis://default:somenicepassword@redis-666.c10.us-east-6-6.ec666.cloud.redislabs.com:1337

# ttl expire cache (in X hour)
EXPIRE_CACHE = 1

# nhentai strategy
# default is true which is assign to request on IP instead of nhentai.net with cloudflare
# if you have instance like vps you need chromium or firefox installed and set it to false
NHENTAI_IP_ORIGIN = true

# you must set COOKIE if NHENTAI_IP_ORIGIN is false, read the jandapress docs 
COOKIE = "cf_clearance=l7RsUjiZ3LHAZZKcM7BcCylwD2agwPDU7l9zkg8MzPo-1676044652-0-250"

# you must set USER_AGENT if NHENTAI_IP_ORIGIN is false, read the jandapress docs
USER_AGENT = "jandapress/1.0.5 Node.js/16.9.1"

Docker

docker pull ghcr.io/sinkaroid/jandapress:latest
docker run -p 3000:3000 -d ghcr.io/sinkaroid/jandapress:latest

Docker (your own)

docker run -d \
  --name=jandapress \
  -p 3000:3000 \
  -e REDIS_URL='redis://default:somenicepassword@redis-666.c10.us-east-6-6.ec666.cloud.redislabs.com:1337' \
  -e EXPIRE_CACHE='1' \
  -e NHENTAI_IP_ORIGIN='false' \
  -e COOKIE='cf_clearance=AbcDefGhijY7RYSKv3YeJUjrI5xQ2Uc-666-0-250' \
  -e USER_AGENT='jandapress/1.0.5 Node.js/16.9.1' \
  ghcr.io/sinkaroid/jandapress:latest

Manual

git clone https://github.com/sinkaroid/jandapress.git
  • Install dependencies
    • npm install / yarn install
  • Jandapress production
    • npm run start:prod
  • Jandapress testing and hot reload
    • npm run start:dev

Nhentai Guide

The problem

https://nhentai.net was Clouflare protection enabled, for default jandapress use real IP address to bypass the protection, but sometimes even it's from IP address the /api path return error that means admins or their maintainer don't allow us to request from the IP address. image

The solution

You will need instance such as VPS and install Chrome or Chromium or Firefox, You have to set NHENTAI_IP_ORIGIN to false, set COOKIE and USER_AGENT. We'll simulate the request with tough-cookie and http-cookie-agent

image

  • set NHENTAI_IP_ORIGIN to false in .env file
  • open browser and go to https://nhentai.net
  • verify you are human
  • open devtools and set custom user agent
  • reload the page and wait cloudflare again
  • open devtools and go to network tab and request
  • get the cf_clearance value and set it to COOKIE in .env file
  • set the user agent to USER_AGENT in .env file
  • test that your cookie is working npm run test:cf
    • it should return 200 status code otherwise watch your step

The documentation said and correct me if I'm wrong:

This cookie expires after 30 minutes of continuous inactivity by the end user. The cookie contains information related to the calculation of Cloudflare’s proprietary bot score and, when Anomaly Detection is enabled on Bot Management, a session identifier.

└── https://developers.cloudflare.com/fundamentals

You will need to make your cookie is not expired otherwise manual update is required, it can be with set interval or cron job to automate your request.

Running tests

Jandapress testing

Start the production server

npm run start:prod

Running development server

npm run start:dev

Check the whole sites, It's available for scraping or not

npm run test

Check nhentai It's under cloudflare protection or not

npm run test:cf

Generating playground like swagger from apidoc definition

npm run build:apidoc

To running other tests, you can see object scripts in file package.json

Playground

https://sinkaroid.github.io/jandapress

  • These parameter?: means is optional

  • / : index page

Nhentai

The missing piece of nhentai.net - https://sinkaroid.github.io/jandapress/#api-nhentai

Pururin

The missing piece of pururin.to - https://sinkaroid.github.io/jandapress/#api-pururin

Hentaifox

The missing piece of hentaifox.com - https://sinkaroid.github.io/jandapress/#api-hentaifox

Asmhentai

The missing piece of asmhentai.com - https://sinkaroid.github.io/jandapress/#api-asmhentai

Hentai2read

The missing piece of hentai2read.com - https://sinkaroid.github.io/jandapress/#api-hentai2read

Simply-hentai

The missing piece of simply-hentai.com - https://sinkaroid.github.io/jandapress/#api-simply-hentai

3hentai

The missing piece of 3hentai.net - https://sinkaroid.github.io/jandapress/#api-3hentai

Nhentai.to

The missing piece of nhentai.to - https://sinkaroid.github.io/jandapress/#api-nhentaito

Status response

"success": true, or "success": false,

HTTP/1.1 200 OK
HTTP/1.1 400 Bad Request
HTTP/1.1 500 Fail to get data

Frequently asked questions

Q: The website response is slow

That's unfortunate, This repository was opensource already, You can host and deploy Jandapress with your own instance. Any fixes and improvements will updating to this repo.

Q: I dont want to host my own instance

That's unfortunate, Hit the "Sponsor this project" button, any kind of donations will helps me to funding the development.

Pronunciation

id_ID/jan·da/ — Dewasa dan mengikat; (?)

Client libraries / Wrappers

Seamlessly integrate with the languages you love, simplified the usage, and intelisense definitions on your IDEs

Legal

This tool can be freely copied, modified, altered, distributed without any attribution whatsoever. However, if you feel like this tool deserves an attribution, mention it. It won't hurt anybody.

Licence: WTF.

jandapress's People

Contributors

sinkaroid avatar soujiokita avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jandapress's Issues

separate between tags, character, author etc..

Describe new features you want

it is possible for next update to filtering tags by category, artists, etc?

Additional context
like this
image
A Gif to describe how you feel when you made this request
image

bug nhentai origin IP site not worked

Error
When deploy on heroku used mirror nhentai site to get doujin. But the site maybe already down so didnt worked when get doujin.

There is another nhentai unofficial site, nhentai.to maybe can used for replacing old one.

cant bypass cloudflare

I've installing chromium with snap, set cookies and user agent, but still got http error 403 when try npm run test:cf.

Can you please make details step by step to install chrome/chromium or firefox and set cookies.

  • OS: Ubuntu 20.04
  • Node version 18.17.0

Request support for 3hentai.net

Requesting support for 3hentai.net, it have some old manga that not available on other platform, the server also seems faster than NH. Thanks.

randomAsmHentai bug

Describe the bug
When making a request to the /random endpoint of asmhentai it returns every time (atleast all the times i tried it, maybe it works sometimes) an error

To Reproduce
Make a request to the /random endpoint of asmhentai

Expected behavior
Hentai json return

What happens
I get this error:
Error: TypeError: Cannot read properties of undefined (reading 'replace') at randomAsmhentai (/home/ubuntu/jandapress/build/src/controller/asmhentai/asmhentaiRandom.js:24:14) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Desktop (please complete the following information):

  • OS: Ubuntu 22.04
  • Node version: 18.12.1
  • Typescript version: 4.6.3

Additional context
The error seems to be cause by the line upload_date: date[1].replace("Added: ", ""), to fast-patch it i put a ternary condition like this: upload_date: date[1] ? date[1].replace("Added: ", "") since i don't really know the structure of the code

Can't connect to janda.mod.land

Hi,
I think janda.mod.land is down again?
Janda is not working. I received following error:
requests.exceptions.SSLError: HTTPSConnectionPool(host='janda.mod.land', port=443): Max retries exceeded with url: /nhentai/get?book=XXX (Caused by SSLError(CertificateError("hostname 'janda.mod.land' doesn't match '*.up.railway.app'")))

[bug] Error: Unexpected token

Describe the bug
When I try running this, I got the error like this image

Desktop (please complete the following information):

  • OS: [Ubuntu 20.04]
  • Node version [18.12.1]
  • Typescript version [4.9]

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.