GithubHelp home page GithubHelp logo

snakehacker / tuixue.online-visa Goto Github PK

View Code? Open in Web Editor NEW

This project forked from trinkle23897/tuixue.online-visa

0.0 0.0 0.0 12.24 MB

https://tuixue.online/visa/ A Real-time Display of U.S. Visa Appointment Status Website 预约美帝签证各个签证处最早时间的爬虫

Home Page: https://tuixue.online/visa

Shell 0.02% JavaScript 21.58% Python 61.79% PHP 15.78% HTML 0.38% Less 0.44%

tuixue.online-visa's Introduction

美国签证预约时间 自动化爬取+推送通知

https://tuixue.online/visa/

包含国内以及国外各种地区,目前包含:

  • CGI系统:**、柬埔寨、新加坡、韩国、越南、巴拿马、澳大利亚、日本、尼泊尔、泰国、德国、多米尼加、哥斯达黎加、摩洛哥、印度
  • AIS系统:英国、加拿大、阿联酋、厄瓜多尔、法国、塞尔维亚、土耳其、希腊、哥伦比亚、墨西哥、巴哈马

Overview

目前采用的是部分前后端分离,姑且将其称之为前端服务器和爬虫服务器。整体逻辑为前端服务器定时向爬虫服务器发送数据请求,拉取到其本地、更新数据库、推送通知。

爬虫服务器使用Django构建,代码位于 api 文件夹下,CGI系统使用纯requests拉取数据,AIS系统由于有recaptcha2,使用requests和selenium/xdotool混合模式拉取数据(感谢z3dd1cu5

前端服务器采用nginx部署,使用python和爬虫服务器进行交互(详见lite_visa.py),和用户交互使用静态html或者php

前端服务器采用Nginx部署,使用FastAPI前后端分离,使用python和爬虫服务器进行交互(详见backend/visa_status_fetcher.py),使用mongo存储数据,前端网页使用React+Redux,展示表格使用EChart,评论区使用Disqus第三方服务,使用WebSocket+WebNotification实现网页版穿透通知(感谢BenjiTheC

邮件通知、QQ群和Telegram channel通知均使用python实现,详见backend/notifier.py

Usage

爬虫的使用请移步 api,包括各种请求字段,甚至免费的CGI验证码破解器也有(准确率大概98%)

前端部署详见 backendfrontend

API

https://api.tuixue.online/docs

请善待它orz,频繁请求会造成服务器卡死,建议一分钟一次请求(不过都做了网页版穿透通知了……直接开着网页或者加了QQ群、Telegram频道就能收到通知,为啥还要自己造个轮子呢?

如有需要请使用requests之类的包直接从api拿取数据,请不要用selenium定时刷新,否则直接封ip(因为会产生过多无用的WebSocket请求导致服务器莫名其妙挂掉……)

tuixue.online-visa's People

Contributors

trinkle23897 avatar benjithec avatar z3dd1cu5 avatar dependabot[bot] avatar joglelew avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.