gavinchiu Goto Github PK

51job

数据抓取/清洗

async-proxy-pool

🔅 Python3 异步爬虫代理池

baize

白泽自动化运维系统：配置管理、网络探测、资产管理、业务管理、CMDB、CD、DevOps、作业编排、任务编排等功能,未来将添加监控、报警、日志分析、大数据分析等部分内容

beepasture

蜜蜂牧场是一个数据采集清洗工具，也是一个ETL工具，同时也是一套脚本语言。

blog

基于django1.10的个人博客

cnblogsspider

用scrapy采集cnblogs列表页爬虫

curlconverter

convert curl commands to python, javascript, php, R

distribute_crawler

使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

django-cms

The easy-to-use and developer-friendly CMS

dlpucsdn

这是我为社团做的网站，基于python的django框架，功能主要有作业发布与查询，资源上传下载与七牛云对接，各个学科的社区：新闻、论坛、博客、科研展示。

ezlog

Easy blog system powered by django

findtrip

机票爬虫（去哪儿和携程网）。flight tickets multiple webspider.(scrapy + selenium + phantomjs + mongodb)

gavinchiu.github.io

gitskills

httpproxymiddleware

A middleware for scrapy. Used to change HTTP proxy from time to time.

interview_python

关于Python的面试题

ipproxypool

IPProxyPool代理池项目，提供代理ip

jd_spider

两只蠢萌京东的分布式爬虫.

keyword_based_sina_weibo_crawler

A web crawler for Sina, search and retrieve microblogs that contain certain keywords 一个简单的python爬虫实践，爬取包含关键词的新浪微博

lbforum

LBForum is a forum engine written in Python using Django

ldapgroup

ldap grouping using python

learngit

luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

misago

Misago is fully featured forum application written in Python and ES6, powered by Django and React.js

mobike-crawler

摩拜单车爬虫

mygit

练习

mytest

pylucid

PyLucid is a Open Source web content management system written in Python using the Django Framework.

python

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

python-gears

A BBS Based On Django.