GithubHelp home page GithubHelp logo

spiderapp's Introduction

spider

为练习方便 暂未对项目进行分类，后边如果做的多的话会进行资源整合

安装相关依赖模块

pip install -r requirements.txt

有道翻译fanyi.py

原文链接见有道翻译sign破解

目前只是中英文翻译 给出实例 返回结果

煎蛋网妹子图 jiandan.py

原文链接见煎蛋网爬虫

原代码在输出的时候出现编码错误，经过笔者(就是我)的不懈努力 终于改好了

具体改动可对比代码

qq音乐 qqmusic.py

这个重点讲一下，因为自己做的(傲娇)

翻了很多博客，就不贴地址了

  • 创建了一个qqMusic类（方便管理，增强可读性）
  • 定义两个方法get_json获取排行榜信息, get_link获取歌曲url
  • 整个爬取过程实际上就是模拟浏览器get请求获取json文件

大家在爬的过程中也会发现源码ul中没有内容

一般人到这就换下一家了 ，

但是

这才是重点，越难爬越有挑战才越有动力

通过审查元素/F12的network直接点js抓包。

可以发现所需要的内容都是在js返回的json文件中

那么你所要做的就是

模拟request请求获取json文件

上图中查看headers你可以看到一些信息

  1. headers请求头
  2. params 请求数据（最下边）
  3. url地址

有用的请求头包括user-agent referer

params一些参数是固定不变的

但是

在获取音乐链接的时候通过换音乐对比请求 发现有两个参数是变化的

就是songmid filename

仔细对比songmid, filename发现

filename的值也就比songmid多的前面的 C400

所以要解决的就只是 songmid

songmid在第一个函数获取的内容中就有。 完美!

然后模拟请求获取 vkey值

这里再说一下。 vkey值是针对于你获取的音乐的专属key值 没有这个key值取不到音乐

http://dl.stream.qqmusic.qq.com/C400004RXylR0adkY1.m4a?vkey=FFED438561F460E854B12428B698E9966EFF9EF576D20BA50E85F59AA204E5CE10338EEF9011DEC7B477E3B4BE2F4902A7B9A3659C8C5193
&guid=9384442260
&uin=0
&fromtag=66

最终拼接url

到这里markdown就结束了

spiderapp's People

Contributors

zhaishenking avatar

Watchers

James Cloos avatar

spiderapp's Issues

Weekly Digest (11 August, 2019 - 18 August, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (30 June, 2019 - 7 July, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

This week, no issues have been created or closed.

PULL REQUESTS

This week, no pull requests has been proposed by the users.

CONTRIBUTORS

This week, no user has contributed to this repository.

STARGAZERS

This week, no user has starred this repository.

COMMITS

This week, there have been no commits.

RELEASES

This week, no releases were published.

That's all for this week, please watch 👀 and star ⭐ ZHAISHENKING/spiderapp to receive next weekly updates. 😃

Weekly Digest (1 May, 2019 - 8 May, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (13 October, 2019 - 20 October, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (19 May, 2019 - 26 May, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (6 October, 2019 - 13 October, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (29 September, 2019 - 6 October, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (5 May, 2019 - 12 May, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (9 June, 2019 - 16 June, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (12 May, 2019 - 19 May, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (22 September, 2019 - 29 September, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (2 June, 2019 - 9 June, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (26 May, 2019 - 2 June, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Weekly Digest (16 June, 2019 - 23 June, 2019)

Here's the Weekly Digest for ZHAISHENKING/spiderapp:

ISSUES

Last week, no issues were created.

PULL REQUESTS

Last week, no pull requests were created, updated or merged.

COMMITS

Last week there were no commits.

CONTRIBUTORS

Last week there were no contributors.

STARGAZERS

Last week there were no stargazers.

RELEASES

Last week there were no releases.

That's all for last week, please 👀 Watch and ⭐ Star the repository ZHAISHENKING/spiderapp to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.