Comments (3)
感谢您的支持,bug已经修复,但是你最后说的那个问题,看不懂,解决办法:在 重试url 入队之前,将 url 移出 self::$collect_urls
from phpspider.
你原来的代码:
爬取(url)
self::$collect_urls[md5(url)]=time();
爬取失败,重新入队 url
准备爬取 url,但是发现 self::$collect_urls[md5(url)] 已经定义,所以就不爬取了
修复:
爬取(url)
self::$collect_urls[md5(url)]=time();
爬取失败,重新入队 url,并且 unset(self::$collect_urls[md5(url)])
准备爬取 url,self::$collect_urls[md5(url)] 未定义,爬取
from phpspider.
感谢支持,已经修复
from phpspider.
Related Issues (20)
- 验证码识别问题 HOT 1
- 我在windows环境下运行了demo下的马蜂窝 HOT 2
- [error] Domain of scan_urls ("https://bbs.zhibo8.cc/forum/list/?fid=62") does not match the domains of the domain name
- 建议用swoole HOT 1
- 文档里的某个xpath不起作用 HOT 3
- redis、mysql 执行长都出现了超时的情况
- 有遇到这个问题的吗? HOT 1
- 最新的知乎应该怎么爬
- 如果知道动态网页的加载API并且也可以请求到json的数据,怎么能通过接口嵌入到框架里进一步抓取 HOT 1
- 高版本PHP已废弃这种 $s0{0} 写法,请使用$s0[0] HOT 2
- 关于知乎用户数据的爬虫我确实想过一个用途
- 在用回调函数on_list_page去获得列表页数据时候,无法真正add_url HOT 2
- 用js渲染数据的页面可以抓去吗?类似vue作为前段的 HOT 1
- 关于attached_url的bug HOT 5
- tp5 默认会写入一下报错到日志里面 HOT 2
- 关于分页采集 怎么搞都不对 HOT 2
- 能不能内容也 先点击一个动作,余下全文,然后再开始采集?
- 修复7.4.16版本报错bug--修复打个tag
- PHP8运行官网demo报错 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from phpspider.