f111fei / article_spider Goto Github PK
View Code? Open in Web Editor NEW微信公众号爬虫
微信公众号爬虫
进行到第三步时,http://127.0.0.1:9222/json 中有URL,但是点击开启新页面时,显示页面无法打开
这个现在失效了吧
options.uri is a required argument when fetching undefined (3 retries left)
node out/index.js
module.js:478
throw err;
^
Error: Cannot find module '/article_spider/out/index.js'
npm install后显示“npm WARN [email protected] No repository field.“。
然后,浏览器访问http://localhost:9222/json,显示拒绝。
谢谢~
找不到这个请求了 https://mp.weixin.qq.com/mp/profile_ext?action=getmsg
而且在浏览器打开时无法加载更多 请问是否是微信改了机制?
在爬取公众号 openmindworld 时发生错误:
正在解析文章数据:{"id":"1000000353","title":""} 2020-03-24T12:58:53.713Z - error: CRAWLER Error Error: options.uri is a required argument when fetching undefined (3 retries left) 2020-03-24T12:59:03.721Z - error: CRAWLER Error Error: options.uri is a required argument when fetching undefined (2 retries left) 2020-03-24T12:59:13.737Z - error: CRAWLER Error Error: options.uri is a required argument when fetching undefined (1 retries left) 2020-03-24T12:59:23.745Z - error: CRAWLER Error Error: options.uri is a required argument when fetching undefined (node:3147) UnhandledPromiseRejectionWarning: Error: Error: options.uri is a required argument at Step.<anonymous> (/home/jeetchan/article_spider/node_modules/step-pipe/out/index.js:31:27) at Generator.throw (<anonymous>) at rejected (/home/jeetchan/article_spider/node_modules/step-pipe/out/index.js:5:65) at <anonymous> at process._tickCallback (internal/process/next_tick.js:188:7) (node:3147) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:3147) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
使用第二种的方法时,我再打开http://localhost:9222/json,并不能打开,显示浏览器拒绝,请问是什么原因?
仅仅填了biz和cookie,输出的错误信息为:
(node:64548) UnhandledPromiseRejectionWarning: Error: Error: {"ret":-3,"errmsg":"no session","cookie_count":1}
at Step. (/Users/chenchao/article_spider/node_modules/step-pipe/out/index.js:31:27)
at Generator.throw ()
at rejected (/Users/chenchao/article_spider/node_modules/step-pipe/out/index.js:5:65)
at
at process._tickCallback (internal/process/next_tick.js:188:7)
(node:64548) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:64548) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
article_spider % tsc
src/wechat/parser.ts:207:29 - error TS2339: Property 'attribs' does not exist on type 'Element'.
Property 'attribs' does not exist on type 'TextElement'.
207 const src = img.attribs['data-src'] || img.attribs['data-backsrc'] || img.attribs['src'];
~~~~~~~
src/wechat/parser.ts:207:56 - error TS2339: Property 'attribs' does not exist on type 'Element'.
Property 'attribs' does not exist on type 'TextElement'.
207 const src = img.attribs['data-src'] || img.attribs['data-backsrc'] || img.attribs['src'];
~~~~~~~
src/wechat/parser.ts:207:87 - error TS2339: Property 'attribs' does not exist on type 'Element'.
Property 'attribs' does not exist on type 'TextElement'.
207 const src = img.attribs['data-src'] || img.attribs['data-backsrc'] || img.attribs['src'];
~~~~~~~
src/wechat/parser.ts:208:30 - error TS2339: Property 'attribs' does not exist on type 'Element'.
Property 'attribs' does not exist on type 'TextElement'.
208 const type = img.attribs['data-type'] || 'jpg';
~~~~~~~
src/wechat/parser.ts:213:38 - error TS2339: Property 'attribs' does not exist on type 'Element'.
Property 'attribs' does not exist on type 'TextElement'.
213 imagesMeta.push({ ...img.attribs });
~~~~~~~
src/wechat/parser.ts:229:28 - error TS2339: Property 'tagName' does not exist on type 'TextElement | TagElement | CommentElement'.
Property 'tagName' does not exist on type 'TextElement'.
229 img.parent.tagName === 'p' && (contentElement.find('p')[0] === img.parent)) {
~~~~~~~
typings/crawler.d.ts:9:16 - error TS2304: Cannot find name 'CheerioAPI'.
9 $: CheerioAPI;
~~~~~~~~~~
Found 7 errors.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.