GithubHelp home page GithubHelp logo

Comments (4)

LuckyPigeon avatar LuckyPigeon commented on May 27, 2024

@maintain99
你好,這是 Scrapy 新版提出的改進。簡單來說,在 self.spider 裡面直接寫入你要呼叫的 spider 就行了,不需要傳參數進去,一來方便,二來安全。

from scrapy-redis.

maintain99 avatar maintain99 commented on May 27, 2024

@LuckyPigeon 我该怎么改能详细说一下吗,谢谢

from scrapy-redis.

LuckyPigeon avatar LuckyPigeon commented on May 27, 2024

@maintain99
我需要你的 code 才能詳細解說,但照理說你只要在 parse 內或任何你呼叫 scrapy 的函數內指定 self.spider = ExampleSpider 就行了。
但如果你對 spider 還不熟悉,建議你不要管 warning 了,還是能過的

from scrapy-redis.

maintain99 avatar maintain99 commented on May 27, 2024

@LuckyPigeon

有警告看着很不爽

`
import scrapy
from scrapy_redis.spiders import RedisSpider

class ShangZhiSpider(RedisSpider):
name= 'shangzhi'
allowed_domains = ['che168.com']
redis_key = 'chaosui_urls'

def parse(self, response,**kwargs):

    lis=response.xpath("//ul[@class='viewlist_ul']/li/a/@href").extract()
    for lj in lis:
        herf=response.urljoin(lj)
        yield scrapy.Request(
             url=herf,
             callback=self.jiexi
        )
        print(herf)

    lst=response.xpath('//div[@class="page fn-clear"]/a/@href').extract()[1:]
    for url in lst:
        urll=response.joinurl(url)
        yield scrapy.Request(
            url=urll,
            callback=self.parse
        )

def jiexi(self, resp):
    titer=resp.xpath('//h3[@class="car-brand-name"]/text()').extract()
    print(titer[0].strip())
    print(resp.url)

`

from scrapy-redis.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.