woxcab / scrapy_rss Goto Github PK

If item is described as

import scrapy
import scrapy_rss
class MyItem(scrapy_rss.RssedItem):
    field = scrapy.Field()

then this statement

item = MyItem({'field': 'value'})

produces

TypeError: init() takes 1 positional argument but 2 were given

Hi.

I just wanted to try out this package for a personal project, tried to install it with poetry (Python 3.8.2, poetry 1.0.3, pip 20.1), and go all kinds of dependency conflicts. It wanted to downgrade scrapy to version 1.4, use some old cryptography version, ...

I did some digging. If you extract the wheel package from Pypi, and look at the file metadata.json, you can see the requirements.

"run_requires": [{"requires": ["cryptography (<2.0)",
                               "pyOpenSSL (<17.3.0)",
                               "python-dateutil",
                               "scrapy (<1.5.0)",
                               "six"]}]

My guess is, that you created the wheel with Python 3.3. The trick that you did in the setup.py file doesn't work, if you package it into a wheel :

install_requires=['python-dateutil',
                  'scrapy<1.5.0' if sys.version_info[:2] == (3, 3)
                  else 'scrapy>=1.1,<1.8.0' if sys.version_info[:2] == (3, 4)
                  else 'scrapy>=1.1' if sys.version_info[:2] == (3, 5)
                  else 'scrapy>=1.3.1',
                  'six']

Not sure if it is possible to create on wheel that chooses different dependencies depending on the Python version. Maybe you need to create multiple wheels, one for each version.

As a fix for me, I'll just install the package directly via git.

I use setting with working credentials
FEED_URI = 'ftp://userName:[email protected]/path/to/file.xml'

But scrapy crawl spider make error with message
ftplib.error_perm: 500 'STOR' not understood

feedUri is working, i test it in browser
spider is working, i test it locally

All i want now to export RSS to FTP. How it can be done?

Hello @woxcab,

Thanks for this package, very cool!

I was wondering, is there a way to produce items with custom namespaces? Like:

<item>
  ...
  <store:price>$42.42</store:price>
  <store:name>$42.42</store:name>
  ...
</item>

And the like? If no, I would be willing to create a PR to support this, can you share some insight on how would you go about implementing such a feature?

Thanks!
Reinier