rmax/scrapy-redis issues and pull requests

#174 - Item pipelines are slowed down

Issue - State: closed - Opened by Rossil2012 over 4 years ago - 2 comments
Labels: question, improvement

#171 - Looking for co-maintainers!

Issue - State: open - Opened by rmax over 4 years ago - 9 comments
Labels: help wanted

#162 - RFPDupeFilter:: It seems this doesn't work

Issue - State: closed - Opened by forgeries almost 5 years ago - 3 comments
Labels: question, improvement

#153 - Scrapy-redis automatically helps create a xxx:request request queue in both libraries of the redis database

Issue - State: closed - Opened by hewm over 5 years ago
Labels: question

#152 - The distributed scrappy crawler idling after requesting for a while and still having a list of requests in redis

Issue - State: closed - Opened by hewm over 5 years ago
Labels: question

#151 - Doc is bad.

Issue - State: closed - Opened by BobDu over 5 years ago - 4 comments
Labels: improvement, help wanted, good first issue

#150 - where method `make_requests_from_url` was defined?

Issue - State: closed - Opened by Vickey-Wu over 5 years ago - 1 comment
Labels: question

#149 - scrapy wouldn't create new request after i pause crawl and resume crawl, and redis wouldn't create `spidername:requests` set again, even i set `SCHEDULER_PERSIST = True` at settings.py.

Issue - State: open - Opened by Vickey-Wu over 5 years ago
Labels: bug, question

#148 - My scrapy program cannot keep pulling data from redis queue

Issue - State: open - Opened by Cehae over 5 years ago - 1 comment
Labels: bug, question

#147 - How do I receive url from redis more quickly ?

Issue - State: open - Opened by CoderZHV over 5 years ago - 1 comment
Labels: question, improvement

#145 - 2019-04-03 12:25:25 [scrapy.extensions.logstats] INFO: Crawled 18852 pages (at 0 pages/min), scraped 1762 items (at 0 items/min)

Issue - State: open - Opened by python-D over 5 years ago - 2 comments
Labels: bug, question, improvement

#144 - Is possible to use bloom filter on scrapy-redis?

Issue - State: closed - Opened by zhangslob over 5 years ago - 4 comments
Labels: question, improvement

#140 - Added support for json data

Pull Request - State: closed - Opened by sahupankaj10 almost 6 years ago - 1 comment

#139 - Update pipeline document

Issue - State: closed - Opened by LuckyPigeon almost 6 years ago - 3 comments
Labels: documentation, help wanted, good first issue

#137 - Return duplicate item in the result

Issue - State: closed - Opened by LuckyPigeon almost 6 years ago

#136 - Read one entry from redis queue

Issue - State: open - Opened by t75bernd almost 6 years ago - 1 comment
Labels: feature

#135 - Something wrong in scheduler

Issue - State: closed - Opened by taozi926494 about 6 years ago - 4 comments
Labels: question

#122 - middleware.process_start_requests only process the first batch

Issue - State: closed - Opened by liangWenPeng over 6 years ago - 1 comment

#121 - Whether scrapy-redis can handle this situation - request urls have priority?

Issue - State: closed - Opened by sunnypandas over 6 years ago - 2 comments
Labels: question

#120 - can have a support to ignore some url with ignore information in request.meta

Issue - State: closed - Opened by jipengzhu over 6 years ago - 2 comments
Labels: question

#119 - Suggesting new way to schedule requests

Issue - State: open - Opened by botzill over 6 years ago - 4 comments
Labels: feature

#115 - How do I only receive url from redis?

Issue - State: closed - Opened by MrTurtleW almost 7 years ago - 2 comments

#113 - Why is scrapy_redis generated in redis that the requests queue is zset type instead of list type

Issue - State: closed - Opened by smartXiaoxie almost 7 years ago
Labels: question

#109 - scrapy-redis conflict with scrapyd

Issue - State: closed - Opened by linkkong over 7 years ago - 2 comments
Labels: question

#108 - Config to set the spider do not crash when the redis is unusble

Issue - State: open - Opened by kingname over 7 years ago - 1 comment
Labels: question, improvement

#104 - Pipeline sending data after spider_closed and opened again.

Issue - State: closed - Opened by ghost over 7 years ago

#100 - Crawl site that needs login

Issue - State: closed - Opened by BoeingX over 7 years ago
Labels: question

#98 - about from_crawler 's first argument

Issue - State: closed - Opened by champbay over 7 years ago
Labels: improvement

#97 - Recommended redis configuration?

Issue - State: closed - Opened by StaymanHou over 7 years ago - 4 comments
Labels: question, documentation

#95 - dmoz.org is not available

Issue - State: closed - Opened by hyanx over 7 years ago - 2 comments
Labels: improvement, documentation

#94 - Convert to new logging convention: spider.log -> spider.logger

Pull Request - State: closed - Opened by chishaku over 7 years ago - 8 comments

#93 - better integration with scrapy shell

Issue - State: closed - Opened by n1kt0 over 7 years ago
Labels: question, improvement

#74 - 12x times faster next_request()/spider_idle() logic

Issue - State: open - Opened by 77cc33 about 8 years ago - 7 comments
Labels: feature

#67 - How to share cookie between different spiders when using scrapy-redis

Issue - State: closed - Opened by awkejiang over 8 years ago - 1 comment
Labels: question

#59 - TypeError: init() got an unexpected keyword argument 'server'

Issue - State: closed - Opened by zwl1619 over 8 years ago - 7 comments
Labels: bug

#56 - Add example integration with pyrebloom

Issue - State: open - Opened by rmax over 8 years ago - 9 comments
Labels: feature

#54 - Ensure serialized requests can be deserialized.

Issue - State: open - Opened by rmax over 8 years ago - 2 comments
Labels: feature

#53 - Add support for python 3.5 (>=3.4?)

Issue - State: closed - Opened by rmax over 8 years ago - 3 comments
Labels: feature

#46 - Add a redis backed StatsCollector

Issue - State: closed - Opened by jc0n over 8 years ago - 3 comments
Labels: feature

GitHub / rmax/scrapy-redis issues and pull requests