Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / istresearch/scrapy-cluster issues and pull requests

#267 - Upgrade the project to python 3.10

Pull Request - State: open - Opened by borisjota about 1 year ago - 4 comments

#266 - Is this project no longer supported?

Issue - State: closed - Opened by flyingdev over 1 year ago - 2 comments

#265 - Update Dockerfile

Pull Request - State: open - Opened by shivaabed1994 over 2 years ago

#264 - fix missing package python-lxml

Pull Request - State: closed - Opened by rasoulgh917 over 2 years ago

#263 - lxml dependency is broken for python3.6 base image

Pull Request - State: open - Opened by rafaharo over 2 years ago

#262 - Updated Names and statement for localsettings.py

Pull Request - State: open - Opened by abhalawat over 2 years ago

#261 - Crawler image - Python dependency fix

Pull Request - State: closed - Opened by igortelheiro almost 3 years ago

#260 - ui exception No connection adapters were found

Issue - State: open - Opened by new-wxw about 3 years ago

#259 - Crawler unhandled exceptions not logged

Issue - State: closed - Opened by getorca over 3 years ago - 1 comment

#258 - scrapy-cluster don't scroll the entire pages

Issue - State: closed - Opened by RochdiBoudokhane over 3 years ago - 7 comments

#257 - Fix online dmoz test

Pull Request - State: closed - Opened by madisonb over 3 years ago

#256 - Update quickstart.rst

Pull Request - State: closed - Opened by oprog over 3 years ago

#255 - Stats keys flooding redis + Crawl resilience + retry strategy

Issue - State: closed - Opened by benjaminelkrieff over 3 years ago - 5 comments

#254 - Incremented fail stats

Issue - State: closed - Opened by mingxuan1 almost 4 years ago

#253 - Upgrading the ELK stack

Issue - State: open - Opened by 4OH4 almost 4 years ago - 2 comments

#252 - TypeError: can't pickle thread.lock objects

Issue - State: open - Opened by benjaminelkrieff almost 4 years ago - 2 comments

#251 - Jobs disappearing: is there a way to monitor crawls ?

Issue - State: closed - Opened by benjaminelkrieff almost 4 years ago - 4 comments

#250 - Scrape delay

Issue - State: closed - Opened by tluyben almost 4 years ago - 2 comments

#249 - docs: fix grammatical errors in docker.rst

Pull Request - State: closed - Opened by mslowikowski almost 4 years ago - 1 comment

#248 - Bump jinja2 from 2.10 to 2.10.1 in /rest

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#247 - Bump requests from 2.18.4 to 2.20.0 in /rest

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#246 - Bump werkzeug from 0.14.1 to 0.15.3 in /rest

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#245 - Bump requests from 2.18.4 to 2.20.0 in /crawler

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#244 - Bump requests from 2.18.4 to 2.20.0 in /kafka-monitor

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#243 - Bump twisted from 18.4.0 to 20.3.0 in /crawler

Pull Request - State: closed - Opened by dependabot[bot] almost 4 years ago - 1 comment
Labels: dependencies

#242 - Dependency Resolution Fixes

Pull Request - State: closed - Opened by madisonb almost 4 years ago

#241 - maxdepth can not large than 2

Issue - State: closed - Opened by anthony9981 almost 4 years ago - 5 comments

#240 - Bump cryptography from 1.8.1 to 3.2

Pull Request - State: closed - Opened by dependabot[bot] about 4 years ago - 1 comment
Labels: dependencies

#239 - Bump cryptography from 1.8.1 to 3.2 in /crawler

Pull Request - State: closed - Opened by dependabot[bot] about 4 years ago - 1 comment
Labels: dependencies

#238 - Update Releases, Docker compose Issues

Issue - State: closed - Opened by MrMoon about 4 years ago - 2 comments

#237 - Ansible deprecated sudo

Pull Request - State: closed - Opened by j3k00 over 4 years ago - 1 comment

#236 - Add missing ELK restart instructions to docker-compose file

Pull Request - State: closed - Opened by Shique over 4 years ago - 1 comment

#235 - Future of the project

Issue - State: open - Opened by demisx over 4 years ago - 9 comments

#234 - Fixed an error with MarkupSafe installation

Pull Request - State: closed - Opened by Shique over 4 years ago - 2 comments

#233 - Fixed an error with MarkupSafe installation

Pull Request - State: closed - Opened by Shique over 4 years ago

#231 - How to submit a POST request through crawler REST API

Issue - State: closed - Opened by YanzhongSu about 5 years ago - 2 comments
Labels: question

#230 - Python 3

Issue - State: open - Opened by deed02392 about 5 years ago - 6 comments

#229 - Why encoding byted body

Issue - State: closed - Opened by YanzhongSu about 5 years ago - 2 comments

#228 - Bump werkzeug from 0.12.1 to 0.15.3

Pull Request - State: closed - Opened by dependabot[bot] about 5 years ago - 1 comment
Labels: dependencies

#227 - Can't up cluster from master branch

Issue - State: closed - Opened by MasterSergius about 5 years ago - 1 comment

#226 - supervisord install error while vagrant up

Issue - State: closed - Opened by fatemeeeeeee about 5 years ago - 2 comments

#225 - A mistake in parameter deliver through two method!

Issue - State: closed - Opened by kevin-ZZZ over 5 years ago - 4 comments

#224 - online integration test error

Issue - State: closed - Opened by rawiafray over 5 years ago - 2 comments

#223 - Cannot turn off DEBUG level log

Issue - State: closed - Opened by YanzhongSu over 5 years ago - 2 comments

#222 - Coveralls fix

Pull Request - State: closed - Opened by madisonb over 5 years ago - 1 comment

#221 - `Demo.incoming` topic keeps piling up

Issue - State: closed - Opened by YanzhongSu over 5 years ago - 6 comments

#220 - Docker ERROR: Could not ping Zookeeper

Issue - State: closed - Opened by Shique over 5 years ago - 6 comments

#219 - how to process different type of item in Processor?

Issue - State: closed - Opened by ChengkaiYang2022 over 5 years ago - 3 comments

#218 - what's the purpose of item_copy in pipelines.py?

Issue - State: closed - Opened by ChengkaiYang2022 over 5 years ago - 2 comments

#217 - Plans to support latest version of Kafka, ELK & Scrapy?

Issue - State: closed - Opened by kumaresra over 5 years ago - 1 comment

#216 - add timeout to urlopen. For some reason urlopen may block for a while.

Pull Request - State: closed - Opened by mooosu over 5 years ago - 4 comments

#215 - Replace Kafka with another pub/sub service?

Issue - State: closed - Opened by jrmlhermitte over 5 years ago - 3 comments

#214 - Website did't response for a long time,how to solve the problem

Issue - State: closed - Opened by Johnson0016 over 5 years ago - 1 comment

#213 - cluster mode online test hangs

Issue - State: closed - Opened by danmsf almost 6 years ago - 3 comments

#212 - How does it work with CrawlSpider?

Issue - State: closed - Opened by kevin-ZZZ almost 6 years ago - 1 comment

#211 - Does it support python3?

Issue - State: closed - Opened by xingzhicn almost 6 years ago - 1 comment

#210 - Fill a single item from many requests / "can't pickle thread.lock objects"

Issue - State: closed - Opened by llermaly about 6 years ago - 3 comments

#209 - Adding support of shared cookies for requests having the same crawlid

Pull Request - State: open - Opened by MohamedMb about 6 years ago - 3 comments

#208 - Adding support of shared cookies for requests having the same crawlid

Pull Request - State: closed - Opened by MohamedMb about 6 years ago - 1 comment

#207 - adding connection timeouts for redis clients

Pull Request - State: closed - Opened by rsdunya about 6 years ago - 2 comments

#206 - Docker compose : kafka and crawler containers restarting forever

Issue - State: closed - Opened by llermaly about 6 years ago - 6 comments

#205 - Pass cookies through the distributed scheduler

Pull Request - State: closed - Opened by devspyrosv about 6 years ago - 1 comment

#204 - Rest - signal only works in main thread

Issue - State: closed - Opened by Shique about 6 years ago - 2 comments

#203 - Lost config from Zookeeper makes spider down

Issue - State: closed - Opened by jamesliu668 about 6 years ago - 3 comments

#202 - Add Kafka Request Size Param to Crawler Producer

Pull Request - State: closed - Opened by madisonb about 6 years ago - 1 comment

#201 - Removed method timer wrapper in non-main thread

Pull Request - State: closed - Opened by madisonb about 6 years ago - 1 comment

#200 - Improve global page per domain filter documentation.

Pull Request - State: closed - Opened by devspyrosv about 6 years ago - 1 comment

#199 - cryptography library update to 2.3

Pull Request - State: closed - Opened by damienkilgannon about 6 years ago

#198 - remove unused code.

Pull Request - State: closed - Opened by Anderson-Liu over 6 years ago - 1 comment

#197 - Rebuilding crawler container fails

Issue - State: closed - Opened by mrvnklm over 6 years ago - 1 comment

#196 - scrapy-cluster stops crawling of some websites after a while

Issue - State: closed - Opened by alotfitakami over 6 years ago - 2 comments

#195 - add redis password

Pull Request - State: closed - Opened by aafeher over 6 years ago - 3 comments

#194 - Missing KAFKA_PRODUCER_TOPIC in docker rest settings

Issue - State: closed - Opened by aafeher over 6 years ago - 1 comment

#193 - Crawl complete signal

Issue - State: closed - Opened by wysiwygism over 6 years ago - 5 comments

#192 - Docker vs Vagrant

Issue - State: closed - Opened by mshahriarinia over 6 years ago - 1 comment

#191 - SC-Threaded-counter:

Pull Request - State: closed - Opened by devspyrosv over 6 years ago - 1 comment

#190 - Adding support for issue #182

Pull Request - State: open - Opened by knirbhay over 6 years ago - 3 comments

#189 - Getting twisted.web._newclient.ResponseNeverReceived exception

Issue - State: closed - Opened by spidysenses over 6 years ago - 2 comments

#188 - Add login in LinkSpider

Issue - State: closed - Opened by josselinlbe over 6 years ago - 2 comments

#187 - Help processing pages returned by crawler

Issue - State: closed - Opened by hellsingnorevy over 6 years ago - 2 comments

#185 - Initial package updates

Pull Request - State: closed - Opened by madisonb over 6 years ago - 1 comment

#184 - :pencil2: Typo on the document of kafka-monitor API section

Pull Request - State: closed - Opened by chihkaiyu over 6 years ago - 1 comment

#183 - Typo on document of Kafka-monitor API section

Issue - State: closed - Opened by chihkaiyu over 6 years ago - 2 comments

#182 - Support for custom header and cookies for the initial request from kafka_monitor.py feed

Issue - State: open - Opened by knirbhay over 6 years ago - 5 comments
Labels: crawler, feature request

#179 - Start supervisord in systemd debian systems

Pull Request - State: closed - Opened by devspyrosv over 6 years ago - 2 comments
Labels: ready to merge

#176 - No output when dumping incoming or outbound_firehose

Issue - State: closed - Opened by cliff-km over 6 years ago - 14 comments

#165 - Add crawler redis pagecount filter

Pull Request - State: closed - Opened by devspyrosv almost 7 years ago - 7 comments

#160 - Update Crawler, Kafka-Monitor, Redis-Monitor requirements

Pull Request - State: closed - Opened by devspyrosv almost 7 years ago - 8 comments

#149 - duplicate logging confusing

Issue - State: closed - Opened by softwarevamp about 7 years ago - 3 comments

#140 - exposing kafka from docker

Issue - State: closed - Opened by KiaraGrouwstra about 7 years ago - 5 comments

#135 - How to deploy on amazon ECS?

Issue - State: closed - Opened by hustshawn about 7 years ago - 4 comments
Labels: help wanted, question

#131 - log failure exception

Pull Request - State: closed - Opened by softwarevamp over 7 years ago - 3 comments

#127 - Travis build sometimes fails

Issue - State: closed - Opened by gas1121 over 7 years ago - 4 comments
Labels: bug, kafka-monitor, unit testing

#114 - Does redis-monitor can connect redis with password?

Issue - State: closed - Opened by movingheart over 7 years ago - 2 comments
Labels: feature request

#108 - Scrapy-Cluster doesn't respect the Request type

Issue - State: closed - Opened by wenxzhen over 7 years ago - 5 comments
Labels: crawler, feature request

#107 - Limit number of crawled pages per domain

Pull Request - State: closed - Opened by kazuar over 7 years ago - 1 comment