Data

Tools

Indexes

Applications

Experiments

Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / scrapy-plugins/scrapy-deltafetch issues and pull requests

#52 - Modernize

Pull Request - State: closed - Opened by Gallaecio 5 months ago - 1 comment

#51 - Resolve request_fingerprint deprecation removal in Scrapy 2.12.0

Pull Request - State: closed - Opened by fsmeraldi 5 months ago - 4 comments

#48 - reset flag can now also be str of spider name in addition to bool

Pull Request - State: open - Opened by isaacimholt almost 4 years ago

#47 - hacktoberfest 2021

Issue - State: closed - Opened by isaacimholt almost 4 years ago - 1 comment

#46 - [github workflows] add publish action

Pull Request - State: closed - Opened by pawelmhm almost 4 years ago

#45 - Prepare release

Pull Request - State: closed - Opened by pawelmhm almost 4 years ago

#44 - Deltafetch ignore complikator

Pull Request - State: closed - Opened by pawelmhm almost 4 years ago

#43 - Does this plugin support redirected url?

Issue - State: open - Opened by mtabbasi almost 4 years ago - 1 comment

#42 - Allow user to ignore some requests

Pull Request - State: closed - Opened by complikator about 4 years ago - 2 comments

#41 - Allow user to ignore some requests

Pull Request - State: closed - Opened by complikator about 4 years ago

#40 - [core change!] replace bsddb3 with default Python dbm

Pull Request - State: closed - Opened by pawelmhm about 4 years ago - 1 comment

#39 - replace BaseItem to Item (BaseItem deprecated in Scrapy 2.3)

Pull Request - State: closed - Opened by pawelmhm about 4 years ago

#38 - switch from travis to github actions

Pull Request - State: closed - Opened by pawelmhm over 4 years ago - 3 comments

#37 - replace bsddb3 with other library

Issue - State: closed - Opened by pawelmhm over 4 years ago - 1 comment

#36 - Make code upto date with latest version of scrapy

Pull Request - State: open - Opened by rabinadk1 almost 5 years ago - 1 comment

#35 - Update README.rst

Pull Request - State: closed - Opened by mflpopescu over 5 years ago - 2 comments

#34 - Filter limited by "DOWNLOAD_DELAY"?

Issue - State: open - Opened by DogeVenci over 6 years ago

#33 - Installation failure on macOS

Issue - State: closed - Opened by adamporad about 7 years ago - 6 comments

#32 - Available On Windows?

Issue - State: open - Opened by mosynaq about 7 years ago - 1 comment

#31 - delta-fetch not scraping new items

Issue - State: open - Opened by clpatterson about 7 years ago

#30 - Facing issue regarding deltafetch

Issue - State: open - Opened by gopal1414 about 7 years ago

#29 - How to stop if encountered visited links?

Issue - State: open - Opened by mosynaq over 7 years ago - 1 comment

#28 - Is this support the mutiple spiders in the same project?

Issue - State: open - Opened by ileadall42 over 7 years ago

#27 - Mac run error python crash

Issue - State: open - Opened by yuzhewo over 7 years ago

#26 - add meta key named "request_need_deltafetch" .

Pull Request - State: open - Opened by Shazoo over 7 years ago - 1 comment

#25 - DBRunRecoveryErro wrong.

Issue - State: open - Opened by Big2Cat over 7 years ago - 3 comments

#24 - Fixed DBError on close, converted to use Pipenv and few other updates

Pull Request - State: closed - Opened by ouahibelhanchi almost 8 years ago - 1 comment

#23 - Trouble installing on my mac due to berkeley-db dependency.

Issue - State: closed - Opened by rodricios almost 8 years ago - 2 comments

#22 - How to use Berkeley DB in a distributed crawler?

Issue - State: closed - Opened by dataknower almost 8 years ago - 1 comment

#21 - Use/store redirect url fingerprint

Pull Request - State: closed - Opened by KrumBoychev about 8 years ago - 2 comments

#20 - This plugin not working in my project somehow

Issue - State: closed - Opened by Big2Cat about 8 years ago - 2 comments

#19 - Added: A DeltaFetchPseudoItem for storing requests with no items yielded

Pull Request - State: open - Opened by starrify about 8 years ago - 2 comments

#18 - DB gets corrupted very often

Issue - State: open - Opened by rmax about 8 years ago

#17 - URLs in start_urls are not affected by this middleware

Issue - State: open - Opened by gfrmin over 8 years ago - 4 comments

#16 - Update required Scrapy version

Pull Request - State: closed - Opened by chekunkov over 8 years ago - 2 comments

#15 - Issue while installing scrapy-deltafetch

Issue - State: closed - Opened by hendisantika over 8 years ago - 3 comments

#14 - Use "key in dict" idiom instead of .has_key(key)

Pull Request - State: closed - Opened by redapple over 8 years ago

#13 - Improvements needed

Issue - State: closed - Opened by ghost over 8 years ago - 5 comments

#12 - Added <deltafetch_passthrough> request metaparameter.

Pull Request - State: closed - Opened by uschmid over 8 years ago - 2 comments

#11 - Mock tests as if running inside a scrapy project

Pull Request - State: closed - Opened by redapple over 8 years ago - 1 comment

#10 - Add ability to force the execution of specific requests

Pull Request - State: closed - Opened by uschmid over 8 years ago

#9 - Fingerprint for initial request is not saved on redirects

Issue - State: open - Opened by mrueegg almost 9 years ago - 5 comments

#8 - Log through logging module instead of (deprecated) scrapy's spider.log() function

Pull Request - State: closed - Opened by redapple about 9 years ago - 1 comment

#7 - fix deltafetch_reset argument example

Pull Request - State: closed - Opened by stummjr about 9 years ago - 2 comments

#6 - Add test for re-creating .db file if corrupt

Pull Request - State: closed - Opened by redapple about 9 years ago

#5 - Support spider callbacks returning dict items

Pull Request - State: closed - Opened by redapple about 9 years ago - 1 comment

#4 - Document middleware design and behavior

Issue - State: open - Opened by redapple about 9 years ago - 2 comments

#3 - DOC Document that bsddb support has been removed

Issue - State: closed - Opened by redapple about 9 years ago

#2 - Add code coverage reports

Pull Request - State: closed - Opened by redapple about 9 years ago

#1 - Migrate scrapylib's DeltaFetch middleware

Pull Request - State: closed - Opened by redapple about 9 years ago