GitHub / eklem/norch-crawlers issues and pull requests
#53 - Bump minimist from 1.2.5 to 1.2.6
Pull Request -
State: open - Opened by dependabot[bot] over 3 years ago
Labels: dependencies
#52 - Bump node-fetch from 2.6.1 to 3.1.1
Pull Request -
State: closed - Opened by dependabot[bot] over 3 years ago
Labels: dependencies
#51 - Bump nth-check from 2.0.0 to 2.0.1
Pull Request -
State: closed - Opened by dependabot[bot] almost 4 years ago
Labels: dependencies
#50 - Bump path-parse from 1.0.6 to 1.0.7
Pull Request -
State: closed - Opened by dependabot[bot] almost 4 years ago
Labels: dependencies
#49 - Bump glob-parent from 5.1.1 to 5.1.2
Pull Request -
State: closed - Opened by dependabot[bot] about 4 years ago
Labels: dependencies
#48 - Bump lodash from 4.17.20 to 4.17.21
Pull Request -
State: closed - Opened by dependabot[bot] about 4 years ago
Labels: dependencies
#47 - [Snyk] Upgrade fast-xml-parser from 3.18.0 to 3.19.0
Pull Request -
State: closed - Opened by snyk-bot over 4 years ago
#46 - [Snyk] Upgrade fast-xml-parser from 3.17.6 to 3.18.0
Pull Request -
State: closed - Opened by eklem over 4 years ago
#45 - [Snyk] Upgrade fs-extra from 9.0.1 to 9.1.0
Pull Request -
State: closed - Opened by eklem over 4 years ago
#44 - [Snyk] Upgrade fast-xml-parser from 3.17.5 to 3.17.6
Pull Request -
State: closed - Opened by snyk-bot over 4 years ago
#43 - [Snyk] Upgrade daq-proc from 5.0.0 to 5.0.2
Pull Request -
State: closed - Opened by snyk-bot over 4 years ago
#42 - [Snyk] Upgrade fast-xml-parser from 3.17.4 to 3.17.5
Pull Request -
State: closed - Opened by snyk-bot over 4 years ago
#41 - Bump ini from 1.3.5 to 1.3.8
Pull Request -
State: closed - Opened by dependabot[bot] over 4 years ago
Labels: dependencies
#40 - Crawl content-function returning a promise
Issue -
State: open - Opened by eklem almost 5 years ago
- 5 comments
Labels: enhancement
#39 - Possiblity to do a full re-crawl
Issue -
State: open - Opened by eklem almost 5 years ago
Labels: enhancement
#38 - Bump node-fetch from 2.6.0 to 2.6.1
Pull Request -
State: closed - Opened by dependabot[bot] almost 5 years ago
Labels: dependencies
#37 - [Snyk] Upgrade node-fetch from 2.5.0 to 2.6.0
Pull Request -
State: closed - Opened by snyk-bot almost 5 years ago
#36 - Make the crawler-function based on promises
Issue -
State: closed - Opened by eklem almost 5 years ago
- 4 comments
Labels: enhancement
#35 - Bump extend from 3.0.1 to 3.0.2
Pull Request -
State: closed - Opened by dependabot[bot] almost 5 years ago
Labels: dependencies
#34 - Make it a browser-based crawler
Issue -
State: closed - Opened by eklem almost 5 years ago
- 2 comments
#33 - Add minimum test scripts
Issue -
State: open - Opened by eklem almost 7 years ago
- 1 comment
Labels: enhancement
#32 - Update request
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: bug
#31 - Figure out headless browsing
Issue -
State: closed - Opened by eklem over 7 years ago
- 2 comments
Labels: research
#30 - Make document crawler truly streamy
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
Labels: enhancement
#29 - Make url crawler streamy
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#28 - Documentation on main use cases
Issue -
State: open - Opened by eklem over 7 years ago
Labels: Documentation
#27 - Function: PlayNice (time between crawls)
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#26 - Function: cheerio pattern to exclude conent
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
Labels: enhancement
#25 - Figure out all config elements
Issue -
State: open - Opened by eklem over 7 years ago
Labels: enhancement
#24 - Function: Delay [n] seconds to follow robots.txt
Issue -
State: open - Opened by eklem over 7 years ago
- 1 comment
Labels: enhancement
#23 - Function: Check if updated or new content
Issue -
State: open - Opened by eklem over 7 years ago
Labels: enhancement
#22 - Function: Define which domain(s) to crawl
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#21 - Function: Site-crawl - Add found URLs to crawl queue
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#20 - Function: crawl with headless browser
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
Labels: enhancement
#19 - Function: check if new content
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
#18 - Function: pagination iteration
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
#17 - Function: list crawling
Issue -
State: closed - Opened by eklem over 7 years ago
#16 - Function: customize header
Issue -
State: open - Opened by eklem over 7 years ago
#15 - Function: follow robots.txt
Issue -
State: closed - Opened by eklem over 7 years ago
- 2 comments
#14 - Re-crawl all / just crawl what's new
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#13 - Change alert
Issue -
State: closed - Opened by eklem over 7 years ago
- 1 comment
#12 - Crawl a wikipedia.org-site
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#11 - robots.txt
Issue -
State: closed - Opened by eklem over 7 years ago
Labels: enhancement
#10 - Make exported functions
Issue -
State: open - Opened by eklem almost 8 years ago
Labels: enhancement
#9 - Make a morph.io-version
Issue -
State: closed - Opened by eklem about 8 years ago
- 1 comment
Labels: enhancement
#8 - Test phantom.js for crawling
Issue -
State: closed - Opened by eklem about 8 years ago
- 1 comment
Labels: enhancement
#7 - crawl NRK.no/mat
Issue -
State: closed - Opened by eklem about 8 years ago
- 1 comment
#6 - crawl godt.no
Issue -
State: closed - Opened by eklem about 8 years ago
- 1 comment
#5 - Custom headers with request
Issue -
State: closed - Opened by eklem about 8 years ago
Labels: enhancement
#4 - Find site to crawl
Issue -
State: closed - Opened by eklem about 8 years ago
- 2 comments
Labels: question
#3 - Blog howto
Issue -
State: open - Opened by eklem about 8 years ago
Labels: demo
#2 - Create a crawler for items to be stored
Issue -
State: open - Opened by eklem about 8 years ago
- 1 comment
Labels: enhancement
#1 - Create crawler for list pages
Issue -
State: closed - Opened by eklem about 8 years ago
- 1 comment
Labels: enhancement