adbar/courlan issues and pull requests

#103 - maintenance: simplify code

Pull Request - State: closed - Opened by adbar 26 days ago - 1 comment

#102 - prepare version 1.2.0

Pull Request - State: closed - Opened by adbar 26 days ago - 1 comment

#101 - maintenance: deprecate timelimit argument

Pull Request - State: closed - Opened by adbar about 2 months ago - 1 comment

#100 - network: use urllib only

Pull Request - State: open - Opened by adbar 2 months ago - 1 comment

#99 - prepare version 1.1.0

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#98 - Convert Readme file to markdown format

Issue - State: closed - Opened by adbar 2 months ago
Labels: maintenance

#97 - convert readme to markdown

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#96 - compatability: restore timelimit parameter

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#95 - simplify cleaning functions

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#94 - filters: simplify code

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#93 - utils: faster tldinfo

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#92 - locales: use babel only and add function cache

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#91 - urlstore: simplify download batch and add max_size arg

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#90 - urlstore: simplify code

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#89 - maintenance: replace langcodes by babel

Pull Request - State: closed - Opened by adbar 2 months ago - 1 comment

#88 - UrlStore: use bytes instead of str for url paths

Pull Request - State: closed - Opened by adbar 3 months ago - 1 comment

#87 - maintenance: update workflow setup

Pull Request - State: closed - Opened by adbar 5 months ago - 1 comment

#86 - build(deps): bump the github-actions group with 5 updates

Pull Request - State: closed - Opened by dependabot[bot] 5 months ago - 1 comment
Labels: dependencies, github_actions

#85 - UrlStore: keep track of last response code and adjust backoff strategy

Issue - State: open - Opened by adbar 5 months ago
Labels: enhancement

#84 - prepare version 1

Pull Request - State: closed - Opened by adbar 5 months ago - 1 comment

#83 - persist UrlStore to disk

Pull Request - State: closed - Opened by adbar 5 months ago - 1 comment

#82 - docs: fix image on Pypi

Pull Request - State: closed - Opened by adbar 5 months ago

#81 - change license

Pull Request - State: closed - Opened by adbar 5 months ago - 1 comment

#80 - Change license to Apache 2.0

Issue - State: closed - Opened by adbar 5 months ago - 1 comment
Labels: maintenance

#79 - simplify code

Pull Request - State: closed - Opened by adbar 6 months ago - 1 comment

#78 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] 6 months ago

#77 - fix in clean_url(): remove whitespace

Pull Request - State: closed - Opened by adbar 6 months ago - 1 comment

#76 - Add support for username in netloc?

Issue - State: open - Opened by adbar 6 months ago - 1 comment
Labels: question

#75 - Add `is_homepage()` heuristic

Issue - State: open - Opened by adbar 6 months ago
Labels: enhancement

#74 - Replace language and country codes in `langinfo.py` by `pycountry`

Issue - State: closed - Opened by adbar 6 months ago - 1 comment
Labels: maintenance

#73 - Deprecate Python 3.6 & 3.7

Issue - State: open - Opened by adbar 6 months ago
Labels: maintenance

#72 - Navigation: add heuristic based on site depth

Issue - State: open - Opened by adbar 6 months ago
Labels: enhancement

#71 - Persistance for `UrlStore` (file I/O)

Issue - State: closed - Opened by adbar 6 months ago
Labels: enhancement

#70 - Check if `langcodes` can be replaced by `babel`

Issue - State: closed - Opened by adbar 6 months ago
Labels: maintenance

#69 - Add a function to check robots.txt rules and page type

Issue - State: open - Opened by adbar 6 months ago
Labels: enhancement

#68 - Add functioning courlan image or link to Pypi readme file

Issue - State: closed - Opened by adbar 6 months ago
Labels: documentation

#67 - fix tests: httpbun.org → .com

Pull Request - State: closed - Opened by adbar 7 months ago - 1 comment

#66 - prepare v0.9.5

Pull Request - State: closed - Opened by adbar 7 months ago - 1 comment

#65 - strip common tracking parameters in queries and fragments

Pull Request - State: closed - Opened by adbar 7 months ago - 1 comment

#64 - hardening of domain filter

Pull Request - State: closed - Opened by adbar 7 months ago - 1 comment

#63 - add function is_valid_url()

Pull Request - State: closed - Opened by adbar 7 months ago - 1 comment

#62 - Provide function `is_valid_url()`

Issue - State: closed - Opened by adbar 8 months ago
Labels: enhancement

#61 - Define option to focus on given extension types

Issue - State: open - Opened by adbar 8 months ago
Labels: enhancement

#60 - fix: account for special chars in normalization

Pull Request - State: closed - Opened by adbar 8 months ago - 1 comment

#59 - add pyproject.toml setup file

Pull Request - State: open - Opened by adbar 8 months ago - 1 comment

#58 - URL normalization: include unicode path

Pull Request - State: closed - Opened by adbar 8 months ago - 1 comment

#57 - Offer IRI to URI conversion

Issue - State: closed - Opened by adbar 9 months ago
Labels: enhancement

#56 - maintenance: simplify regexes

Pull Request - State: closed - Opened by adbar 10 months ago - 1 comment

#55 - maintenance: simplify regexes

Pull Request - State: closed - Opened by adbar 10 months ago - 1 comment

#54 - CLI: remove --samplesize

Pull Request - State: closed - Opened by adbar 10 months ago - 1 comment

#53 - prepare v0.9.4

Pull Request - State: closed - Opened by adbar 10 months ago - 1 comment

#52 - experimental normalization: systematically remove trailing slashes

Pull Request - State: closed - Opened by adbar 12 months ago - 1 comment

#51 - compatibility: extract_links + is_exhausted_domain

Pull Request - State: closed - Opened by adbar 12 months ago - 1 comment

#50 - replace remaining urlparse calls by urlsplit (2x faster)

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#49 - CLI sampling: consume input in chunks

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#48 - add domain/host name filter

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#47 - More efficient CLI multiprocessing

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#46 - Supply full URL to extract_links() and fix handling of relative URLs

Pull Request - State: closed - Opened by feltcat about 1 year ago - 4 comments

#45 - process_response does not properly resolve urls

Issue - State: closed - Opened by Ristellise about 1 year ago - 13 comments
Labels: bug

#44 - UrlStore: add ternary state and discard function

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#43 - store: add total and review counts

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#42 - add URLs straight from HTML

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#41 - adjust unvisited urls, domains and done meta-info

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#40 - UrlStore: add method get_unvisited_domains()

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#39 - CLI: better parallel processing

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#38 - prepare release 0.9.3

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#37 - CLI: add partial support for parallel processing

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#36 - refine link extraction and scrubbing

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#35 - sampling: discard homepages

Pull Request - State: closed - Opened by adbar about 1 year ago

#34 - add meta function to clear known lru_cache instances

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#33 - use urlunsplit and simplify urlutils

Pull Request - State: closed - Opened by adbar about 1 year ago

#32 - more efficient normalization

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#31 - more efficient sampling

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#30 - review path filters

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#29 - prepare version 0.9.2

Pull Request - State: closed - Opened by adbar about 1 year ago

#28 - adapt to urllib3 v2.0+

Pull Request - State: closed - Opened by adbar about 1 year ago

#27 - build(deps): update urllib3 requirement from <2,>=1.26 to >=1.26,<3

Pull Request - State: closed - Opened by dependabot[bot] about 1 year ago - 2 comments
Labels: dependencies

#26 - maintenance: review code

Pull Request - State: closed - Opened by adbar about 1 year ago

#25 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] about 1 year ago

#24 - add archives to type filter

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#23 - prepare v0.9.1

Pull Request - State: closed - Opened by adbar about 1 year ago

#22 - Add reset method to UrlStore

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#21 - urlstore: optional compression of rules

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#20 - fix: only use signal if verbose (Sourcery refactored)

Pull Request - State: closed - Opened by sourcery-ai[bot] about 1 year ago

#19 - fix: only use signal if verbose (Sourcery refactored)

Pull Request - State: closed - Opened by sourcery-ai[bot] about 1 year ago

#18 - fix: only use signal if verbose

Pull Request - State: closed - Opened by adbar about 1 year ago - 1 comment

#17 - Make use of signal optional

Issue - State: closed - Opened by adbar about 1 year ago
Labels: bug

#16 - Courlan does not load `/page/` links

Issue - State: open - Opened by sbusso over 1 year ago - 3 comments
Labels: enhancement

#15 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] over 1 year ago - 2 comments

#14 - Domain/subdomain confusion in link extraction

Issue - State: closed - Opened by adbar over 1 year ago
Labels: bug

#13 - create codeql.yml

Pull Request - State: closed - Opened by adbar over 1 year ago

#12 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] almost 2 years ago - 1 comment

#11 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] almost 2 years ago - 1 comment

#10 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] about 2 years ago - 2 comments

#9 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] over 2 years ago - 1 comment

#8 - Sourcery refactored master branch

Pull Request - State: closed - Opened by sourcery-ai[bot] over 2 years ago - 2 comments

#7 - Investigate sampling issue

Issue - State: closed - Opened by adbar over 2 years ago

#6 - Test and fix URL sampling to support Python 3.11

Issue - State: closed - Opened by adbar over 2 years ago
Labels: enhancement

#5 - Remove tox test settings

Issue - State: closed - Opened by adbar over 2 years ago
Labels: enhancement

#4 - Drop support for Python 3.5

Issue - State: closed - Opened by adbar over 2 years ago
Labels: enhancement

GitHub / adbar/courlan issues and pull requests