Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / howie6879/ruia issues and pull requests

#159 - Feat: process_start_urls in parallel

Pull Request - State: open - Opened by aircloud over 1 year ago - 2 comments

#158 - 请问如何判断发生了跳转呢？

Issue - State: open - Opened by ray0728 over 1 year ago - 1 comment

#157 - Logs

Issue - State: closed - Opened by dickermoshe over 1 year ago - 1 comment

#156 - 如果能支持分布式就好了

Issue - State: open - Opened by xmydjx almost 2 years ago

#155 - docs.python-ruia.org is not available

Issue - State: open - Opened by asluchevskiy almost 2 years ago - 4 comments

#154 - 通过中间件添加 socks5 代理后如何关闭 session？

Issue - State: closed - Opened by killbus about 2 years ago

#152 - Fixed: RegexField default

Pull Request - State: closed - Opened by 123seven over 2 years ago

#151 - ruia 使用lxml编码xml文档时报错

Issue - State: closed - Opened by showthesunli over 2 years ago - 1 comment

#150 - worker_numbers 数值多少合适

Issue - State: closed - Opened by atzouhua almost 3 years ago - 1 comment

#149 - 示例代码运行报错

Issue - State: closed - Opened by mogeko almost 3 years ago - 3 comments

#148 - 我应当如何向 Spider 传递 start_urls？

Issue - State: closed - Opened by mogeko about 3 years ago - 1 comment

#147 - spider 添加类型注释

Pull Request - State: closed - Opened by Vastxiao about 3 years ago

#146 - httpx替换aiohttp支持http2

Issue - State: open - Opened by Vastxiao about 3 years ago - 1 comment

#145 - refactor: remove redundant param "is_async_start" in Spider

Pull Request - State: closed - Opened by laggardkernel about 3 years ago

#144 - refactor: remove redundant param "_signal" in Spider.stop()

Pull Request - State: closed - Opened by laggardkernel about 3 years ago

#143 - fix: limit filter in _parse_html() to skip only field "target_item"

Pull Request - State: closed - Opened by laggardkernel about 3 years ago

#142 - POST发送请求，收不到请求中的body

Issue - State: closed - Opened by superniao666 over 3 years ago - 2 comments

#140 - 是否可以用模式匹配工具-pampy来实现对json解析的支持

Issue - State: closed - Opened by jiangfubang over 3 years ago - 1 comment
Labels: question

#139 - 代理使用问题

Issue - State: closed - Opened by atzouhua over 3 years ago - 1 comment

#138 - 运行示例代码报错

Issue - State: closed - Opened by qgyhd1234 over 3 years ago - 10 comments

#137 - 【suggestion】重试逻辑可以添加或更换代理ip

Issue - State: closed - Opened by michael-liumh over 3 years ago - 8 comments

#136 - python3.9 remove asyncio.Task.all_tasks()

Issue - State: closed - Opened by happyli0826 over 3 years ago - 3 comments

#135 - Trouble scraping deck.tk/deckstats.net

Issue - State: closed - Opened by Triquetra over 3 years ago - 7 comments
Labels: bug, enhancement

#134 - Would be nice to be able to pass in "start_urls"

Issue - State: closed - Opened by JacobJustice over 3 years ago - 7 comments

#133 - Improve Chinese documentation

Issue - State: open - Opened by howie6879 over 3 years ago
Labels: enhancement

#132 - Update SOCKS5 proxy example

Pull Request - State: closed - Opened by Leezj9671 over 3 years ago - 1 comment

#131 - Is it possible to use SOCKS5 proxy?

Issue - State: closed - Opened by Leezj9671 over 3 years ago - 9 comments

#130 - AttributeError: 'Response' object has no attribute 'html'

Issue - State: closed - Opened by xiaoniaoyouhuajiang over 3 years ago - 6 comments

#129 - Supported Python3.9

Issue - State: closed - Opened by howie6879 over 3 years ago

#128 - 请问怎么添加proxy和headers 以及post

Issue - State: closed - Opened by yangtengtx over 3 years ago - 5 comments

#127 - A Ruia plugin for building Distributed crawling/scraping

Issue - State: open - Opened by howie6879 over 3 years ago - 3 comments
Labels: enhancement, Plugin

#125 - 框架使用问题：图片下载场景，CPU 为何会跑满？网络 IO 却几乎为零？

Issue - State: closed - Opened by lonsty almost 4 years ago - 3 comments
Labels: bug, enhancement

#121 - Extension for batch POST requests with different payload data

Issue - State: closed - Opened by peiyaoli almost 4 years ago - 4 comments

#120 - XML parse issue

Issue - State: closed - Opened by peiyaoli almost 4 years ago - 3 comments

#119 - Add trackback to logger.error() for spider

Issue - State: closed - Opened by ts709 almost 4 years ago - 5 comments

#118 - Meta字段怎么获取content?

Issue - State: closed - Opened by jackkam85 about 4 years ago - 1 comment

#117 - ruia 怎么使用伪造ip或者使用ip池

Issue - State: closed - Opened by shuqian2017 about 4 years ago - 1 comment

#116 - 用pip安装bug

Issue - State: closed - Opened by heyaug about 4 years ago - 3 comments

#115 - 文档方便详细点吗?

Issue - State: closed - Opened by lyg4795 about 4 years ago - 5 comments

#114 - Memory Leak on big number of pages

Issue - State: closed - Opened by vkudyushev about 4 years ago - 7 comments

#113 - 请问可以用beautifulsoup来代替默认的解析器吗？

Issue - State: closed - Opened by synodriver over 4 years ago - 3 comments

#112 - Add support for parsing JSON on item definition?

Issue - State: closed - Opened by owen800q over 4 years ago - 2 comments

#111 - RecursionError: maximum recursion depth exceeded while calling a Python object

Issue - State: closed - Opened by Vastxiao over 4 years ago - 1 comment

#110 - fix Chinese character gash problem in RegexField.extract

Pull Request - State: closed - Opened by fengdongfa1995 over 4 years ago

#109 - RegexField.extract()转中文乱码

Issue - State: closed - Opened by fengdongfa1995 over 4 years ago - 3 comments

#108 - could ruia fetch images? 可以下载图片吗

Issue - State: closed - Opened by scil over 4 years ago - 1 comment

#107 - data cleaning with staticmethod

Issue - State: closed - Opened by alenucci over 4 years ago - 1 comment

#106 - 定义的Field类对json数据结构的抽取

Issue - State: closed - Opened by YellowDong over 4 years ago - 7 comments

#104 - Piping multiple scrapers

Issue - State: closed - Opened by lormayna over 4 years ago - 5 comments

#103 - Fix typo

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#102 - Add plugin documentation and examples

Pull Request - State: closed - Opened by abmyii over 4 years ago - 7 comments

#101 - Calling `self.start` as an instance method for a `Spider`

Issue - State: closed - Opened by abmyii over 4 years ago - 11 comments

#100 - Log crucial information regardless of log-level

Issue - State: closed - Opened by abmyii over 4 years ago - 13 comments

#99 - 如何将response中的metadata传递给item

Issue - State: closed - Opened by YISION over 4 years ago - 6 comments

#97 - Fix asyncio error

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#96 - Fix #94

Pull Request - State: closed - Opened by panhaoyu over 4 years ago - 1 comment

#95 - Characters not supported

Issue - State: closed - Opened by panhaoyu over 4 years ago - 1 comment

#94 - Documentation about IgnoreThisItem

Issue - State: closed - Opened by panhaoyu over 4 years ago - 3 comments

#93 - Default (when many=True) shouldn't be enclosed in list

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#92 - Default shoudn't be enclosed in list

Issue - State: closed - Opened by abmyii over 4 years ago - 6 comments

#91 - asyncio `RuntimeError`

Issue - State: closed - Opened by abmyii over 4 years ago - 11 comments

#90 - Show URL in Error for easier debugging

Issue - State: closed - Opened by abmyii over 4 years ago - 11 comments

#89 - Add ElementField

Pull Request - State: closed - Opened by abmyii over 4 years ago - 2 comments

#88 - No field for capturing raw LXML elements

Issue - State: closed - Opened by abmyii over 4 years ago - 5 comments

#87 - Fix indentation

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#86 - Fix and test for selectors with text() failing

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#85 - Remove .strip() from TextField parsing

Pull Request - State: closed - Opened by abmyii over 4 years ago - 1 comment

#84 - `TextField` strips strings which may not be desirable

Issue - State: closed - Opened by abmyii over 4 years ago - 9 comments

#83 - `text()` in xpath selector causes an error

Issue - State: closed - Opened by abmyii over 4 years ago - 7 comments

#82 - Don't delay retries

Pull Request - State: closed - Opened by abmyii over 4 years ago - 6 comments

#81 - Add RETRY_DELAY option

Pull Request - State: closed - Opened by abmyii over 4 years ago - 19 comments

#80 - `DELAY` attribute specifically for retries

Issue - State: closed - Opened by abmyii over 4 years ago - 13 comments

#79 - target_item is expected error

Issue - State: closed - Opened by r3v1 almost 5 years ago - 2 comments

#78 - Rate Limiting?

Issue - State: closed - Opened by FaizShah almost 5 years ago - 5 comments

#76 - question: Is there any option for continuous scraping

Issue - State: closed - Opened by iAnanich almost 5 years ago - 6 comments

#75 - process_item is only call when the callback_result is an Item

Issue - State: closed - Opened by hiancdtrsnm about 5 years ago - 1 comment

#74 - Update Documentation of use of spider.request method

Issue - State: closed - Opened by hiancdtrsnm about 5 years ago - 2 comments

#73 - spider.request is not awaitable

Issue - State: closed - Opened by hiancdtrsnm about 5 years ago - 1 comment

#72 - Pass kwargs to init and return Spider instance

Pull Request - State: closed - Opened by maxzheng about 5 years ago - 1 comment

#71 - Add re_flags to pass thru flags keyword to re.compile

Pull Request - State: closed - Opened by maxzheng about 5 years ago - 1 comment

#70 - 如何在爬取过程中增加新的目标url页面呢

Issue - State: closed - Opened by gxtrobot over 5 years ago - 4 comments

#69 - 多个 spider 同时开始，实现真的异步

Issue - State: closed - Opened by ctaoist over 5 years ago - 6 comments
Labels: bug, enhancement

#67 - 建议和疑惑

Issue - State: closed - Opened by Developer27149 over 5 years ago - 2 comments

#65 - correction "parst" to "parse"

Pull Request - State: closed - Opened by duolaAOA over 5 years ago - 1 comment

#56 - A Ruia plugin that uses the motor to store data

Issue - State: closed - Opened by howie6879 over 5 years ago
Labels: enhancement, Plugin

#32 - A Ruia plugin for debugging

Issue - State: closed - Opened by howie6879 over 5 years ago - 2 comments
Labels: enhancement, Plugin

#28 - write a middleware to filter repeat request

Issue - State: closed - Opened by panhaoyu over 5 years ago - 2 comments

#21 - Write a website for Ruia

Issue - State: closed - Opened by howie6879 over 5 years ago - 1 comment
Labels: enhancement

#17 - HtmlField?

Issue - State: closed - Opened by panhaoyu over 5 years ago - 6 comments
Labels: enhancement, Plugin