Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / bm-sms/daimon_skycrawlers issues and pull requests

#121 - docker: ruby-2.4.1-alpine

Pull Request - State: closed - Opened by tmitz about 7 years ago - 1 comment

#120 - travis: update ruby 2.4.1, 2.3.4, 2.2.6

Pull Request - State: closed - Opened by tmitz about 7 years ago

#119 - Organize storage API to read data

Pull Request - State: closed - Opened by okkez over 7 years ago

#118 - Add test for spider

Pull Request - State: closed - Opened by okkez over 7 years ago

#117 - Add Enumerable check

Pull Request - State: closed - Opened by tmitz over 7 years ago - 2 comments

#116 - Change to use extract_link in retrieve_links

Pull Request - State: closed - Opened by tmitz over 7 years ago - 1 comment

#115 - Fix undefined method

Pull Request - State: closed - Opened by tmitz over 7 years ago - 1 comment

#114 - Improve test stability

Pull Request - State: closed - Opened by okkez over 7 years ago

#113 - Fix callback method names

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#112 - Fix log message (print which crawler/processor to skip)

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#111 - Fix typo

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#110 - Fix message (verb conjugation)

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#109 - Extract crawler/processor

Pull Request - State: closed - Opened by okkez over 7 years ago

#108 - Add RubyGems badge

Pull Request - State: closed - Opened by changeworld over 7 years ago - 1 comment

#107 - Pass the copy of message hash to be used among all crawlers

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#106 - Wait for services

Pull Request - State: closed - Opened by okkez over 7 years ago

#105 - Fix docker-compose up sequence

Pull Request - State: closed - Opened by okkez over 7 years ago

#104 - Add `before_process` to crawler

Pull Request - State: closed - Opened by okkez over 7 years ago

#102 - Support list/details pattern

Pull Request - State: closed - Opened by okkez over 7 years ago

#101 - Change filter API

Pull Request - State: closed - Opened by okkez over 7 years ago

#100 - Support list/details pattern

Issue - State: closed - Opened by okkez over 7 years ago

#99 - Load app/filters directory when exec command is invoked

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 2 comments

#98 - Create filter file and directory when new command is invoked

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#97 - Correct the spelling of the product name

Pull Request - State: closed - Opened by snoozer05 over 7 years ago - 1 comment

#96 - Support docker-compose

Pull Request - State: closed - Opened by okkez over 7 years ago

#95 - Fix block processor

Pull Request - State: closed - Opened by okkez over 7 years ago

#94 - Add template files for docker-compose

Issue - State: closed - Opened by okkez over 7 years ago

#93 - Disable shutdown_interval

Issue - State: closed - Opened by okkez over 7 years ago - 1 comment

#92 - Use ERB when load database.yml

Pull Request - State: closed - Opened by okkez over 7 years ago

#91 - Use index for RDB storage

Pull Request - State: closed - Opened by okkez over 7 years ago

#90 - Comparison between page.last_modified_at and headers["last-modified"]

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 3 comments

#89 - bundler is a development dependency

Pull Request - State: closed - Opened by tricknotes over 7 years ago

#88 - Update bundler for Travis CI

Pull Request - State: closed - Opened by tricknotes over 7 years ago

#87 - We are daimon developers

Pull Request - State: closed - Opened by tricknotes over 7 years ago

#86 - Fix typo

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#85 - Add crawler/processor sample for itp.ne.jp

Pull Request - State: closed - Opened by okkez over 7 years ago

#84 - Load filters in crawler base class

Pull Request - State: closed - Opened by yucao24hours over 7 years ago - 1 comment

#83 - Multiple database

Pull Request - State: closed - Opened by okkez over 7 years ago

#82 - Graceful shudown

Issue - State: closed - Opened by okkez over 7 years ago - 1 comment

#81 - Add SitemapParser

Pull Request - State: closed - Opened by okkez over 7 years ago

#80 - Use timestamps macro properly

Pull Request - State: closed - Opened by okkez over 7 years ago

#79 - timestamps カラムの型について

Issue - State: closed - Opened by yucao24hours over 7 years ago - 4 comments

#78 - processor: Add hook to skip processing before invoke call

Pull Request - State: closed - Opened by okkez over 7 years ago

#77 - filter: Add good name aliases

Pull Request - State: closed - Opened by okkez over 7 years ago

#76 - filter: Fix UpdateChecker

Pull Request - State: closed - Opened by okkez over 7 years ago

#74 - Apply template method pattern

Pull Request - State: closed - Opened by okkez over 7 years ago

#73 - Add generator for crawler/processor

Pull Request - State: closed - Opened by okkez over 7 years ago - 1 comment

#72 - robots.txt によって拒否されているか否かのチェックについて

Issue - State: closed - Opened by yucao24hours almost 8 years ago - 2 comments

#71 - Obey robots.txt

Pull Request - State: closed - Opened by okkez almost 8 years ago

#70 - DaimonSkycrawlers::Filter::UpdateChecker の名前について

Issue - State: closed - Opened by yucao24hours almost 8 years ago - 3 comments

#69 - Processor pipeline

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#68 - Add screenshot Processor

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#67 - Add simple sitemap support

Pull Request - State: closed - Opened by okkez almost 8 years ago - 1 comment

#66 - Add commands/{runner,enqueue}.rb

Pull Request - State: closed - Opened by okkez almost 8 years ago

#65 - Add generator to generate crawler/processor template

Issue - State: closed - Opened by okkez almost 8 years ago

#64 - Support robots.txt

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#63 - XML sitemap support

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#62 - Organize directory structure of generated crawler

Pull Request - State: closed - Opened by okkez almost 8 years ago - 1 comment

#60 - Add about empty queue

Pull Request - State: closed - Opened by okkez almost 8 years ago

#59 - Generate migration files using MigrationGenerator

Pull Request - State: closed - Opened by okkez almost 8 years ago

#58 - Fix markdown syntax

Pull Request - State: closed - Opened by okkez almost 8 years ago

#57 - Display ordered list number as described

Pull Request - State: closed - Opened by yucao24hours almost 8 years ago - 1 comment

#56 - どのような使われ方を想定しているのかわからなかった

Issue - State: closed - Opened by yucao24hours almost 8 years ago - 2 comments

#55 - new コマンドでできる migration ファイルで deprecation warning

Issue - State: closed - Opened by yucao24hours almost 8 years ago - 2 comments

#54 - Add YARD style document partially

Pull Request - State: closed - Opened by okkez almost 8 years ago

#53 - Add test support

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment
Labels: nice to have

#52 - Propagate configurations

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#51 - Configure logger and use logger

Issue - State: closed - Opened by okkez almost 8 years ago - 1 comment

#50 - Parser as processor

Pull Request - State: closed - Opened by okkez over 8 years ago

#49 - Add caution to README.md

Pull Request - State: closed - Opened by sunaot over 8 years ago - 1 comment

#48 - Support params

Issue - State: closed - Opened by myokoym over 8 years ago - 1 comment

#47 - Stop crawling same page

Pull Request - State: closed - Opened by myokoym over 8 years ago - 4 comments

#46 - Add usage and sample scripts for new project [close #30]

Pull Request - State: closed - Opened by mtsmfm over 8 years ago - 1 comment

#45 - Add $LOAD_PATH for local testing

Pull Request - State: closed - Opened by mtsmfm over 8 years ago - 8 comments

#44 - Fix Gem::LoadError for generated application

Pull Request - State: closed - Opened by myokoym over 8 years ago - 1 comment

#43 - Fix features

Pull Request - State: closed - Opened by tricknotes over 8 years ago - 4 comments

#42 - Handle cookie

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment

#41 - Set referer dynamicaly

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment

#40 - Add `generate migration`

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment
Labels: nice to have

#39 - Add default migration

Pull Request - State: closed - Opened by okkez over 8 years ago

#38 - Test crawlers for CP932, EUC-JP and etc.

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment
Labels: nice to have

#37 - Add database configuration

Pull Request - State: closed - Opened by okkez over 8 years ago

#36 - Add database related tasks

Pull Request - State: closed - Opened by okkez over 8 years ago

#35 - Created crawler/processor should be a command

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment

#30 - Write document

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment

#29 - How to finish crawlers/processors?

Issue - State: closed - Opened by okkez over 8 years ago - 1 comment

#27 - Extract database schema definition

Issue - State: closed - Opened by okkez over 8 years ago

#26 - Extract database configuration

Issue - State: closed - Opened by okkez over 8 years ago

#24 - Filter URLs by Content-Type

Issue - State: closed - Opened by okkez over 8 years ago - 4 comments
Labels: nice to have

#20 - Stop crawling same URL many times

Issue - State: closed - Opened by okkez over 8 years ago - 2 comments

#19 - Add generate command

Pull Request - State: closed - Opened by myokoym over 8 years ago - 2 comments

#17 - Add provisioning

Issue - State: closed - Opened by okkez over 8 years ago - 2 comments
Labels: nice to have

#8 - Show stats

Issue - State: closed - Opened by tricknotes over 8 years ago - 2 comments
Labels: nice to have

#7 - Better testing

Issue - State: closed - Opened by tricknotes over 8 years ago - 3 comments

#3 - Configuration for RabbitMQ

Issue - State: closed - Opened by tricknotes over 8 years ago - 2 comments

#2 - Example client implementation

Issue - State: closed - Opened by tricknotes over 8 years ago - 3 comments

#1 - CLI

Issue - State: closed - Opened by tricknotes over 8 years ago - 5 comments