DOI-USGS/ds-pipelines-targets-example-wqp issues and pull requests

#116 - Add instructions for using this template outside of GitHub

Issue - State: open - Opened by lkoenig-usgs about 1 year ago - 2 comments

#115 - fix urls

Pull Request - State: closed - Opened by jpadilla-usgs over 1 year ago

#114 - Update code.json

Pull Request - State: closed - Opened by jpadilla-usgs over 1 year ago

#113 - Prepare for Upcoming DGEC Transition

Pull Request - State: closed - Opened by jesse-ross over 1 year ago - 2 comments

#112 - Simplify how duplicate records are handled in `clean_wqp_data`

Pull Request - State: closed - Opened by lekoenig over 1 year ago

#111 - Consider simplifying the harmonize phase

Issue - State: closed - Opened by lekoenig over 1 year ago
Labels: documentation, data cleaning

#110 - Use tags to document code changes

Pull Request - State: closed - Opened by lekoenig over 1 year ago

#109 - fix code.json

Pull Request - State: closed - Opened by padilla410 over 1 year ago - 3 comments

#108 - Describe use of intermediate data files

Pull Request - State: closed - Opened by lekoenig over 1 year ago

#107 - Prepare for Upcoming DGEC Transition

Pull Request - State: closed - Opened by jesse-ross over 1 year ago - 2 comments

#106 - Make this template a remote of user repos to get updates?

Issue - State: open - Opened by lindsayplatt over 1 year ago - 2 comments
Labels: enhancement

#105 - Print error message if inventory is empty

Pull Request - State: closed - Opened by lekoenig almost 2 years ago - 1 comment

#104 - Add informative error message if the inventory is empty

Issue - State: closed - Opened by lekoenig almost 2 years ago
Labels: documentation, data pull

#103 - Consider creating formal code releases

Issue - State: closed - Opened by lekoenig almost 2 years ago - 2 comments
Labels: documentation

#102 - Save "siteInfo" attributes in a new target

Pull Request - State: closed - Opened by lekoenig almost 2 years ago - 4 comments

#101 - Add changelog

Pull Request - State: closed - Opened by lekoenig almost 2 years ago - 2 comments

#100 - Add project changelog

Issue - State: closed - Opened by lekoenig almost 2 years ago
Labels: documentation

#99 - Document pattern for saving intermediate files containing the downloaded data

Issue - State: closed - Opened by lekoenig about 2 years ago
Labels: documentation

#98 - Change repo reference to GitLab after migration

Issue - State: open - Opened by padilla410 about 2 years ago - 3 comments
Labels: documentation

#97 - Make inventory step robust to empty queries

Pull Request - State: closed - Opened by lekoenig about 2 years ago

#96 - Empty queries are missing expected columns in WQP inventory

Issue - State: closed - Opened by lekoenig about 2 years ago - 1 comment
Labels: bug, data pull

#95 - Omit site/query attributes from the downloaded data?

Issue - State: closed - Opened by lekoenig about 2 years ago - 1 comment
Labels: data pull

#94 - retry downloading WQP data if timeout is reached

Pull Request - State: closed - Opened by lekoenig about 2 years ago - 12 comments

#93 - Should we make this repo a "template" repository?

Issue - State: closed - Opened by lekoenig over 2 years ago - 11 comments
Labels: question

#92 - Keep `results_count` specific to each char name

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 3 comments

#91 - Update regex used to find bad identifiers

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#90 - Convert missing value strings to NA

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#89 - Update regex when WQP is deployed to prod

Issue - State: closed - Opened by jordansread over 2 years ago
Labels: data pull

#88 - Consider keeping resultCount to be specific to the group of CharNames

Issue - State: closed - Opened by jordansread over 2 years ago - 6 comments
Labels: enhancement, data pull

#87 - Unexpected `p2_wqp_data_aoi` rebuilds due to attributes

Issue - State: closed - Opened by padilla410 over 2 years ago - 5 comments
Labels: help wanted, data pull

#86 - Simplify p3_wqp_param_cleaning_info

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#85 - Add a step to `format_columns()` that converts blanks to NA

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data cleaning

#84 - Check dataRetrieval package version

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#83 - Simplify `identify_bad_ids()`

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 2 comments

#82 - Add assert statement for dataRetrieval package version

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: data pull, pipeline

#81 - Order of duplicate rows varies across machines

Issue - State: closed - Opened by lekoenig over 2 years ago - 2 comments
Labels: bug, data cleaning

#80 - Log pipeline metadata

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 22 comments

#79 - Simplify `p3_wqp_param_cleaning_info`

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data cleaning

#78 - Delete example proxies files

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#77 - Force a retry when a query hangs?

Issue - State: closed - Opened by lindsayplatt over 2 years ago - 3 comments
Labels: enhancement, data pull

#76 - Simplify `identify_bad_ids()`

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#75 - Harmonize temperature and conductivity units

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 5 comments

#74 - Flag duplicated records

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#73 - Modify how we name download groups

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#72 - Change to how we name `download_grp`

Issue - State: closed - Opened by lindsayplatt over 2 years ago - 1 comment
Labels: data pull

#71 - Harmonize sets of characteristic names

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#70 - Flip wqp inventory and data pull comparisons back

Issue - State: closed - Opened by lindsayplatt over 2 years ago - 1 comment
Labels: bug, data summary

#69 - Convert select columns back to numeric

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#68 - Update documentation related to data pull

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 5 comments

#67 - Rename target phases

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#66 - Map over characteristic names when building the WQP download target

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#65 - Map over characteristic names when building the WQP download target

Issue - State: closed - Opened by lekoenig over 2 years ago - 2 comments
Labels: enhancement, data pull

#64 - Address 'greedy' rebuilds in `p1_wqp_data_aoi`

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#63 - Unexpected branches in `p1_wqp_data_aoi` rebuilding with change in AOI

Issue - State: closed - Opened by lekoenig over 2 years ago - 2 comments
Labels: bug

#62 - Add optional variable to force pipeline rebuild

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 3 comments

#61 - Rename target phases for this pipeline?

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: pipeline

#60 - Add handling for site identifiers containing "/"

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 2 comments

#59 - Add `last_forced_build` variable to force new builds of WQP inventory

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#58 - Chunk data pulls using grids, number of sites, and total records

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#57 - Map over characteristic names and grids when building `p1_wqp_inventory`

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#56 - Downloading data for site identifiers containing "/"

Issue - State: closed - Opened by lekoenig over 2 years ago - 2 comments
Labels: data pull

#55 - Retry WQP download upon error

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 4 comments

#54 - Use default buffer distance of 0

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#53 - Convert some columns back to numeric

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: data cleaning

#52 - Coerce all columns to character when building `p1_wqp_data_aoi`

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#51 - Expand and simplify big (CONUS) grid

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#50 - Map over the characteristic names when building the WQP inventory target

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: enhancement, data pull

#49 - Include resultCount in grouped sites

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#48 - Handling scaling when some sites have a lot of data

Issue - State: closed - Opened by jordansread over 2 years ago - 4 comments
Labels: data pull

#47 - Handle different column classes in merge of different `p1_wqp_data_aoi`'s

Issue - State: closed - Opened by jordansread over 2 years ago - 5 comments
Labels: data pull

#46 - Delete proxies example files

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: pipeline

#45 - Edit inventory status message in inventory_wqp function

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#44 - Include resultCount in p1_site_ids_grouped

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#43 - Set all buffer distances to zero as default

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#42 - Simplify p1_conus_grid by omitting CONUS and tigris dependencies

Issue - State: closed - Opened by lekoenig over 2 years ago - 6 comments
Labels: data pull

#40 - Explicitly load sf package when building WQP inventory target

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#39 - Error when re-building p1_wqp_inventory

Issue - State: closed - Opened by lekoenig over 2 years ago - 3 comments
Labels: bug, pipeline

#38 - Add log files to summarize data returned from WQP

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 4 comments

#37 - Move additional WQP arguments to _targets.R

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#36 - Consider chunking up data pulls by total records rather than sites

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: enhancement, data pull

#35 - Consider moving startDateLo and minresults to wqp_args list in _targets.R

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#34 - Add 1_fetch/out/.placeholder

Issue - State: closed - Opened by lindsayplatt over 2 years ago - 2 comments
Labels: pipeline

#33 - Pull WQP data based on inventory

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 1 comment

#32 - Omit sites outside of AOI from WQP inventory

Pull Request - State: closed - Opened by lekoenig over 2 years ago

#31 - Omit sites outside of AOI from WQP inventory

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#30 - Write log file containing similar characteristic names

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 2 comments

#29 - Check whether characteristic names are valid

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 2 comments

#28 - Download WQP data

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data pull

#27 - Harmonize reported units for a given characteristic name

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data cleaning

#26 - Check whether characteristic names are valid

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: data pull

#24 - Write log file that includes options for characteristicNames

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: enhancement

#23 - create log files to summarize data returned from WQP

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: enhancement, pipeline

#22 - Add documentation for data pull steps

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: documentation, data pull

#21 - Add wqp inventory

Pull Request - State: closed - Opened by lekoenig over 2 years ago - 9 comments

#20 - Which siteTypes should we query?

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: question, data pull

#19 - Should we use space or time to generate targets branches?

Issue - State: closed - Opened by lekoenig over 2 years ago - 4 comments
Labels: question, data pull

#18 - Should we use static or dynamic branching for querying the data?

Issue - State: closed - Opened by lekoenig over 2 years ago - 1 comment
Labels: question, data pull

#17 - Normalize characteristicNames

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data cleaning

#16 - Check for duplicate values

Issue - State: closed - Opened by lekoenig over 2 years ago
Labels: data cleaning

#12 - log pipeline metadata

Issue - State: closed - Opened by lekoenig almost 3 years ago
Labels: enhancement, data summary, pipeline

GitHub / DOI-USGS/ds-pipelines-targets-example-wqp issues and pull requests