Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / ccao-data/data-architecture issues and pull requests

#100 - [Data catalog] Investigate failure of source freshness tests to run on schedule

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment
Labels: bug

#99 - Convert `aws-s3/scripts-ccao-data-warehouse-us-east-1` to dbt DAG components

Issue - State: open - Opened by dfsnow over 1 year ago
Labels: dbt

#98 - Add conditional uniqueness tests for iasWorld tables

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#95 - [Data catalog] Add tests for the reporting views

Issue - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#94 - Add deploy environment config to deploy_dbt_docs workflow

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#93 - 86 add uniqueness tests for iasworld tables

Pull Request - State: closed - Opened by wagnerlmichael over 1 year ago

#92 - Update `ratio_stats` glue job to use AssessPy package

Issue - State: closed - Opened by dfsnow over 1 year ago

#91 - Remove prefix underscore from slugified schema names

Pull Request - State: closed - Opened by dfsnow over 1 year ago - 1 comment

#90 - Move dbt cache to remote S3 storage

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#89 - Fix broken cache restoration

Issue - State: closed - Opened by jeancochrane over 1 year ago
Labels: bug

#88 - 70 create reporting view with most recent boundaries

Pull Request - State: closed - Opened by wrridgeway over 1 year ago

#87 - Add conditional uniqueness tests for iasWorld tables

Issue - State: closed - Opened by jeancochrane over 1 year ago

#86 - Add uniqueness tests for iasWorld tables

Issue - State: closed - Opened by jeancochrane over 1 year ago

#85 - Add deactivat and cur conditionals to additional tables

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#82 - Fix dbt production deploy problems

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#80 - Correct table prefix for run_id column

Pull Request - State: closed - Opened by wrridgeway over 1 year ago

#79 - Fix table misspecification in vw_ratio_stats

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#78 - Merge data catalog branch into master

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#77 - Replace all aggregate land calculations with pulls from agg land view

Pull Request - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#74 - Remove deactivated rows from views

Pull Request - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#73 - Remove deactivated rows from appeal view

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#72 - Add remaining views and tables to sql guide document

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#71 - Update dbt views to select from other dbt models where possible

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#70 - Create reporting view with most recent boundaries

Issue - State: closed - Opened by wrridgeway over 1 year ago

#68 - [Data catalog] dbt CI is not properly selecting for new and modified models

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment
Labels: bug

#67 - [Data catalog] Flesh out dev docs on how to build and test with dbt

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#66 - Fix land sf aggregation in res and condo chars views

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#65 - [Data catalog] Update views to point to dbt stub models

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#64 - Add stub dbt models for the remaining Athena views and tables

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 3 comments

#63 - Tweak failing dbt tests to make them easier to debug

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 3 comments

#62 - Investigate existing data integrity issues

Issue - State: open - Opened by jeancochrane over 1 year ago
Labels: dbt

#60 - [Data catalog] Document existing data integrity problems

Issue - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#59 - Rename dbt models to remove the _test suffix

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#58 - Fix cleanup-dbt-resources workflow to run on data-catalog branch

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#57 - Deploy dbt docs to GitHub Pages

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#56 - Deploy dbt docs to GitHub Pages

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#55 - [Data catalog] Merge `data-catalog` branch into `master`

Issue - State: closed - Opened by jeancochrane over 1 year ago

#54 - Add conditionals that make asmt_all unique by pin and year

Pull Request - State: closed - Opened by wrridgeway over 1 year ago - 3 comments

#52 - Add stage indicators and deduplicate conditionally rather than through aggregation

Pull Request - State: closed - Opened by wrridgeway over 1 year ago - 2 comments

#50 - Define GitHub actions workflows for building and testing dbt

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 9 comments

#49 - Add sqlfmt pre-commit hook for autoformatting dbt files

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#47 - Add card to partition for recent renovation

Pull Request - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#45 - Support incremental testing of dbt models through failure thresholds

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#44 - Adjust dbt profiles and schema macro to support prod/CI targets

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#43 - Setup S3 triggers for Glue crawlers

Issue - State: closed - Opened by dfsnow over 1 year ago - 1 comment
Labels: aws

#42 - Investigate replacing AWS Athena with DuckDB

Issue - State: closed - Opened by dfsnow over 1 year ago - 2 comments
Labels: aws

#41 - Symlink dbt models to Athena views to reduce duplication

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#40 - Remove data catalog R script

Pull Request - State: closed - Opened by dfsnow over 1 year ago

#39 - 20 decompose modelling views into reusable components

Pull Request - State: closed - Opened by wrridgeway over 1 year ago

#38 - Simplify, update, and de-duplicate condo characteristics view

Issue - State: closed - Opened by wrridgeway over 1 year ago - 3 comments

#37 - Fix superlinter on the master branch and ensure it runs on PRs

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#35 - [ WIP ] Add sqlfmt pre-commit hook for autoformatting dbt SQL files

Pull Request - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#33 - [Data catalog] Deploy the dbt-generated documentation as a static site

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#32 - [Data catalog] Add script to support incremental testing of dbt models

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#30 - [Data catalog] Improve Jinja templating indentation in dbt files

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#29 - [Data catalog] Rename dbt models to remove the `_test` suffix

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#28 - [Data catalog] Add production profile to the dbt configuration

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#27 - [Data catalog] Add remaining tests to DAG models

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#26 - [Data catalog] Document remaining tables and views

Issue - State: closed - Opened by jeancochrane over 1 year ago - 2 comments

#25 - [Data catalog] Add documentation for tables/views defined outside the DAG

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#24 - [Data catalog] Add stub models for tables/views defined outside the DAG

Issue - State: closed - Opened by jeancochrane over 1 year ago - 1 comment

#23 - Initialize dbt for data cataloguing and validation

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#22 - Add design doc for data catalog

Pull Request - State: closed - Opened by jeancochrane over 1 year ago

#21 - Gather 2023 existing data for 2024 modeling

Issue - State: closed - Opened by dfsnow over 1 year ago - 3 comments
Labels: new data/feature

#20 - Decompose modelling views into reusable components

Issue - State: closed - Opened by wrridgeway over 1 year ago

#19 - Add sorting columns to avoid Athena randomness

Pull Request - State: closed - Opened by wrridgeway over 1 year ago

#17 - Scoping a "data mart"

Issue - State: open - Opened by dfsnow over 1 year ago

#16 - Store data infrastructure configurations as code

Issue - State: closed - Opened by dfsnow over 1 year ago

#15 - Automate data integrity checks

Issue - State: closed - Opened by dfsnow over 1 year ago - 1 comment

#14 - Update internal views

Issue - State: closed - Opened by dfsnow over 1 year ago - 2 comments

#13 - Add check for 299 switching to vacant land PIN

Issue - State: open - Opened by dfsnow over 1 year ago
Labels: dbt

#12 - Add pre-commit styling and linting to R and Python scripts

Issue - State: closed - Opened by dfsnow over 1 year ago

#11 - 120 consolidate ic valuation model data

Pull Request - State: closed - Opened by wrridgeway over 1 year ago

#10 - Implement a data catalog system for existing CCAO data architecture

Issue - State: closed - Opened by wrridgeway over 1 year ago - 5 comments

#9 - Collect aggregate UAD statistics in S3

Issue - State: open - Opened by wrridgeway over 1 year ago
Labels: new data/feature

#8 - Update existing hydrology/distance to water feature

Issue - State: open - Opened by wrridgeway over 1 year ago
Labels: new data/feature

#7 - Add building touching / shared wall indicator

Issue - State: open - Opened by wrridgeway over 1 year ago
Labels: new data/feature

#6 - Combine MS building footprint data with LIDAR to estimate building sqft

Issue - State: open - Opened by wrridgeway over 1 year ago - 3 comments
Labels: new data/feature

#5 - Update data onboarding ticket

Issue - State: open - Opened by wrridgeway over 1 year ago
Labels: documentation

#4 - Ingest CSDS travel time matrices

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#3 - Monitor taxdist field in LEGDAT

Issue - State: open - Opened by wrridgeway over 1 year ago
Labels: dbt

#2 - Consolidate IC valuation model data

Issue - State: closed - Opened by wrridgeway over 1 year ago - 1 comment

#1 - Create corner lot indicator

Issue - State: closed - Opened by wrridgeway over 1 year ago - 6 comments
Labels: new data/feature