Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / openclimatefix/dagster-dags issues and pull requests

#84 - feat(meteomatics): Add basic ws archival pipeline

Pull Request - State: closed - Opened by devsjc 9 months ago

#83 - Migrate all cron jobs to dagster on Leonardo

Issue - State: open - Opened by devsjc 9 months ago

#83 - Migrate all cron jobs to dagster on Leonardo

Issue - State: open - Opened by devsjc 9 months ago

#81 - Add diagram showing data flows

Issue - State: open - Opened by peterdudfield 9 months ago

#81 - Add diagram showing data flows

Issue - State: open - Opened by peterdudfield 9 months ago

#80 - Remove regex validation on consumer tag

Pull Request - State: closed - Opened by devsjc 10 months ago

#79 - Update consumer to better handle missing MARS fields

Pull Request - State: closed - Opened by devsjc 10 months ago

#78 - Increase timeout waiting for larger pods to finish pending

Pull Request - State: closed - Opened by devsjc 10 months ago

#77 - Increase wait for kbatch, capture 503 errors

Pull Request - State: closed - Opened by devsjc 10 months ago

#76 - Request larger pods to run icon jobs

Pull Request - State: closed - Opened by devsjc 10 months ago

#72 - 2023 PV data on HF

Issue - State: closed - Opened by peterdudfield 10 months ago - 1 comment

#72 - 2023 PV data on HF

Issue - State: closed - Opened by peterdudfield 10 months ago - 1 comment

#71 - ICON EU Dagster

Issue - State: closed - Opened by peterdudfield 10 months ago - 3 comments
Labels: timeline

#71 - ICON EU Dagster

Issue - State: closed - Opened by peterdudfield 10 months ago - 3 comments
Labels: timeline

#70 - Handle ConnectTimeout exception

Pull Request - State: closed - Opened by devsjc 10 months ago

#69 - Catch readtimeout error listing pods

Pull Request - State: closed - Opened by devsjc 10 months ago

#69 - Catch readtimeout error listing pods

Pull Request - State: closed - Opened by devsjc 10 months ago

#68 - Resolve ReadTimeout error in kbatch jobs

Issue - State: closed - Opened by devsjc 10 months ago

#66 - Prevent temporary name resolution error crashing kbatch jobs

Pull Request - State: closed - Opened by devsjc 10 months ago

#66 - Prevent temporary name resolution error crashing kbatch jobs

Pull Request - State: closed - Opened by devsjc 10 months ago

#65 - Prevent icon backfills from hitting commit limits in huggingface

Pull Request - State: closed - Opened by devsjc 10 months ago

#65 - Prevent icon backfills from hitting commit limits in huggingface

Pull Request - State: closed - Opened by devsjc 10 months ago

#64 - Use httpx's exception in guard, clean up raw dir

Pull Request - State: closed - Opened by devsjc 10 months ago

#64 - Use httpx's exception in guard, clean up raw dir

Pull Request - State: closed - Opened by devsjc 10 months ago

#63 - Handle the RemoteProcedureError from h11

Pull Request - State: closed - Opened by devsjc 10 months ago

#63 - Handle the RemoteProcedureError from h11

Pull Request - State: closed - Opened by devsjc 10 months ago

#62 - Exit quicker on kbatch pods which fail

Issue - State: closed - Opened by devsjc 10 months ago

#62 - Exit quicker on kbatch pods which fail

Issue - State: closed - Opened by devsjc 10 months ago

#61 - Exit quicker on pod failure with kbatch

Pull Request - State: closed - Opened by devsjc 10 months ago

#61 - Exit quicker on pod failure with kbatch

Pull Request - State: closed - Opened by devsjc 10 months ago

#60 - Guard against Partial Read Errors in Dagster

Issue - State: closed - Opened by devsjc 10 months ago

#60 - Guard against Partial Read Errors in Dagster

Issue - State: closed - Opened by devsjc 10 months ago

#59 - Guard against partial read errors when streaming logs from kbatch

Pull Request - State: closed - Opened by devsjc 10 months ago

#59 - Guard against partial read errors when streaming logs from kbatch

Pull Request - State: closed - Opened by devsjc 10 months ago

#58 - Fix the huggingface op output into the icon graph

Pull Request - State: closed - Opened by devsjc 10 months ago

#58 - Fix the huggingface op output into the icon graph

Pull Request - State: closed - Opened by devsjc 10 months ago

#57 - Add kbatch

Pull Request - State: closed - Opened by devsjc 10 months ago

#56 - ECMWF UK 2017 - 2019 (Current steps and params)

Issue - State: closed - Opened by peterdudfield 10 months ago
Labels: timeline

#56 - ECMWF UK 2017 - 2019 (Current steps and params)

Issue - State: closed - Opened by peterdudfield 10 months ago
Labels: timeline

#55 - Investigate and Fix erroring auto materialisations in dagster

Issue - State: closed - Opened by devsjc 10 months ago - 1 comment

#54 - Migrate dagster backend to postgres, modify partition format

Issue - State: closed - Opened by devsjc 10 months ago - 1 comment
Labels: enhancement

#54 - Migrate to postgres, modify partitions

Issue - State: open - Opened by devsjc 10 months ago
Labels: enhancement

#53 - Enable auto-materialisation; remove source asset

Pull Request - State: closed - Opened by devsjc 10 months ago

#53 - Enable auto-materialisation; remove source asset

Pull Request - State: closed - Opened by devsjc 10 months ago

#52 - Satellite Planetary Dagster

Issue - State: open - Opened by peterdudfield 11 months ago - 2 comments

#51 - NWP dagster Planetary

Issue - State: closed - Opened by peterdudfield 11 months ago - 21 comments
Labels: timeline

#50 - Add running jobs in Planetary Computer

Issue - State: open - Opened by jacobbieker 11 months ago - 4 comments

#49 - Ideas for running dagster in the cloud

Issue - State: open - Opened by peterdudfield 11 months ago - 8 comments
Labels: discussion

#48 - More verbose logging in cams

Pull Request - State: closed - Opened by devsjc 11 months ago

#47 - Make ICON request and download in same step, reduce number of requests

Pull Request - State: closed - Opened by devsjc 11 months ago

#46 - Add schedules to run scan jobs on a daily cadence

Pull Request - State: closed - Opened by devsjc 11 months ago

#45 - Use subset of variables for cams global

Pull Request - State: closed - Opened by devsjc 11 months ago

#44 - Consolidate jobs to just scan raw and scan zarr for NWP

Pull Request - State: closed - Opened by devsjc 11 months ago

#43 - Use Software-defined Assets

Pull Request - State: closed - Opened by devsjc 12 months ago

#42 - Add Storing IODC, RSS, 0 Degree EUMETSAT data in GCP

Issue - State: open - Opened by jacobbieker 12 months ago
Labels: enhancement

#41 - Add Arpege Global Model processing

Issue - State: open - Opened by jacobbieker 12 months ago
Labels: enhancement

#40 - Update Satellite jobs to use partitions

Pull Request - State: closed - Opened by jacobbieker 12 months ago

#39 - Add job timeouts

Pull Request - State: closed - Opened by jacobbieker 12 months ago

#38 - Icon All Parameter Backup

Issue - State: closed - Opened by devsjc 12 months ago - 2 comments
Labels: enhancement

#37 - Limit number of variables for CAMS backfill

Pull Request - State: closed - Opened by jacobbieker 12 months ago

#36 - Allow setting CAMS variables programmatically

Pull Request - State: closed - Opened by jacobbieker about 1 year ago - 1 comment

#35 - Check for nonetype

Pull Request - State: closed - Opened by devsjc about 1 year ago

#34 - Rename jobs

Pull Request - State: closed - Opened by devsjc about 1 year ago

#33 - Add CEDA download job, backdate partitions to 2017

Pull Request - State: closed - Opened by devsjc about 1 year ago - 1 comment

#32 - Backdate nwp jobs to 2017

Pull Request - State: closed - Opened by devsjc about 1 year ago

#31 - Move start date of ECMWF UK Dag to 2017

Issue - State: closed - Opened by devsjc about 1 year ago - 1 comment
Labels: enhancement

#30 - Add Dag for downloading CEDA data on a rolling basis

Issue - State: closed - Opened by devsjc about 1 year ago - 1 comment
Labels: enhancement

#29 - Continue downloading data even if some are missing

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#28 - Add tags to jobs

Pull Request - State: closed - Opened by devsjc about 1 year ago

#27 - Add CAMS EU Downloading

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#26 - Split out CAMS variables

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#25 - Add CAMS job and op

Pull Request - State: closed - Opened by devsjc about 1 year ago

#24 - Enable step selection for india

Pull Request - State: closed - Opened by devsjc about 1 year ago

#23 - Download India data with an 84-hour time range

Issue - State: closed - Opened by devsjc about 1 year ago
Labels: enhancement

#22 - Passiv data

Issue - State: open - Opened by peterdudfield about 1 year ago

#21 - Update consumer args to add new 'env' option as False

Pull Request - State: closed - Opened by devsjc about 1 year ago

#20 - Update consumer version

Pull Request - State: closed - Opened by devsjc about 1 year ago

#19 - Update variable handling (stop using reserved keyword)

Pull Request - State: closed - Opened by devsjc about 1 year ago

#18 - Add malta job, fix context manager

Pull Request - State: closed - Opened by devsjc about 1 year ago

#17 - Add india ECMWF Job

Pull Request - State: closed - Opened by devsjc about 1 year ago

#16 - Enable backdating of ECMWF data to 2020

Pull Request - State: closed - Opened by devsjc about 1 year ago

#15 - Update calls to consumer for synchronous stdout capture

Pull Request - State: closed - Opened by devsjc about 1 year ago

#14 - Explicit path to nwp-consumer executable

Pull Request - State: closed - Opened by devsjc about 1 year ago

#13 - Change nwp-consumer job to call python package

Pull Request - State: closed - Opened by devsjc about 1 year ago

#12 - Fix partitioning for ECMWF

Pull Request - State: closed - Opened by devsjc about 1 year ago

#11 - Create partitioned, backfillable versions of ICON today runs

Pull Request - State: closed - Opened by devsjc about 1 year ago - 3 comments

#10 - Add partitioned schedule config to ECMWF job

Pull Request - State: closed - Opened by devsjc about 1 year ago

#9 - More fixes for IODC downloading

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#8 - Add ecmwf daily scheduled job

Pull Request - State: closed - Opened by devsjc about 1 year ago

#7 - Add test ecmwf job

Pull Request - State: closed - Opened by devsjc about 1 year ago

#6 - Add simple IODC downloader

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#5 - Fix path for ICON archiving

Pull Request - State: closed - Opened by jacobbieker about 1 year ago

#4 - Add simple ecmwf asset

Pull Request - State: closed - Opened by devsjc about 1 year ago

#3 - Update dependency versions

Pull Request - State: closed - Opened by devsjc about 1 year ago

#2 - Add IODC EUMETSAT Satellite processing

Issue - State: open - Opened by jacobbieker about 1 year ago
Labels: enhancement

#1 - Cleanup defaults and uneeded files

Pull Request - State: closed - Opened by devsjc about 1 year ago