teamclairvoyant/airflow-maintenance-dags issues and pull requests

#146 - Improve log cleanup script and enhance error handling

Pull Request - State: open - Opened by mnott about 1 month ago

#145 - Update README.md

Pull Request - State: open - Opened by rodigu about 1 month ago

#144 - log_cleanup dag fails

Issue - State: open - Opened by e-compagno 6 months ago

#143 - It's broken from Airflow 2.7

Issue - State: open - Opened by chriske 8 months ago

#142 - exit early if a user opted to skip deleting the DAG files

Pull Request - State: open - Opened by OleksiiMamontov 8 months ago

#141 - airflow-log-cleanup Can not find log file correctly

Issue - State: open - Opened by deryann 10 months ago - 1 comment

#140 - Update airflow-db-cleanup.py to work with airflow 2.x

Pull Request - State: open - Opened by nukima about 1 year ago - 2 comments

#139 - Add support for Mysql Backed airflow.

Issue - State: open - Opened by amolsr about 1 year ago - 1 comment

#138 - cleanup-db problem when querying some airflow db models

Issue - State: closed - Opened by juanfassio about 1 year ago - 1 comment

#137 - fix(log-cleanup): Argument list too long

Pull Request - State: open - Opened by dallyger over 1 year ago

#136 - clear-missing-dags failuring

Issue - State: open - Opened by CarlosSilva1994 over 1 year ago - 7 comments

#135 - (#133) feat: add doc_md

Pull Request - State: open - Opened by seunggabi over 1 year ago - 1 comment

#134 - Add compress feature to the log-cleanup

Issue - State: open - Opened by arturbrandysonwelo over 1 year ago - 1 comment

#133 - [docs] add airflow docs by markdown

Issue - State: open - Opened by seunggabi over 1 year ago

#132 - issue on airflow 2.6.0 - airflow-db-cleanup.py - No module named 'airflow.jobs.base_job'

Issue - State: open - Opened by ITJamie almost 2 years ago - 5 comments

#131 - Dependency rule tried to blank-out primary key column 'serialized_dag.dag_id'

Issue - State: closed - Opened by obautista almost 2 years ago - 1 comment

#130 - fix(log-cleanup): use -delete replaces -exec to solve the find command get "line 25: /usr/bin/find: Argument list too long"

Pull Request - State: closed - Opened by LinShihJhang about 2 years ago

#129 - Improve log deletion

Pull Request - State: closed - Opened by merwan over 2 years ago - 1 comment

#128 - sla_readme_images

Issue - State: closed - Opened by nikhilmn over 2 years ago - 1 comment

#127 - Disable strict host checking for SSHing

Pull Request - State: closed - Opened by prakshalj0512 over 2 years ago

#126 - SLA Report

Pull Request - State: closed - Opened by nikhilmn over 2 years ago - 1 comment

#125 - Task 'cleanup_DagModel' fails bcs it cannot delete on table 'dag' - violates foreign key constraint

Issue - State: closed - Opened by Patrikios over 2 years ago - 1 comment

#124 - Airflow 2.2.5 cleanup_DagModel issue

Issue - State: closed - Opened by AlekseiSaff almost 3 years ago - 2 comments

#123 - cleanup_XCom Task | Task exited with return code -9

Issue - State: closed - Opened by LingarajVB almost 3 years ago - 1 comment

#122 - Refactor the db cleanup routine

Pull Request - State: closed - Opened by dudesam about 3 years ago

#121 - Db cleanup failes after upgrade to python3.9 and airflow2.2.2

Issue - State: open - Opened by hanizaidi110 about 3 years ago - 9 comments

#120 - Folder are not deleted

Issue - State: open - Opened by Lazloo over 3 years ago

#119 - Fix: Issue when the database objects passed to the cleanup task via params aren't JSON serializable

Pull Request - State: open - Opened by ealebed over 3 years ago - 3 comments

#118 - [Airflow 2.0] Using log-cleanup in Docker

Issue - State: open - Opened by masbayuw over 3 years ago - 7 comments

#117 - Airflow 2.2.0 compatibility

Issue - State: open - Opened by aa-matthias over 3 years ago - 13 comments

#116 - Clean dependent DagTags prior Dags cleanup

Pull Request - State: closed - Opened by vadipo over 3 years ago - 1 comment

#115 - fix(backup-configs): backup directory variable and check exists

Pull Request - State: closed - Opened by noelmcloughlin over 3 years ago

#114 - fix(airflow2): support last_scheduler_run and last_parsed_time

Pull Request - State: closed - Opened by noelmcloughlin over 3 years ago

#113 - chore(tags): add tags for tracking purposes

Pull Request - State: closed - Opened by noelmcloughlin over 3 years ago

#112 - rename last_scheduler_run -> last_parsed_time

Pull Request - State: closed - Opened by mathi123 over 3 years ago

#111 - DagModel cleanup fails due to dependency record in the DagTag table

Issue - State: closed - Opened by vadipo over 3 years ago - 3 comments

#110 - Airflow 2.0.2 renames the last_scheduler_run column (used in db cleanup job)

Issue - State: closed - Opened by swiffer almost 4 years ago - 1 comment

#109 - Another task is already deleting logs on this worker node.

Issue - State: open - Opened by gfelot almost 4 years ago - 1 comment

#108 - Naive datetime is disallowed in db cleanup dag

Issue - State: open - Opened by JavierLopezT almost 4 years ago

#107 - scripts break airflow?

Issue - State: closed - Opened by picousse almost 4 years ago

#106 - Where to set restore_load_context=True?

Issue - State: closed - Opened by aspain almost 4 years ago - 2 comments

#105 - db-cleanup -> keep last X entries (instead of age based cleaning)

Issue - State: open - Opened by jaceq almost 4 years ago - 1 comment

#104 - Logging Change

Pull Request - State: closed - Opened by prakshalj0512 almost 4 years ago

#103 - cleanup-logs.py - 'dict object' has no attribute 'maxLogAgeInDays'

Issue - State: open - Opened by HansBambel almost 4 years ago - 2 comments

#102 - missing class BaseJob in Airflow 2

Issue - State: closed - Opened by xhuyvn almost 4 years ago - 3 comments

#101 - Check file hashes

Issue - State: open - Opened by Limess about 4 years ago

#100 - Airflow2.0 compatibility

Pull Request - State: closed - Opened by prakshalj0512 about 4 years ago - 5 comments

#99 - airflow 2 compatibility

Issue - State: closed - Opened by swiffer about 4 years ago

#98 - refactor code to work on multiple hosts

Pull Request - State: closed - Opened by prakshalj0512 about 4 years ago

#97 - .lock file created but no permission

Issue - State: open - Opened by gfelot over 4 years ago

#96 - Not deleting airflow logs for master node

Issue - State: open - Opened by chetkhatri over 4 years ago - 2 comments

#95 - [FIX] Added missing quotes for string

Pull Request - State: closed - Opened by mek97 over 4 years ago

#94 - [FIX] Added missing valid 'queued' task state

Pull Request - State: closed - Opened by mek97 over 4 years ago

#93 - Valid queued state task are killed in airflow-kill-halted-tasks DAG

Issue - State: closed - Opened by mek97 over 4 years ago

#92 - db-cleanup also removing dag_run logs (log table)

Issue - State: open - Opened by ghost over 4 years ago

#91 - last char slash fix

Pull Request - State: closed - Opened by prakshalj0512 over 4 years ago - 2 comments

#90 - Files don't get deleted when BASE_LOG_FOLDER ends with /

Issue - State: closed - Opened by prakshalj0512 over 4 years ago

#89 - Log cleanup missing dag_processor_manager.log

Issue - State: closed - Opened by emarthinsen over 4 years ago - 3 comments

#88 - ERROR - pickle data was truncated on TaskInstance: airflow-db-cleanup.cleanup_BaseXCom

Issue - State: open - Opened by artik00 over 4 years ago - 2 comments

#87 - Will log cleanup dag work with kubernetes?

Issue - State: open - Opened by axk7812 over 4 years ago

#86 - Configuration variable empty

Issue - State: open - Opened by gfelot over 4 years ago - 5 comments

#85 - Fix Keep Last Logic

Pull Request - State: closed - Opened by xyu over 4 years ago

#84 - shouldn't log-cleanup task run in sequence?

Issue - State: closed - Opened by woorulez over 4 years ago - 2 comments

#83 - airflow-db-cleanup - RenderedTaskInstanceFields could not be found

Issue - State: closed - Opened by match-gabeflores over 4 years ago - 1 comment

#82 - [FIX] rendered_task_instance not available in airflow 1.10.9

Pull Request - State: closed - Opened by prakshalj0512 over 4 years ago

#81 - which Airflow versions does this repo support?

Issue - State: closed - Opened by feng-tao over 4 years ago - 2 comments

#80 - [FIX] checks for celery imports

Pull Request - State: closed - Opened by prakshalj0512 over 4 years ago

#79 - clean-db broken unless you install celerity

Issue - State: closed - Opened by Daniel-SBD over 4 years ago - 1 comment

#78 - [ADD] added more metadata models and pep8 formatting

Pull Request - State: closed - Opened by prakshalj0512 over 4 years ago

#77 - adding session.commit() in this script else the changes will not be committed after delete

Pull Request - State: closed - Opened by soupam05 over 4 years ago

#76 - Airflow runs out of memory on large MySQL databases

Issue - State: closed - Opened by criccomini over 4 years ago - 2 comments

#75 - celery_taskmeta and celery_tasksetmeta tables

Issue - State: closed - Opened by royt-via over 4 years ago - 2 comments

#74 - Add directory id to task_id to avoid task dependency duplication

Pull Request - State: closed - Opened by lx950627 almost 5 years ago

#73 - flake8 compliance

Pull Request - State: closed - Opened by 7yl4r almost 5 years ago - 1 comment

#72 - Adding the backup-configs workflow

Pull Request - State: closed - Opened by rssanders3 almost 5 years ago

#71 - Guaranteed cleanup in all workers?

Issue - State: closed - Opened by akhilbatra898 almost 5 years ago - 2 comments

#70 - Fix issue #69

Pull Request - State: closed - Opened by prakshalj0512 almost 5 years ago

#69 - airflow-log-cleanup: line 56: syntax error near unexpected token `}'

Issue - State: closed - Opened by lekseven almost 5 years ago - 1 comment

#69 - airflow-log-cleanup: line 56: syntax error near unexpected token `}'

Issue - State: closed - Opened by lekseven almost 5 years ago - 1 comment

#68 - [RESOLVE] issue 64 & 67

Pull Request - State: closed - Opened by prakshalj0512 almost 5 years ago

#67 - log-cleanup tries to delete all files when BASE_LOG_FOLDER is empty string

Issue - State: closed - Opened by bbonagura9 about 5 years ago

#66 - Fix Boolean error in DagRun db-cleanup (postgresql)

Pull Request - State: closed - Opened by flolas about 5 years ago

#65 - Query error in airflow db cleanup (DagRunModel)

Issue - State: closed - Opened by flolas about 5 years ago

#64 - airflow_log_cleanup DAG creating DUPLICATE task instances when there a change in Code and Deploy

Issue - State: closed - Opened by ajay999aj about 5 years ago - 1 comment

#63 - Adding support for preserving last_runs & adding filters to DELETE queries

Pull Request - State: closed - Opened by prakshalj0512 about 5 years ago

#62 - How to guarantee that the maintenance will run on every worker.

Issue - State: closed - Opened by akhilbatra898 about 5 years ago - 1 comment

#61 - airflow-log-cleanup task may fail if BASE_LOG_FOLDER is parent dir of CHILD_PROCESS_LOG_DIRECTORY

Issue - State: closed - Opened by ChenyuanZ about 5 years ago - 2 comments

#60 - Bugfix/dag id for delete broken dags

Pull Request - State: closed - Opened by emartgu about 5 years ago

#59 - Delete broken dags

Pull Request - State: closed - Opened by emartgu over 5 years ago

#58 - Import error while execution of script

Issue - State: closed - Opened by ajay999aj over 5 years ago - 2 comments

#57 - MySQL Table specified twice issue

Pull Request - State: closed - Opened by sudhakar-nallam over 5 years ago

#56 - Missing creation of now variable in latest commits

Issue - State: closed - Opened by julien-dan over 5 years ago - 1 comment

#55 - disk space not changed after running this cleanup dag

Issue - State: closed - Opened by qiangbo over 5 years ago - 2 comments

#54 - Invalid syntax error

Issue - State: closed - Opened by GiancarloFusiello over 5 years ago

#53 - fix invalid exception syntax

Pull Request - State: closed - Opened by GiancarloFusiello over 5 years ago - 1 comment

#52 - cleanup_DagRun crash

Issue - State: closed - Opened by Stormhand over 5 years ago - 3 comments

#51 - Use '-exec rm' in find cmd to delete files/dirs

Pull Request - State: closed - Opened by emartgu over 5 years ago

#50 - Says: Only works with the CeleryExecutor, sorry

Issue - State: closed - Opened by joaolcorreia over 5 years ago - 3 comments

#49 - Improve process parsing

Pull Request - State: closed - Opened by emartgu over 5 years ago

#48 - What Airflow version does the this repo support?

Issue - State: closed - Opened by feng-tao over 5 years ago - 1 comment

GitHub / teamclairvoyant/airflow-maintenance-dags issues and pull requests