getzlab/canine issues and pull requests

#153 - Update file_handlers.py get_requester_pays

Pull Request - State: closed - Opened by cchu70 about 2 months ago - 1 comment

#152 - Drs via drshub - faster localization

Pull Request - State: closed - Opened by evergreen700 6 months ago

#151 - Replaced almost all gsutil calls with gcloud storage

Pull Request - State: closed - Opened by evergreen700 6 months ago

#150 - Improved delocalization checksum

Pull Request - State: open - Opened by dpmerrell 6 months ago

#149 - GS localization fails if user doesn't have list permissions for bucket of input

Pull Request - State: closed - Opened by njharlen 6 months ago

#148 - Implement DRS URI localization.

Pull Request - State: closed - Opened by dheiman 8 months ago

#147 - Added -m flag to `gsutil cp` in GSURL file handler localization command

Pull Request - State: open - Opened by dpmerrell 8 months ago

#146 - Add localizer for DRS URIs, and update GDC handler to use it when available

Issue - State: open - Opened by dheiman 9 months ago - 7 comments
Labels: enhancement, Triage

#143 - Delete file on md5 check failure

Issue - State: open - Opened by dheiman about 1 year ago
Labels: enhancement, Triage

#142 - Recover from controller timeout

Pull Request - State: closed - Opened by julianhess about 1 year ago

#141 - Add new condition here for localization that hit quota issue; infinite retries

Issue - State: open - Opened by julianhess about 1 year ago - 3 comments

#140 - Wipe output directory upon job retry

Issue - State: open - Opened by julianhess over 1 year ago

#139 - mount error not caught

Issue - State: closed - Opened by jalberge over 1 year ago - 2 comments

#138 - NFS improvements

Pull Request - State: closed - Opened by julianhess over 1 year ago

#137 - Distinguish restarts due to preemptions from restarts due to failures

Pull Request - State: closed - Opened by julianhess over 1 year ago

#136 - Suspend job that's been preempted too often

Pull Request - State: closed - Opened by julianhess over 1 year ago

#135 - delete corrupted files upon md5 failure

Pull Request - State: closed - Opened by oliverpriebe over 1 year ago

#134 - Disk fixes

Pull Request - State: closed - Opened by julianhess over 1 year ago

#133 - Localization tweaks

Pull Request - State: closed - Opened by julianhess over 1 year ago

#132 - Add support for common VM/Docker images

Pull Request - State: closed - Opened by julianhess over 1 year ago

#131 - Enable toggling of scratch disk job avoidance

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#130 - More improvements to disk handling

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#129 - Compute partial hashes for big outputs

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#128 - Don't kill straggling jobs upon backend exit

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#127 - Make workflow log more user friendly

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#126 - Fix bug in scatter of gather jobs

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#125 - Dramatically increase speed of GCS operations

Pull Request - State: closed - Opened by julianhess almost 2 years ago

#124 - More disk robustness updates

Pull Request - State: closed - Opened by julianhess about 2 years ago

#123 - Scratch disk robustness improvements

Pull Request - State: closed - Opened by julianhess about 2 years ago

#122 - Make inspecting disks much faster (use API vs. gcloud CLI)

Pull Request - State: closed - Opened by julianhess about 2 years ago

#121 - fix localized_path issue

Pull Request - State: closed - Opened by julianhess about 2 years ago

#120 - Revamp how job spec is created from raw input arrays

Pull Request - State: closed - Opened by julianhess about 2 years ago

#119 - Properly mangle all inputs if their filenames are duplicated

Pull Request - State: closed - Opened by julianhess about 2 years ago

#118 - Disable exit 15 avoidance in debug mode

Pull Request - State: closed - Opened by julianhess about 2 years ago

#117 - Do not run delocalization script in debug mode, or if we had an exit 15 avoidance

Pull Request - State: closed - Opened by julianhess about 2 years ago

#116 - Revamp delocalization hashing code

Pull Request - State: closed - Opened by julianhess about 2 years ago

#115 - Add pass-through support for rodisk URLs

Pull Request - State: closed - Opened by julianhess about 2 years ago

#114 - Remove trailing slashes from local paths

Pull Request - State: closed - Opened by julianhess over 2 years ago

#113 - Patch _get_size() to recursively determine total disk utilization for…

Pull Request - State: closed - Opened by yoakiyama over 2 years ago

#112 - Block when trying to create a RODISK whose creation is already in progress

Pull Request - State: closed - Opened by julianhess over 2 years ago

#111 - added stream capabilities for GDC and AWS URLs

Pull Request - State: closed - Opened by oliverpriebe over 2 years ago

#110 - crossmnt to allow for nested mounts in NFS

Pull Request - State: closed - Opened by julianhess over 2 years ago

#109 - Automatically purge files in working dir that aren't captured as outputs

Pull Request - State: closed - Opened by julianhess over 2 years ago

#108 - Add dry run support for persistent disk creation

Pull Request - State: closed - Opened by julianhess over 2 years ago

#107 - Symlink check

Pull Request - State: closed - Opened by julianhess over 2 years ago

#106 - Don't clobber existing files when symlinking from common

Pull Request - State: closed - Opened by julianhess over 2 years ago

#105 - Fix copying gs:// directories

Pull Request - State: closed - Opened by julianhess over 2 years ago

#104 - Localize AWS S3 files

Pull Request - State: closed - Opened by julianhess over 2 years ago

#103 - Overhaul localization to disk

Pull Request - State: closed - Opened by julianhess almost 3 years ago

#102 - Allow for job avoidance to be disabled

Pull Request - State: closed - Opened by julianhess about 3 years ago

#101 - Use manifest to track partial gsutil downloads

Pull Request - State: closed - Opened by julianhess about 3 years ago - 1 comment

#100 - Use different gce client per process

Pull Request - State: closed - Opened by Marlin-Na over 3 years ago

#99 - Make stderr steps clearer

Pull Request - State: closed - Opened by julianhess over 3 years ago

#98 - Use np.datatime64 for Submit default value

Pull Request - State: closed - Opened by Marlin-Na over 3 years ago

#97 - Fix job avoidance when .sacct file is missing

Pull Request - State: closed - Opened by Marlin-Na over 3 years ago - 3 comments

#96 - Stringify type checks

Pull Request - State: closed - Opened by julianhess over 3 years ago

#95 - Allow rodisk url with nested directory, e.g. rodisk://a/b/c

Pull Request - State: closed - Opened by Marlin-Na over 3 years ago - 1 comment

#94 - gce thread lock -> process lock

Pull Request - State: closed - Opened by Marlin-Na over 3 years ago - 1 comment

#93 - Use delayed localization by default for gs:// files

Pull Request - State: closed - Opened by julianhess over 3 years ago - 6 comments

#92 - Reopen #89 against master

Pull Request - State: closed - Opened by Marlin-Na almost 4 years ago

#91 - Docker backend serve NFS from controller

Pull Request - State: closed - Opened by Marlin-Na almost 4 years ago

#90 - Fix tests and enable CI through actions

Pull Request - State: closed - Opened by agraubert almost 4 years ago

#89 - Use increased logging level (INFO + 1)

Pull Request - State: closed - Opened by Marlin-Na almost 4 years ago

#88 - Miscellaneous rare bug fixes

Pull Request - State: closed - Opened by julianhess almost 4 years ago

#87 - Use CRC32C instead of SHA1 for hashing outputs

Pull Request - State: closed - Opened by julianhess almost 4 years ago

#86 - Array inputs, v2

Pull Request - State: closed - Opened by julianhess almost 4 years ago - 2 comments

#85 - Suppress 'ls' exit code in localization.sh

Pull Request - State: closed - Opened by Marlin-Na almost 4 years ago - 1 comment

#84 - Symlink canine common inputs to job specific inputs directory

Pull Request - State: closed - Opened by Marlin-Na almost 4 years ago