cscfi/kielipankki-nlf-harvester issues and pull requests

#64 - KP-7988 Downgrade to a large Pouta instance

Pull Request - State: closed - Opened by Traubert 3 months ago

#64 - KP-7988 Downgrade to a large Pouta instance

Pull Request - State: closed - Opened by Traubert 3 months ago

#63 - KP-8921 Unpin dependencies and add vars

Pull Request - State: closed - Opened by Traubert 3 months ago

#63 - KP-8921 Unpin dependencies and add vars

Pull Request - State: closed - Opened by Traubert 3 months ago

#62 - KP-7511 update functionality

Pull Request - State: closed - Opened by Traubert 11 months ago

#62 - KP-7511 update functionality

Pull Request - State: closed - Opened by Traubert 11 months ago

#61 - Encourage using Puhti with restic again

Pull Request - State: closed - Opened by aajarven 12 months ago

#61 - Encourage using Puhti with restic again

Pull Request - State: closed - Opened by aajarven 12 months ago

#60 - KP-7861

Pull Request - State: closed - Opened by aajarven 12 months ago

#60 - KP-7861

Pull Request - State: closed - Opened by aajarven 12 months ago

#59 - KP-7522 Rest of the docs

Pull Request - State: closed - Opened by aajarven 12 months ago - 1 comment

#59 - KP-7522 Rest of the docs

Pull Request - State: closed - Opened by aajarven 12 months ago - 1 comment

#58 - KP-7610 Using SEQ attribute to determine page download URLs

Pull Request - State: closed - Opened by aajarven about 1 year ago

#58 - KP-7610 Using SEQ attribute to determine page download URLs

Pull Request - State: closed - Opened by aajarven about 1 year ago

#57 - Add option to zip-updater to skip compression for some file types & reorganize

Pull Request - State: closed - Opened by Traubert about 1 year ago

#57 - Add option to zip-updater to skip compression for some file types & reorganize

Pull Request - State: closed - Opened by Traubert about 1 year ago

#56 - Zip updater fixes

Pull Request - State: closed - Opened by Traubert about 1 year ago

#56 - Zip updater fixes

Pull Request - State: closed - Opened by Traubert about 1 year ago

#55 - KP-7716 Fixes for doing the image run

Pull Request - State: closed - Opened by aajarven about 1 year ago

#54 - WIP: KP-7522 Docs

Pull Request - State: closed - Opened by aajarven about 1 year ago - 2 comments

#54 - WIP: KP-7522 Docs

Pull Request - State: closed - Opened by aajarven about 1 year ago - 2 comments

#53 - KP-7716 Misc life quality improvements and fixes

Pull Request - State: closed - Opened by aajarven about 1 year ago

#53 - KP-7716 Misc life quality improvements and fixes

Pull Request - State: closed - Opened by aajarven about 1 year ago

#52 - KP-7522 Docs cleanup

Pull Request - State: closed - Opened by aajarven about 1 year ago - 1 comment

#52 - KP-7522 Docs cleanup

Pull Request - State: closed - Opened by aajarven about 1 year ago - 1 comment

#51 - KP-7159 zip targets

Pull Request - State: closed - Opened by Traubert about 1 year ago

#51 - KP-7159 zip targets

Pull Request - State: closed - Opened by Traubert about 1 year ago

#50 - KP-7159 Factor out code for creating needed directories

Pull Request - State: closed - Opened by Traubert about 1 year ago

#50 - KP-7159 Factor out code for creating needed directories

Pull Request - State: closed - Opened by Traubert about 1 year ago

#49 - KP-6210 Operator extendability refactoring

Pull Request - State: closed - Opened by aajarven about 1 year ago

#49 - KP-6210 Operator extendability refactoring

Pull Request - State: closed - Opened by aajarven about 1 year ago

#48 - KP-6210 Split custom operator test module

Pull Request - State: closed - Opened by aajarven about 1 year ago

#48 - KP-6210 Split custom operator test module

Pull Request - State: closed - Opened by aajarven about 1 year ago

#47 - KP-6210 Download access images

Pull Request - State: closed - Opened by aajarven about 1 year ago

#47 - KP-6210 Download access images

Pull Request - State: closed - Opened by aajarven about 1 year ago

#46 - KP-6210 split operator module

Pull Request - State: closed - Opened by aajarven about 1 year ago

#46 - KP-6210 split operator module

Pull Request - State: closed - Opened by aajarven about 1 year ago

#45 - Bump minimum Ansible version requirement

Pull Request - State: closed - Opened by aajarven about 1 year ago

#45 - Bump minimum Ansible version requirement

Pull Request - State: closed - Opened by aajarven about 1 year ago

#44 - Cleanup fixes

Pull Request - State: closed - Opened by Traubert about 1 year ago

#44 - Cleanup fixes

Pull Request - State: closed - Opened by Traubert about 1 year ago

#43 - Disable ignoring files from old images

Pull Request - State: closed - Opened by Traubert over 1 year ago

#43 - Disable ignoring files from old images

Pull Request - State: closed - Opened by Traubert over 1 year ago

#42 - Fix sqfstar arg order

Pull Request - State: closed - Opened by Traubert over 1 year ago

#42 - Fix sqfstar arg order

Pull Request - State: closed - Opened by Traubert over 1 year ago

#41 - KP-6678 Create restic snapshot to allas after each download

Pull Request - State: closed - Opened by helhotti over 1 year ago - 5 comments

#41 - KP-6678 Create restic snapshot to allas after each download

Pull Request - State: closed - Opened by helhotti over 1 year ago - 5 comments

#40 - Switch to using `find` to find .tmp files

Pull Request - State: closed - Opened by Traubert over 1 year ago

#40 - Switch to using `find` to find .tmp files

Pull Request - State: closed - Opened by Traubert over 1 year ago

#39 - Switch to using `find` to find .tmp files

Pull Request - State: closed - Opened by Traubert over 1 year ago - 1 comment

#39 - Switch to using `find` to find .tmp files

Pull Request - State: closed - Opened by Traubert over 1 year ago - 1 comment

#38 - If downloading fails with a RequestException, clean up the .tmp file

Pull Request - State: closed - Opened by Traubert over 1 year ago

#38 - If downloading fails with a RequestException, clean up the .tmp file

Pull Request - State: closed - Opened by Traubert over 1 year ago

#37 - KP-7399 Image creation via tar approach

Pull Request - State: closed - Opened by Traubert over 1 year ago - 2 comments

#37 - KP-7399 Image creation via tar approach

Pull Request - State: closed - Opened by Traubert over 1 year ago - 2 comments

#36 - KP-7389 Fix broken old image extraction

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#36 - KP-7389 Fix broken old image extraction

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#35 - KP-7399 Reduce the max number of bindings in an image

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#34 - KP-7398 Use SSH key with Puhti connection

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#34 - KP-7398 Use SSH key with Puhti connection

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#33 - KP-7397 Limit mksquashfs memory usage to 2GB

Pull Request - State: closed - Opened by aajarven over 1 year ago

#33 - KP-7397 Limit mksquashfs memory usage to 2GB

Pull Request - State: closed - Opened by aajarven over 1 year ago

#32 - KP-7389 post-refactoring

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#32 - KP-7389 post-refactoring

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#31 - KP-7389 Refactoring before functional changes

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#31 - KP-7389 Refactoring before functional changes

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#30 - Allow provisioning DAGs only, and from a specific branch

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#30 - Allow provisioning DAGs only, and from a specific branch

Pull Request - State: closed - Opened by aajarven over 1 year ago - 1 comment

#29 - KP-7389 Download individual files to fast local disk instead of lustre

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#29 - KP-7389 Download individual files to fast local disk instead of lustre

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#28 - KP-7312 & KP-6749

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#28 - KP-7312 & KP-6749

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#27 - Procedure for creating disk images of NLF data

Pull Request - State: closed - Opened by helhotti over 1 year ago - 4 comments

#26 - Implement logic to divide bindings into disc images

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#26 - Implement logic to divide bindings into disc images

Pull Request - State: closed - Opened by helhotti over 1 year ago - 1 comment

#25 - Move ansible files from Kielipankki repo

Pull Request - State: closed - Opened by helhotti over 1 year ago

#25 - Move ansible files from Kielipankki repo

Pull Request - State: closed - Opened by helhotti over 1 year ago

#24 - KP-7021 Construct subdirectory structure

Pull Request - State: closed - Opened by helhotti over 1 year ago - 3 comments

#24 - KP-7021 Construct subdirectory structure

Pull Request - State: closed - Opened by helhotti over 1 year ago - 3 comments

#23 - Separate DAGs for collections & batch download

Pull Request - State: closed - Opened by helhotti over 1 year ago

#23 - Separate DAGs for collections & batch download

Pull Request - State: closed - Opened by helhotti over 1 year ago

#22 - KP-6842 Skip download of existing non-empty files

Pull Request - State: closed - Opened by helhotti over 1 year ago

#22 - KP-6842 Skip download of existing non-empty files

Pull Request - State: closed - Opened by helhotti over 1 year ago

#21 - WIP: KP-6684 Ensure that empty files are not created

Pull Request - State: closed - Opened by helhotti almost 2 years ago - 1 comment

#21 - WIP: KP-6684 Ensure that empty files are not created

Pull Request - State: closed - Opened by helhotti almost 2 years ago - 1 comment

#20 - KP-6748 Use ListIdentifiers instead of ListRecords

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#20 - KP-6748 Use ListIdentifiers instead of ListRecords

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#19 - KP-6483 Remove stale operators and DAGs

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#19 - KP-6483 Remove stale operators and DAGs

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#18 - KP-6483 Depth-first download procedure

Pull Request - State: closed - Opened by helhotti almost 2 years ago - 1 comment

#17 - Fixes for breadth-first ALTO download

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#17 - Fixes for breadth-first ALTO download

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#16 - Handle errors related to DC identifier listing

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#16 - Handle errors related to DC identifier listing

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#15 - KP-6534 Change print statements to proper logging

Pull Request - State: closed - Opened by helhotti almost 2 years ago - 1 comment

#14 - KP-6568 Fixes to enable download of all available collections

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#13 - KP-6211 Specify requirements for Airflow pipeline too

Pull Request - State: closed - Opened by aajarven almost 2 years ago - 2 comments

#12 - KP-6520 Hack ALTO file paths for download links

Pull Request - State: closed - Opened by helhotti almost 2 years ago

#11 - KP-5823 Download all METS and ALTO files for one collection to Puhti

Pull Request - State: closed - Opened by helhotti about 2 years ago - 2 comments

#10 - KP-6206/KP-6208 Airflow testing POC

Pull Request - State: closed - Opened by aajarven about 2 years ago

GitHub / cscfi/kielipankki-nlf-harvester issues and pull requests