argonne-lcf/dlio_benchmark issues and pull requests

#165 - DataLoader num_workers > 0 causes CPU memory from parent process to be replicated in all worker processes

Issue - State: open - Opened by sethiay 7 months ago

#164 - Fixed random samples issue and added more github actions to test the configuration files

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#164 - Fixed random samples issue and added more github actions to test the configuration files

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#163 - Fixing github action issues (#162)

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#163 - Fixing github action issues (#162)

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#162 - Fixing github action issues

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#162 - Fixing github action issues

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#161 - Emulate GPU without sleep

Issue - State: open - Opened by spitzcor 7 months ago

#161 - Emulate GPU without sleep

Issue - State: open - Opened by spitzcor 7 months ago

#160 - multiple GPUs accessing the same part of the dataset

Issue - State: closed - Opened by yxkyong 7 months ago - 4 comments

#160 - multiple GPUs accessing the same part of the dataset

Issue - State: closed - Opened by yxkyong 7 months ago - 4 comments

#159 - Mlperf storage v1.0

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#159 - Mlperf storage v1.0

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#158 - DALI data loader fix and configuration files update for new batch sizes

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#158 - DALI data loader fix and configuration files update for new batch sizes

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#157 - Pytorch taking much longer than TensorFlow for Bert workload

Issue - State: closed - Opened by TheAssembler1 7 months ago - 1 comment

#157 - Pytorch taking much longer than TensorFlow for Bert workload

Issue - State: closed - Opened by TheAssembler1 7 months ago - 1 comment

#156 - Optimization to sample distribution

Pull Request - State: closed - Opened by TheAssembler1 7 months ago

#156 - Optimization to sample distribution

Pull Request - State: closed - Opened by TheAssembler1 7 months ago

#155 - Sample Distribution Optimization

Issue - State: closed - Opened by TheAssembler1 7 months ago - 3 comments

#155 - Sample Distribution Optimization

Issue - State: closed - Opened by TheAssembler1 7 months ago - 3 comments

#154 - Fix sample shuffling

Pull Request - State: closed - Opened by hariharan-devarajan 7 months ago

#154 - Fix sample shuffling

Pull Request - State: closed - Opened by hariharan-devarajan 7 months ago

#152 - fixed sample distribution bugs

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#152 - fixed sample distribution bugs

Pull Request - State: closed - Opened by zhenghh04 7 months ago

#151 - TensorFlow Training Array Out Of Bounds Exception Error

Issue - State: closed - Opened by TheAssembler1 8 months ago - 4 comments

#151 - TensorFlow Training Array Out Of Bounds Exception Error

Issue - State: closed - Opened by TheAssembler1 8 months ago - 4 comments

#149 - Seems like different threads are reading some overlapping files in torch data loader

Issue - State: open - Opened by sethiay 8 months ago - 12 comments

#149 - Seems like different threads are reading some overlapping files in torch data loader

Issue - State: open - Opened by sethiay 8 months ago - 12 comments

#148 - Feature/parallel io generator

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#148 - Feature/parallel io generator

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#147 - fix random bugs and printing

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#147 - fix random bugs and printing

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#146 - Doc changes to fix DLIO profiler and remove IOStat

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#146 - Doc changes to fix DLIO profiler and remove IOStat

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#145 - Fix random sampling pytorch non-determinism.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#145 - Fix random sampling pytorch non-determinism.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#143 - Fix dataloader initialization to only happen once. Not on every epoch.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#143 - Fix dataloader initialization to only happen once. Not on every epoch.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago

#142 - Fixed printing for DLIO output.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago - 1 comment

#142 - Fixed printing for DLIO output.

Pull Request - State: closed - Opened by hariharan-devarajan 8 months ago - 1 comment

#141 - any way to profile memory usage in dlio?

Issue - State: open - Opened by krehm 8 months ago - 18 comments

#141 - any way to profile memory usage in dlio?

Issue - State: open - Opened by krehm 8 months ago - 18 comments

#139 - Fix MPI finalization.

Pull Request - State: closed - Opened by hariharan-devarajan 9 months ago

#139 - Fix MPI finalization.

Pull Request - State: closed - Opened by hariharan-devarajan 9 months ago

#137 - Support for custom checkpointing.

Pull Request - State: closed - Opened by hariharan-devarajan 9 months ago - 1 comment

#137 - Support for custom checkpointing.

Pull Request - State: closed - Opened by hariharan-devarajan 9 months ago - 1 comment

#128 - Reducing Github actions time

Pull Request - State: closed - Opened by zhenghh04 10 months ago

#128 - Reducing Github actions time

Pull Request - State: closed - Opened by zhenghh04 10 months ago

#123 - Added force install for profiler for avoiding caching issues

Pull Request - State: closed - Opened by hariharan-devarajan 10 months ago

#123 - Added force install for profiler for avoiding caching issues

Pull Request - State: closed - Opened by hariharan-devarajan 10 months ago

#120 - Switch dlio_profiler to use pypi instead of github

Pull Request - State: closed - Opened by hariharan-devarajan 10 months ago

#120 - Switch dlio_profiler to use pypi instead of github

Pull Request - State: closed - Opened by hariharan-devarajan 10 months ago

#118 - Adding Native Dali Data Loader support for TFRecord, Images, and NPZ files

Pull Request - State: closed - Opened by zhenghh04 10 months ago - 7 comments
Labels: enhancement

#118 - Adding Native Dali Data Loader support for TFRecord, Images, and NPZ files

Pull Request - State: closed - Opened by zhenghh04 10 months ago - 7 comments
Labels: enhancement

#114 - added checkpointing to support LLMs

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago - 4 comments

#114 - added checkpointing to support LLMs

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago - 4 comments

#113 - Release for v2.0

Pull Request - State: closed - Opened by zhenghh04 11 months ago

#113 - Release for v2.0

Pull Request - State: closed - Opened by zhenghh04 11 months ago

#112 - Workloads

Pull Request - State: closed - Opened by zhenghh04 11 months ago

#112 - Workloads

Pull Request - State: closed - Opened by zhenghh04 11 months ago

#111 - Making DLIO Profiler default for dlio_benchmark.

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago - 1 comment

#111 - Making DLIO Profiler default for dlio_benchmark.

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago - 1 comment

#109 - Adding dlp logger.

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago

#109 - Adding dlp logger.

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago

#108 - Fix the data dir path

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago

#108 - Fix the data dir path

Pull Request - State: closed - Opened by hariharan-devarajan 11 months ago

#104 - Added Info logging for profiler and removed unnecessary bracket calls.

Pull Request - State: closed - Opened by hariharan-devarajan 12 months ago

#104 - Added Info logging for profiler and removed unnecessary bracket calls.

Pull Request - State: closed - Opened by hariharan-devarajan 12 months ago

#103 - Update typo which gives issue for pytorch 1.3.1

Pull Request - State: closed - Opened by hariharan-devarajan 12 months ago

#103 - Update typo which gives issue for pytorch 1.3.1

Pull Request - State: closed - Opened by hariharan-devarajan 12 months ago

#93 - Fixing Dali Data loader Parallelism and Pipelining.

Pull Request - State: closed - Opened by hariharan-devarajan almost 1 year ago

#93 - Fixing Dali Data loader Parallelism and Pipelining.

Pull Request - State: closed - Opened by hariharan-devarajan almost 1 year ago

#89 - Changes to improve documentation

Pull Request - State: closed - Opened by hariharan-devarajan about 1 year ago - 3 comments

#89 - Changes to improve documentation

Pull Request - State: closed - Opened by hariharan-devarajan about 1 year ago - 3 comments

GitHub / argonne-lcf/dlio_benchmark issues and pull requests