NVIDIA/spark-rapids-tools issues and pull requests

#264 - Updating profiling docs to fix usability issues

Pull Request - State: closed - Opened by mattahrens over 1 year ago

#263 - [FEA] Dynamic factors for qualification tool

Issue - State: closed - Opened by nvliyuan over 1 year ago - 6 comments
Labels: feature request, tools

#262 - [FEA] Add support in Profiling tool for multithreaded settings

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, tools

#261 - Add GPU cluster creation script to Qualification output

Pull Request - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: feature request, user_tools

#259 - [DOC] Suggestions for spark-rapids-profiling-tool.md

Issue - State: closed - Opened by jbrennan333 over 1 year ago - 1 comment
Labels: documentation

#258 - [FEA] Enhance qualification tool to report with granularity on unsupported operations

Issue - State: open - Opened by mattahrens over 1 year ago - 5 comments
Labels: feature request, tools

#257 - [FEA] Create platform-independent diagnostic tool

Issue - State: open - Opened by mattahrens over 1 year ago - 4 comments
Labels: feature request, user_tools

#255 - [FEA] Add info on how to run bootstrap in qualification output for user tools

Issue - State: closed - Opened by mattahrens over 1 year ago - 2 comments
Labels: feature request, user_tools

#252 - [FEA] Generate speedup factors for Dataproc L4 cluster

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, tools

#251 - [FEA] Add qualification support for Databricks Photon event logs

Issue - State: open - Opened by mattahrens over 1 year ago - 2 comments
Labels: feature request, tools

#250 - [FEA] Support target platform of Dataproc for user tools qualification for on-prem platform

Issue - State: open - Opened by mattahrens over 1 year ago - 1 comment
Labels: feature request, user_tools

#249 - [FEA] Autotuner should recommend tuned settings (batchSizeBytes, concurrentGpuTasks) for OOM conditions

Issue - State: open - Opened by mattahrens over 1 year ago - 2 comments
Labels: feature request, tools, reliability

#248 - [FEA] Autotuner should recommend tuned settings (advisorySizeInBytes, shuffle partitions) based on shuffle sizes

Issue - State: open - Opened by mattahrens over 1 year ago - 2 comments
Labels: feature request, tools, reliability

#244 - [DOC] Update doc for supported ML functions in scala and pyspark

Issue - State: open - Opened by nartal1 over 1 year ago
Labels: documentation, tools

#235 - [BUG] Create separate test module in core

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: bug, tools, build

#233 - [BUG] Autotuner runs against CPU eventLogs

Issue - State: closed - Opened by razajafri over 1 year ago
Labels: bug, tools

#225 - [FEA] Add user tools qualification support for the Databricks Azure platform

Issue - State: open - Opened by mattahrens over 1 year ago - 1 comment
Labels: feature request, user_tools

#223 - [FEA] Add support in user tools qualification for on-prem platform

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, user_tools

#218 - [BUG] Qualification tool output columns with commas need to be in quotations for parsing CSV

Issue - State: open - Opened by mattahrens over 1 year ago - 3 comments
Labels: bug, tools

#217 - Job frequency estimation

Pull Request - State: closed - Opened by rwlee over 1 year ago - 4 comments
Labels: user_tools, tools

#215 - [FEA] Add multitenant support for qualification user tools for more accurate cost calculations

Issue - State: closed - Opened by mattahrens over 1 year ago - 1 comment
Labels: feature request, user_tools

#214 - [FEA] Add validation tool to core module

Issue - State: open - Opened by nartal1 over 1 year ago
Labels: feature request, user_tools, tools

#213 - Disable recommendations for CPU logs

Pull Request - State: closed - Opened by razajafri over 1 year ago - 2 comments
Labels: tools

#212 - [FEA] Validate estimated speedup for Spark-ETL + Spark ML/XGBoost functions

Issue - State: closed - Opened by nartal1 over 1 year ago - 1 comment
Labels: feature request, tools

#211 - [FEA] Generate speedup factors for ML operators (#2)

Issue - State: open - Opened by nartal1 over 1 year ago - 1 comment
Labels: feature request, tools

#182 - [FEA] Add support to cluster-mode for dataproc in new CLI

Issue - State: closed - Opened by amahussein over 1 year ago - 4 comments
Labels: feature request, user_tools

#181 - Detect the user-tools version number by parsing the manifest of Jar file

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: user_tools

#176 - [FEA] Update user tools functionality to support Windows usage

Issue - State: open - Opened by mattahrens over 1 year ago - 5 comments
Labels: feature request, user_tools

#173 - add metadata validation for hive format

Pull Request - State: closed - Opened by nvliyuan over 1 year ago - 3 comments

#167 - [FEA] Support nested directories in qualification tool

Issue - State: closed - Opened by mattahrens over 1 year ago - 3 comments
Labels: feature request, good first issue, tools

#156 - [FEA] Follow-up on Databricks qualification tool support

Issue - State: closed - Opened by cindyyuanjiang over 1 year ago - 1 comment
Labels: feature request, user_tools

#153 - [FEA] Add support in Bootstrap tool for multithreaded settings

Issue - State: open - Opened by mattahrens over 1 year ago - 3 comments
Labels: feature request, user_tools

#149 - [FEA] Profiler recommendations should include upgrade to newer Spark RAPIDS jar

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, tools

#141 - [DOC] Update spark-rapids-user-tools README

Issue - State: closed - Opened by sameerz over 1 year ago - 2 comments
Labels: documentation, ? - Needs Triage

#140 - [Build] Pre-commit checks and build profiles

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: feature request, cicd, build

#139 - [BUG] Revisit parsing of some Execs that produce incorrect expressions

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: bug, tools

#135 - [BUG] Unit test failures on Spark3.3.0

Issue - State: closed - Opened by amahussein over 1 year ago - 2 comments
Labels: bug, tools

#122 - [BUG] Use build configurations to load package metadata

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: bug, user_tools

#117 - [FEA] Update unit tests for Dataproc user tools to not depend on external gcloud resources

Issue - State: closed - Opened by mattahrens over 1 year ago - 1 comment
Labels: feature request, user_tools

#111 - With automatic speedup generation, how do integration tests works?

Issue - State: closed - Opened by tgravescs over 1 year ago - 2 comments
Labels: feature request, tools

#110 - Profiling tool: I/O metrics follow on

Issue - State: open - Opened by nartal1 over 1 year ago
Labels: feature request, tools

#108 - [BUG] Inconsistent GPU speedups when running on different environment

Issue - State: closed - Opened by amahussein over 1 year ago - 4 comments
Labels: bug, user_tools, tools

#103 - [FEA] Add spreadsheet template for qualification tool output to estimate annualized cost savings

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, user_tools

#102 - [FEA] Add support in qualification tool to predict Delta-related operator speed-up

Issue - State: open - Opened by mattahrens over 1 year ago
Labels: feature request, tools, epic

#101 - [FEA] Add savings estimates to the UI summary report

Issue - State: closed - Opened by amahussein over 1 year ago - 1 comment
Labels: feature request, user_tools

#96 - [BUG] Empty strings not generated for Unsupported Execs andUnsupported Expressions columns

Issue - State: closed - Opened by mattahrens over 1 year ago - 2 comments
Labels: bug, tools

#95 - [BUG] Scala tools do not return code to indicate errors

Issue - State: closed - Opened by amahussein over 1 year ago - 2 comments
Labels: bug, user_tools, tools

#88 - [FEA] Add in estimated frequency of job in qual tool output to help with estimated annualized cost savings

Issue - State: closed - Opened by mattahrens over 1 year ago - 1 comment
Labels: feature request, user_tools

#82 - [FEA] Reduce speedup factors for complex string expressions (regexp/split)

Issue - State: closed - Opened by mattahrens over 1 year ago - 2 comments
Labels: feature request, wontfix, tools

#73 - [FEA] Possible improvements to Qualification tool

Issue - State: closed - Opened by tgravescs over 1 year ago - 1 comment
Labels: feature request, tools

#72 - [FEA] Update estimated speedup values in user tools qualification CLI output to append x character

Issue - State: closed - Opened by mattahrens over 1 year ago
Labels: feature request, user_tools

#66 - [BUG] Need to update equiJoinSupportedTypes after changes to full join

Issue - State: closed - Opened by jbrennan333 over 1 year ago - 1 comment
Labels: bug, tools

#62 - [FEA] Qualification tool include event log path in csv output

Issue - State: closed - Opened by tgravescs over 1 year ago - 1 comment
Labels: good first issue, tools

#60 - [FEA] Qualification tool - Allow disabling generation of exec and stages csv files

Issue - State: open - Opened by tgravescs over 1 year ago - 1 comment
Labels: good first issue, tools

#44 - [FEA] Verify that wrapper works with Persistent HS

Issue - State: closed - Opened by amahussein almost 2 years ago
Labels: good first issue, user_tools

#43 - [FEA] Allow specifying remote bucket for storing wrapper output

Issue - State: closed - Opened by amahussein almost 2 years ago - 1 comment
Labels: user_tools

#42 - [FEA] user_tools module can create clusters to run tools

Issue - State: closed - Opened by amahussein almost 2 years ago - 1 comment
Labels: user_tools

#37 - Add unit tests for Profiling

Issue - State: closed - Opened by amahussein almost 2 years ago - 1 comment
Labels: user_tools

#36 - Add unit tests for Qualification

Issue - State: closed - Opened by amahussein almost 2 years ago - 1 comment
Labels: user_tools