NVIDIA/spark-rapids-tools issues and pull requests

#1335 - Add support to MaxBy and MinBy in Qualification tool

Pull Request - State: open - Opened by amahussein 26 days ago
Labels: feature request, tools

#1334 - [FEA] Qual tool should recommend spark.executor.cores based on best TCO value from internal benchmark

Issue - State: open - Opened by viadea 26 days ago
Labels: feature request, ? - Needs Triage

#1333 - [DOC] Example code and description of qualx plugin framework

Issue - State: open - Opened by wjxiz1992 28 days ago
Labels: documentation, user_tools

#1332 - Update signoff usage [skip ci]

Pull Request - State: closed - Opened by pxLi 28 days ago
Labels: cicd

#1331 - Updated models for EMR NDS-H dataset

Pull Request - State: closed - Opened by leewyang 28 days ago
Labels: user_tools

#1330 - Follow-up 1318: Fix QualX fallback with default speedup and duration columns

Pull Request - State: closed - Opened by parthosa 28 days ago - 1 comment
Labels: bug, user_tools

#1329 - [FEA] Add support to MaxBy and MinBy in Qualification tool

Issue - State: open - Opened by amahussein 29 days ago
Labels: feature request, tools

#1328 - Sync up DAYTIME and YEARMONTH fields with CSV plugin files

Pull Request - State: closed - Opened by amahussein 29 days ago
Labels: tools

#1327 - [BUG] Sync up DAYTIME and YEARMONTH fields with CSV plugin files

Issue - State: closed - Opened by amahussein 29 days ago
Labels: bug, tools

#1326 - Allow spark dependency to be configured dynamically

Pull Request - State: open - Opened by amahussein 29 days ago
Labels: feature request, user_tools

#1325 - [FEA] Qualification tool: Report supported expressions in the output file

Issue - State: open - Opened by nartal1 about 1 month ago
Labels: feature request, tools, affect-output

#1324 - Add safeguards to prevent older attempts from generating metrics output in Scala Tool

Pull Request - State: closed - Opened by parthosa about 1 month ago
Labels: bug, tools

#1323 - Mark decimalsum as supported in Qualification tool

Pull Request - State: closed - Opened by amahussein about 1 month ago - 2 comments
Labels: feature request, tools

#1322 - [FEA] Mark decimalsum as supported in Qualification tool

Issue - State: closed - Opened by amahussein about 1 month ago
Labels: feature request, tools

#1321 - [DOC] qualx related tool lacks of the doc to list required dependencies

Issue - State: open - Opened by wjxiz1992 about 1 month ago - 1 comment
Labels: documentation, user_tools

#1320 - [FEA] Add total core seconds in Qualification core tool output

Pull Request - State: open - Opened by cindyyuanjiang about 1 month ago - 2 comments
Labels: tools, affect-output

#1319 - [BUG] Eventlogs with application re-runs can add duplicate entries to `qualification_summary.csv`

Issue - State: closed - Opened by kuhushukla about 1 month ago - 3 comments
Labels: bug, user_tools, tools

#1318 - Remove legacy SpeedupFactor from core output files

Pull Request - State: closed - Opened by amahussein about 1 month ago - 3 comments
Labels: tools, affect-output

#1317 - [DOC] spark_rapids CLI help cmd still shows cost savings

Pull Request - State: closed - Opened by cindyyuanjiang about 1 month ago
Labels: documentation, user_tools

#1316 - [BUG] Event parsing error: String length (...) exceeds the maximum length (20000000)

Issue - State: open - Opened by tgravescs about 1 month ago - 3 comments
Labels: bug, user_tools

#1315 - [BUG] Qualification Tool does not mark array_join as unsupported

Issue - State: open - Opened by amahussein about 1 month ago - 1 comment
Labels: bug, tools

#1314 - [BUG] Change autotuner memory to small error to a warning

Issue - State: open - Opened by tgravescs about 1 month ago
Labels: bug, tools

#1313 - Add end-to-end behavioural tests for the python CLI

Pull Request - State: open - Opened by parthosa about 1 month ago - 3 comments
Labels: feature request, user_tools

#1312 - Fix Qualification and Profiling tools CLI argument shorthands

Pull Request - State: closed - Opened by cindyyuanjiang about 1 month ago
Labels: bug, user_tools

#1311 - Remove arguments and code related to the html-report

Pull Request - State: closed - Opened by amahussein about 1 month ago
Labels: tools, build, dependencies

#1310 - Remove arguments and code related to the html-report

Issue - State: closed - Opened by amahussein about 1 month ago
Labels: tools, build

#1309 - Mark SMJ as unsupported operator for corner cases in left join

Pull Request - State: closed - Opened by nartal1 about 1 month ago
Labels: tools

#1308 - Append HADOOP_CONF_DIR to the tools CLASSPATH execution cmd

Pull Request - State: closed - Opened by amahussein about 1 month ago - 1 comment
Labels: bug, user_tools

#1307 - [FEA] Consider core-seconds in TCL heuristics

Issue - State: open - Opened by cindyyuanjiang about 1 month ago - 1 comment
Labels: feature request, user_tools

#1306 - [BUG] `rapids_4_spark_qualification_output_status.csv` file does not report apps with zero SQL time

Issue - State: open - Opened by kuhushukla about 1 month ago - 1 comment
Labels: bug, tools

#1305 - [BUG] qualification_summary.csv is absent when an app is processed but does not fit qualification criteria

Issue - State: closed - Opened by kuhushukla about 1 month ago - 2 comments
Labels: bug, invalid, user_tools

#1304 - [FEA] HDFS Support in Tools

Issue - State: open - Opened by parthosa about 1 month ago
Labels: feature request, user_tools

#1303 - [BUG] python --output_folder doesn't have what filesystems are supported

Issue - State: closed - Opened by tgravescs about 1 month ago - 1 comment
Labels: bug, user_tools

#1302 - [BUG] user_tools fails to pick up hadoop_conf_dir when reading from hdfs

Issue - State: closed - Opened by kuhushukla about 1 month ago - 1 comment
Labels: bug, user_tools

#1301 - [BUG] spark_rapids CLI fails when using `-e` as shorthand for `--eventlogs`

Issue - State: closed - Opened by cindyyuanjiang about 1 month ago - 1 comment
Labels: bug, user_tools

#1300 - Raise error for enum creation from invalid string values

Pull Request - State: closed - Opened by parthosa about 1 month ago
Labels: bug, user_tools

#1299 - Qualification tool support filtering by a filesystem time range

Pull Request - State: closed - Opened by tgravescs about 1 month ago
Labels: feature request, tools

#1298 - Fix key error and cross-join error during qualx evaluate

Pull Request - State: closed - Opened by leewyang about 1 month ago
Labels: bug, user_tools

#1297 - Enable recursive search for event logs by default and optional `--no-recursion` flag

Pull Request - State: closed - Opened by parthosa about 2 months ago - 2 comments
Labels: feature request, tools

#1296 - [FEA] Improve hardcoded SparkRapidsBuildInfoEvent in tools

Issue - State: open - Opened by cindyyuanjiang about 2 months ago
Labels: feature request, tools

#1295 - [FEA] Investigate generate-timeline for incomplete eventlogs

Issue - State: open - Opened by nartal1 about 2 months ago
Labels: feature request, tools

#1294 - [FEA] Clean up console output: Contents of `raw_metrics` and `tuning` dirs should not be displayed

Issue - State: open - Opened by parthosa about 2 months ago - 1 comment
Labels: feature request, user_tools

#1293 - [FEA] Qualification tool support filtering by a time range

Issue - State: closed - Opened by tgravescs about 2 months ago
Labels: feature request, tools

#1292 - Qual tool: Print more useful log messages when failures happen downloading dependencies

Pull Request - State: closed - Opened by tgravescs about 2 months ago
Labels: bug, user_tools

#1291 - Qualification tool - Add option to filter by minimum event log size

Pull Request - State: closed - Opened by tgravescs about 2 months ago - 2 comments
Labels: feature request, tools

#1290 - Skip generating timeline for stages that do not have completion time

Pull Request - State: closed - Opened by nartal1 about 2 months ago - 4 comments
Labels: bug, tools

#1289 - Remove restricted google sheets link and outdated TCO section

Pull Request - State: closed - Opened by parthosa about 2 months ago
Labels: bug, documentation

#1288 - [BUG] Google sheet link in markdown requires authentication to read

Issue - State: closed - Opened by parthosa about 2 months ago
Labels: bug, documentation

#1287 - [BUG] Format Report Summary in STDOUT to include count of failed/unsupported apps

Issue - State: open - Opened by parthosa about 2 months ago - 3 comments
Labels: bug, user_tools, usability

#1286 - [BUG] Improve error report during downloads or http connections going out of the tools package

Issue - State: open - Opened by kuhushukla about 2 months ago - 1 comment
Labels: user_tools, ? - Needs Triage, usability

#1285 - Fix --help text for custom_model_file option

Pull Request - State: closed - Opened by kuhushukla about 2 months ago
Labels: bug, user_tools

#1284 - [BUG] Fix --help text for custom_model_file option

Issue - State: closed - Opened by kuhushukla about 2 months ago
Labels: usability

#1283 - [BUG] Error when platform is unspecified as onprem with hdfs input paths

Issue - State: closed - Opened by kuhushukla about 2 months ago
Labels: bug, user_tools

#1282 - [FEA] Remove log file output from qualification tool output

Issue - State: open - Opened by nartal1 about 2 months ago
Labels: feature request, tools

#1281 - Include exception message for unknown app status in core tool

Pull Request - State: closed - Opened by kuhushukla about 2 months ago - 6 comments
Labels: tools, usability

#1280 - [BUG] Improve log message when qual tool could not access an eventlog

Issue - State: closed - Opened by kuhushukla about 2 months ago - 2 comments
Labels: usability

#1279 - Remove unused argument `--target_platform` in Python Tool

Pull Request - State: closed - Opened by parthosa about 2 months ago
Labels: bug, user_tools

#1278 - Remove calculation of gpu cluster recommendation from python tool when cluster argument is passed

Pull Request - State: closed - Opened by parthosa about 2 months ago
Labels: bug, user_tools

#1277 - [BUG] Unused argument `--target_platform` should be removed from python tool

Issue - State: closed - Opened by parthosa about 2 months ago
Labels: bug

#1276 - [FEA] Extend max-event-log-size log filtering

Issue - State: open - Opened by tgravescs about 2 months ago - 1 comment
Labels: feature request, tools

#1275 - Qualification tool: Add option to filter event logs for a maximum file system size

Pull Request - State: closed - Opened by tgravescs about 2 months ago - 1 comment
Labels: feature request, tools

#1274 - [FEA] AutoTuner on GPU event log should verify spark.task.resource.gpu.amount setting

Issue - State: open - Opened by tgravescs about 2 months ago
Labels: feature request, tools

#1273 - [BUG] Incorrect cluster recommendation due to legacy gpu cluster creation in python tool

Issue - State: closed - Opened by parthosa about 2 months ago
Labels: bug, user_tools

#1272 - [TASK] Optimize the storage used by SQLAccumProfileResults

Issue - State: open - Opened by amahussein about 2 months ago
Labels: tools

#1271 - [BUG] SPARK-34388 may affect Rapids qualification tool's ability of finding UDF

Issue - State: open - Opened by YuzhouSun over 2 years ago - 7 comments
Labels: bug, tools

#1270 - [BUG] Onprem user qualification tool throws FileNotFoundError

Issue - State: closed - Opened by cindyyuanjiang about 2 months ago - 3 comments
Labels: bug, user_tools

#1269 - Save core tools logs to output log file

Pull Request - State: closed - Opened by nartal1 about 2 months ago - 4 comments
Labels: feature request, tools, usability

#1268 - Remove speedup based recommendation column from qual_summary csv

Pull Request - State: closed - Opened by amahussein about 2 months ago - 3 comments
Labels: user_tools, affect-output

#1267 - Fix prediction CSV files for multiple qual directories

Pull Request - State: closed - Opened by leewyang about 2 months ago - 1 comment
Labels: bug, user_tools

#1266 - Sync GetJsonObject support with Rapids-Plugin

Pull Request - State: closed - Opened by amahussein about 2 months ago
Labels: feature request, tools

#1265 - Include GPU information in the cluster recommendation for Dataproc and OnPrem

Pull Request - State: closed - Opened by parthosa about 2 months ago - 2 comments
Labels: bug, user_tools, tools

#1264 - [BUG] Scalable solution for output files location in the console output

Issue - State: open - Opened by parthosa about 2 months ago - 2 comments
Labels: bug, user_tools

#1263 - [TASK] Optimize the storage of accumulables in core tools

Pull Request - State: closed - Opened by amahussein about 2 months ago
Labels: tools

#1262 - [FEA] Switch get_json_object as supported in qualification tool

Issue - State: closed - Opened by mattahrens about 2 months ago
Labels: feature request, tools

#1261 - Do not create new StageInfo object

Pull Request - State: closed - Opened by nartal1 about 2 months ago
Labels: bug, tools

#1260 - [BUG] Exception running on Spark 3.1.x and 3.2.x due to different constructors of StageInfo

Issue - State: open - Opened by amahussein about 2 months ago - 1 comment
Labels: bug, tools

#1259 - [FEA] Add a dry-run or list-event-logs-info type option to qualification tool

Issue - State: open - Opened by tgravescs about 2 months ago
Labels: feature request, tools

#1258 - Rename cluster shape columns to use 'worker' prefix in the output files and rename metadata file

Pull Request - State: closed - Opened by parthosa about 2 months ago
Labels: bug, user_tools, tools

#1257 - [BUG] Filter-out CSP-specific Spark configurations generated by the AutoTuner

Issue - State: open - Opened by amahussein about 2 months ago
Labels: bug, tools

#1256 - Clean up tools after removing CLI dependency

Pull Request - State: closed - Opened by cindyyuanjiang 2 months ago
Labels: user_tools

#1255 - [BUG] Cluster recommendation for executors with varying number of cores

Issue - State: open - Opened by parthosa 2 months ago - 1 comment
Labels: bug, tools

#1254 - [BUG] Improve node type recommendation for instances exceeding max core count

Issue - State: open - Opened by parthosa 2 months ago
Labels: bug, tools

#1253 - [BUG] Namenode URI should not be required when processing HDFS event logs

Issue - State: closed - Opened by parthosa 2 months ago - 2 comments
Labels: bug, user_tools, tools

#1252 - Replace split_nds with split_train_val

Pull Request - State: closed - Opened by leewyang 2 months ago - 1 comment
Labels: user_tools

#1251 - Fix stage level metrics output csv file

Pull Request - State: closed - Opened by nartal1 2 months ago - 2 comments
Labels: bug, tools

#1250 - [BUG] Stage level metrics output result is incorrect

Issue - State: closed - Opened by nartal1 2 months ago
Labels: bug, tools

#1249 - [FEA] Processing of Large Scale Event Logs

Issue - State: open - Opened by parthosa 2 months ago - 2 comments
Labels: feature request, tools

#1248 - [FEA] Add support for `map_from_arrays` in qualification tools

Pull Request - State: closed - Opened by cindyyuanjiang 2 months ago
Labels: feature request, tools

#1247 - [BUG] Tools have some duplicate rows in data_source_information CSV file

Issue - State: open - Opened by amahussein 2 months ago
Labels: bug, tools

#1246 - [BUG] Cluster argument in EMR should support <cluster-id>

Issue - State: open - Opened by parthosa 2 months ago
Labels: bug, user_tools

#1245 - Remove CLI dependency in Dataproc `_pull_gpu_hw_info` implementation

Pull Request - State: closed - Opened by cindyyuanjiang 2 months ago
Labels: feature request, user_tools, usability

#1244 - Update xgboost models and metrics

Pull Request - State: closed - Opened by leewyang 2 months ago
Labels: user_tools

#1243 - Add footnotes for config recommendations and speedup category in top candidate view

Pull Request - State: closed - Opened by parthosa 2 months ago - 2 comments
Labels: bug, user_tools

#1242 - [BUG] Update Dataproc instance catalog for n1 series GPU info

Pull Request - State: closed - Opened by cindyyuanjiang 2 months ago - 3 comments
Labels: bug, user_tools

#1241 - Improvements in Cluster Config Recommender

Pull Request - State: closed - Opened by parthosa 2 months ago - 2 comments
Labels: bug, user_tools

#1240 - [BUG] Top level candidate table * and ** notes should be directly under the table.

Issue - State: closed - Opened by tgravescs 2 months ago
Labels: bug, user_tools

#1239 - [BUG] Review and update recommended Cluster info in metadata json file

Issue - State: closed - Opened by tgravescs 2 months ago - 8 comments
Labels: bug, tools

#1238 - [BUG] All Dataproc n1 series instances support GPUs

Issue - State: closed - Opened by cindyyuanjiang 2 months ago
Labels: bug, user_tools

#1237 - Handle event logs with wildcards in status report generation

Pull Request - State: closed - Opened by parthosa 2 months ago
Labels: bug, tools

#1236 - [BUG] Handle event logs with wildcards in status report generation

Issue - State: closed - Opened by parthosa 2 months ago
Labels: bug, tools

GitHub / NVIDIA/spark-rapids-tools issues and pull requests