databricks/spark-sql-perf issues and pull requests

#218 - add tpcds-v4.0 support

Pull Request - State: open - Opened by heyujiao99 4 months ago

#217 - Wrong URL for SBT

Issue - State: open - Opened by elazarl about 1 year ago

#216 - does it has plan to support tpcds 3.2

Issue - State: open - Opened by yixi-gu over 1 year ago

#215 - genData，the tpchdata always stored in the dbgen directory.

Issue - State: open - Opened by ruclz almost 3 years ago

#214 - genData, the data isn`t stored the location I set.

Issue - State: closed - Opened by ruclz almost 3 years ago - 1 comment

#213 - download sbt-launch-lib from github instead of dl.binary

Pull Request - State: open - Opened by Kikyou1997 over 3 years ago

#212 - bump Spark version "3.0.0" -> "3.2.0"

Pull Request - State: open - Opened by satyakommula96 over 3 years ago

#211 - command "build/sbt .." failed with unresolved dependency

Issue - State: open - Opened by Fourth-fresh-man over 3 years ago - 1 comment

#210 - Removing unnecessary joins

Pull Request - State: open - Opened by abhisheksunny over 3 years ago

#209 - Compilation failed for Spark 3.2.0

Issue - State: open - Opened by abin-tiger over 3 years ago

#208 - Build failed

Issue - State: open - Opened by frankliee almost 4 years ago

#207 - Update maven repository after bintray sunset

Pull Request - State: open - Opened by mprashanthsagar almost 4 years ago

#206 - Update Spark repository for sbt

Pull Request - State: closed - Opened by franklsf95 about 4 years ago - 2 comments

#205 - executor_per_core is fixed to 1 vCores in spark-sql-perf on EMR

Issue - State: open - Opened by Rastogii about 4 years ago

#204 - move from sunset bintray to repos.spark-packages.org

Pull Request - State: open - Opened by mattf about 4 years ago

#203 - sbt package failed with unresolved dependency

Issue - State: open - Opened by haojinIntel about 4 years ago - 5 comments

#202 - NoSuchMethodError on Spark 3.1 in Databricks

Issue - State: open - Opened by mithun1979 over 4 years ago - 1 comment

#201 - Update the TPCDS schema based on the Spark codebase

Pull Request - State: open - Opened by maropu over 4 years ago - 2 comments

#200 - sbt run error with unresolved dependency

Issue - State: open - Opened by Qinghe12 over 4 years ago

#199 - Error when trying to create binary from source code

Issue - State: open - Opened by mostrovoi over 4 years ago

#198 - Use CHAR/VARCHAR types in TPCDSTables

Issue - State: open - Opened by maropu over 4 years ago - 2 comments

#197 - Fix output from dbgen

Pull Request - State: open - Opened by lvaz over 4 years ago

#196 - Add a convenient class to generate TPC-DS data

Pull Request - State: closed - Opened by wangyum over 4 years ago - 3 comments

#195 - Fix the CI cannot install Oracle JDK 8 issue

Pull Request - State: closed - Opened by wangyum over 4 years ago - 3 comments

#194 - Updating to Spark 3.0.1 due to dependency errors during build using Spark 3.0.0

Pull Request - State: open - Opened by alanmazankiewicz over 4 years ago

#193 - Fix prewarming query planning unintentionally

Pull Request - State: open - Opened by ejono over 4 years ago

#192 - Getting error when analyzing the columns

Issue - State: open - Opened by jitheshksn over 4 years ago

#191 - Update for Spark 3.0.0 compatibility

Pull Request - State: closed - Opened by npoggi almost 5 years ago - 8 comments

#190 - Spark 3.0.0 compile error

Issue - State: open - Opened by JamesPodogorski almost 5 years ago

#189 - build errors due to dependencies

Issue - State: open - Opened by mpkmtv about 5 years ago - 2 comments

#188 - Add header while generating csv format data and infer schema and header when creating external table.

Pull Request - State: open - Opened by q2w over 5 years ago

#187 - Spark3 tpcds setup

Pull Request - State: open - Opened by Peach-He over 5 years ago - 1 comment

#186 - For spark-3.0.0, there is no method called org.apache.spark.sql.SQLContext.createExternalTable

Issue - State: closed - Opened by Peach-He over 5 years ago

#185 - The Query and Generate mismatch

Issue - State: open - Opened by william-wang almost 6 years ago

#184 - Validating the correctness of results

Issue - State: open - Opened by yuelimv almost 6 years ago - 2 comments

#183 - suitable exector-memory for spark-sql-perf testing

Issue - State: open - Opened by william-wang almost 6 years ago

#182 - How to put data into external storage?

Issue - State: open - Opened by k82cn almost 6 years ago

#181 - what is the difference between tpcds2_4Queries and tpcds1_4Queries

Issue - State: open - Opened by huijuanl about 6 years ago

#180 - Fix files truncating according to maxRecordPerFile

Pull Request - State: closed - Opened by gcz2022 about 6 years ago - 5 comments

#179 - just try

Pull Request - State: closed - Opened by tohaowu about 6 years ago

#178 - looking for a way to get execution time per operator.

Issue - State: open - Opened by AsmaZgo over 6 years ago

#177 - Enable cluster mode for ml bench

Pull Request - State: open - Opened by silveryfu over 6 years ago

#176 - Updates for scala 2.12 compatibility

Pull Request - State: closed - Opened by LucaCanali over 6 years ago - 3 comments

#175 - TPC-DS table primary key constraint not enforced during data generation

Issue - State: closed - Opened by twdsilva over 6 years ago - 4 comments

#174 - [ML-5437] Build with spark-2.4.0 and resolve build issues

Pull Request - State: closed - Opened by MrBago over 6 years ago - 1 comment

#173 - AnalysisException when calling genData

Issue - State: open - Opened by parsifal-47 almost 7 years ago

#172 - Revert "Update Scala Logging to officially supported one "

Pull Request - State: closed - Opened by npoggi almost 7 years ago - 1 comment

#171 - Revert "Update Scala Logging to officially supported one"

Pull Request - State: closed - Opened by npoggi almost 7 years ago

#170 - wait for all broadcast being cleaned after each benchmark

Pull Request - State: closed - Opened by cloud-fan almost 7 years ago - 3 comments

#169 - Rebase for PR 87: Add -m for custom master, use SBT_HOME if set

Pull Request - State: closed - Opened by npoggi almost 7 years ago

#168 - Bumping version to 0.5.1-SNAPSHOT

Pull Request - State: closed - Opened by npoggi almost 7 years ago - 2 comments

#167 - Fixing TPCH DDL datatype of customer.c_nationkey to long

Pull Request - State: closed - Opened by npoggi almost 7 years ago

#166 - Make TPCDS queryNames public so it can be accessed from notebooks.

Pull Request - State: closed - Opened by npoggi almost 7 years ago - 1 comment

#165 - Fix 3 local benchmark classes

Pull Request - State: closed - Opened by g1thubhub almost 7 years ago - 3 comments

#164 - Fix compile for Spark 2.4 SNAPSHOT and only catch NonFatal

Pull Request - State: closed - Opened by mengxr almost 7 years ago - 3 comments

#163 - Benchmark for SparkR UDF *apply() APIs

Pull Request - State: closed - Opened by morewood about 7 years ago - 2 comments

#162 - [ML-4154] Added testing for before/after of ml benchmarks.

Pull Request - State: closed - Opened by MrBago about 7 years ago - 1 comment

#161 - [ML-4069] Improve timing of estimators

Pull Request - State: closed - Opened by jkbradley about 7 years ago - 4 comments

#160 - Increase mllib-large test timeout to 12 hours

Pull Request - State: closed - Opened by mengxr about 7 years ago

#159 - [ML-2918] Call count() in default score() to improve timing of transform()

Pull Request - State: closed - Opened by jkbradley about 7 years ago - 9 comments

#158 - Update VectorAssembler test such that the dataset size is numExamples * numFeatures

Pull Request - State: closed - Opened by mengxr about 7 years ago - 1 comment

#157 - Update Scala Logging to officially supported one

Pull Request - State: closed - Opened by mrow4a about 7 years ago - 3 comments

#156 - [ML-3844] Add GBTRegression benchmark

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 1 comment

#155 - [ML-3870] Make spark-sql-perf master compiled with spark 2.3 and scala 2.11

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 2 comments

#154 - [ML-3869] Make Quantilediscretizer work with spark-2.3

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 1 comment

#153 - [ML-3915] add additionalTests to MLMetrics

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 1 comment

#152 - [ML-3583] Add benchmarks to mllib-large.yaml for featurization

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 2 comments

#151 - [ML-3824] Add benchmarks to mllib-large.yaml for FPGrowth

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 3 comments

#150 - [ML-3581] Add benchmarks to mllib-large.yaml for regression

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 2 comments

#149 - [ML-3585] Added benchmarks to mllib-large.yaml for clustering

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 1 comment

#148 - Output Training Time as metrics

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago

#147 - [ML-3584] Added benchmarks to mllib-large.yaml for ALS

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago - 1 comment

#146 - [ML-3775] Add "benchmarkId" to BenchmarkResult

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago

#145 - [ML-3753] Log "value" instead of "Some(value)" for ML params in results

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago

#144 - [ML-3749] Log metric name and isLargerBetter in BenchmarkResult

Pull Request - State: closed - Opened by lu-wang-dl about 7 years ago

#143 - Added benchmarks to mllib-large.yaml for classifcation Estimators.

Pull Request - State: closed - Opened by MrBago about 7 years ago - 3 comments

#142 - Fix bug with ML additional method tests

Pull Request - State: closed - Opened by jkbradley about 7 years ago - 1 comment

#141 - Run mllib small in unit tests

Pull Request - State: closed - Opened by MrBago about 7 years ago

#140 - Add decision tree benchmark

Pull Request - State: closed - Opened by MrBago about 7 years ago - 1 comment

#139 - Additional method test for some ML algos

Pull Request - State: closed - Opened by WeichenXu123 over 7 years ago - 2 comments

#138 - Coalesce(n) instead of hardcoded (1) for large tables/partitions

Pull Request - State: closed - Opened by npoggi over 7 years ago - 1 comment

#137 - Adds an optional version of the SS_MAX query

Pull Request - State: closed - Opened by npoggi over 7 years ago

#136 - TPC-H datagenerator and instructions

Pull Request - State: closed - Opened by npoggi over 7 years ago

#135 - Quantile discretizer benchmark

Pull Request - State: closed - Opened by WeichenXu123 over 7 years ago - 8 comments

#134 - Standalone TPCDS tester

Pull Request - State: closed - Opened by ileshko over 7 years ago - 5 comments

#133 - Run TPC-H

Issue - State: closed - Opened by hahasdnu1029 over 7 years ago - 3 comments

#132 - OneHotEncoderEstimator benchmark

Pull Request - State: open - Opened by WeichenXu123 over 7 years ago - 1 comment

#131 - Cannot create tables in cluster mode - Unable to infer schema for Parquet

Issue - State: closed - Opened by Panos-Bletsos over 7 years ago - 2 comments

#130 - Use DECIMAL and DATE in the default TPCDS notebooks.

Pull Request - State: closed - Opened by juliuszsompolski over 7 years ago - 2 comments

#127 - Word2Vec benchmark

Pull Request - State: closed - Opened by WeichenXu123 over 7 years ago - 7 comments

#126 - dataGen error

Issue - State: closed - Opened by hahasdnu1029 over 7 years ago - 14 comments

#125 - [ML-3342] Bug fixes to make mllib benchmarks work with dbr-4.0.

Pull Request - State: closed - Opened by MrBago over 7 years ago - 3 comments

#124 - establish bds-hive-2.1 branch

Pull Request - State: closed - Opened by ziff-verticloud over 7 years ago - 2 comments

#122 - run full gc on the execturos after each run

Pull Request - State: closed - Opened by liufengdb almost 8 years ago - 1 comment

#102 - "bin/run --benchmark DatasetPerformance" shows too many errors.

Issue - State: closed - Opened by ghost about 8 years ago - 1 comment

#100 - Sharing TPC-DS test results of HAWQ & SparkSQL

Issue - State: open - Opened by ktania over 8 years ago - 1 comment

#99 - setup the benchmark

Issue - State: open - Opened by idragus over 8 years ago - 3 comments

#97 - failed to compile spark-sql-perf for spark 2.1.0

Issue - State: open - Opened by ccwgit over 8 years ago - 4 comments

#96 - Show std dev as a percentage, show all queries

Pull Request - State: closed - Opened by a-roberts over 8 years ago - 2 comments

#95 - java.lang.RuntimeException: [1.1] failure: ``with'' expected but identifier CREATE found

Issue - State: open - Opened by wangli86 over 8 years ago - 4 comments

GitHub / databricks/spark-sql-perf issues and pull requests