memverge/splash issues and pull requests

#69 - Bump org.apache.hadoop:hadoop-common from 2.7.4 to 3.4.0

Pull Request - State: open - Opened by dependabot[bot] 6 days ago
Labels: dependencies

#68 - Bump hadoop-common from 2.7.4 to 3.2.3

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 1 comment
Labels: dependencies

#67 - Bump hadoop-common from 2.7.4 to 2.10.1

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 1 comment
Labels: dependencies

#66 - java.lang.ClassNotFoundException: org.apache.spark.shuffle.SplashShuffleManager

Issue - State: open - Opened by Roy-Rupak over 3 years ago - 4 comments

#65 - Bump commons-io from 2.4 to 2.7

Pull Request - State: open - Opened by dependabot[bot] over 3 years ago - 1 comment
Labels: dependencies

#64 - Add support for HDFS compliant file systems

Issue - State: open - Opened by drnushooz about 4 years ago

#63 - [GH-64] Add support for HDFS compliant file systems

Pull Request - State: closed - Opened by drnushooz about 4 years ago - 2 comments

#62 - java.lang.IllegalArgumentException because of spark version string being blank

Issue - State: closed - Opened by drnushooz about 4 years ago - 1 comment

#61 - [GH-60] Support fast merge in storage plugin iface.

Pull Request - State: closed - Opened by jealous almost 5 years ago - 1 comment
Labels: enhancement

#60 - Support fast merge in the storage plugin interface.

Issue - State: open - Opened by jealous almost 5 years ago
Labels: enhancement

#59 - java.lang.IllegalAccessError when running with Spark 2.4.0

Issue - State: open - Opened by amorskoy about 5 years ago - 4 comments

#58 - [GH-57] Shuffle tool output is not precise

Pull Request - State: closed - Opened by jealous about 5 years ago
Labels: bug

#57 - The statistics output of the shuffle tool is not precise enough

Issue - State: open - Opened by jealous about 5 years ago
Labels: bug

#56 - [GH-55] Improve the unsafe code path performance.

Pull Request - State: closed - Opened by jealous about 5 years ago - 1 comment
Labels: enhancement

#55 - Improve the performance and memory efficiency in the unsafe code path

Issue - State: closed - Opened by jealous about 5 years ago
Labels: enhancement

#54 - [GH-53] Batch deploy script.

Pull Request - State: closed - Opened by jealous about 5 years ago - 1 comment
Labels: enhancement

#53 - Add deploy script for different Spark & Hadoop versions

Issue - State: closed - Opened by jealous about 5 years ago
Labels: enhancement

#52 - [GH-51] Cache shuffle index in memory.

Pull Request - State: closed - Opened by jealous about 5 years ago - 1 comment
Labels: enhancement

#51 - Cache shuffle index in memory

Issue - State: closed - Opened by jealous about 5 years ago
Labels: enhancement

#50 - [GH-49] Shuffle performance tool.

Pull Request - State: closed - Opened by jealous about 5 years ago - 2 comments
Labels: enhancement

#49 - Create a shuffle tool for performance test

Issue - State: closed - Opened by jealous about 5 years ago
Labels: enhancement

#48 - [GH-47] Improve merge sort efficiency.

Pull Request - State: closed - Opened by jealous over 5 years ago - 2 comments
Labels: bug, enhancement

#47 - Improve the merge sort performance of the external sorter.

Issue - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#46 - [GH-45] Stop shuffle manager raises exception

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#45 - Stop the shuffle manager when the app hasn't started raise an exception

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#44 - [GH-43] Structure enhancement of the shuffle reader

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#43 - Structure enhancement of the shuffle reader

Issue - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#42 - [GH-41] Create empty shuffle data file failed

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#41 - Create empty shuffle data file failed when the file already exists.

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#40 - [GH-39] Do not use `OutputStream.available()`.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#39 - Do not rely on `OutputStream.available()`

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#38 - [GH-37] Recuce the number of calls to getSize.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#37 - Reduce the number of calls to ShuffleFile.getSize

Issue - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#36 - [GH-35] Print version during shuffle manager init.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#35 - Print version info during shuffle manager initialization

Issue - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#34 - Remove local storage plugin.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#33 - Fix compatibility issue for Spark 2.4.0.

Pull Request - State: closed - Opened by jealous over 5 years ago - 2 comments
Labels: bug

#32 - Change shuffleBlocks from Seq to Iterator in SplashShuffleReader.

Pull Request - State: closed - Opened by weixumv over 5 years ago - 1 comment

#31 - Splash doesn't compile with Spark 2.4.0

Issue - State: closed - Opened by weixumv over 5 years ago - 3 comments

#30 - Put shuffle files in separate folders.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#29 - Enhance hash based shuffle (bypassMergeSort).

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#28 - Write spill instead of partition data file

Pull Request - State: closed - Opened by jealous over 5 years ago - 2 comments
Labels: bug

#27 - Remove the file exists check.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#26 - How to delete the shuffle data

Issue - State: closed - Opened by thousandhu over 5 years ago - 5 comments
Labels: question

#25 - [GH-24] Dump partition when read error happens

Pull Request - State: closed - Opened by jealous over 5 years ago - 2 comments
Labels: enhancement

#24 - Dump shuffle partition when error happens during shuffle read

Issue - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#23 - [GH-22] Adjust the location of `onApplicationStart`.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#22 - SparkConf is not available during `onApplicationStart`

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#21 - [GH-20] Deal with empty file in merge.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#20 - File not found during merge in bypass merge sort code path

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#19 - filter non-existing files before merging them into one file

Pull Request - State: closed - Opened by sheperdh over 5 years ago - 1 comment

#18 - [GH-17] Debug log partition md5 in shuffle write

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#17 - Add MD5 for each partition when we write

Issue - State: closed - Opened by sheperdh over 5 years ago
Labels: enhancement

#16 - Test reports warning log related to memory pool.

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#15 - Enhance the REAME document.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#14 - [GH-13] Compatible changes for Spark 2.1.0.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: bug

#13 - Build failed for Spark 2.1.0.

Issue - State: closed - Opened by jealous over 5 years ago
Labels: bug

#12 - Integrate continuous integration tools on Cloud.

Pull Request - State: closed - Opened by jealous over 5 years ago - 1 comment
Labels: enhancement

#11 - Typo/language/grammar

Pull Request - State: closed - Opened by RonNiles over 5 years ago

#10 - Add more log for partition read debug

Pull Request - State: closed - Opened by jealous over 5 years ago
Labels: enhancement

#9 - Index/Data file not found error.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: bug

#8 - Do not create file until write.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: bug

#7 - [GH-6] Do not clean shuffle data on executor failure.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: bug

#6 - Dmo crash due to application folder deletion.

Issue - State: closed - Opened by wanghy73 almost 6 years ago - 1 comment
Labels: bug

#5 - Create and close stream even if it's empty.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: bug

#4 - Add design document for splash.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: enhancement

#3 - Add bypass merge sort code path support.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: enhancement

#2 - Implement the local plugin.

Pull Request - State: closed - Opened by jealous almost 6 years ago
Labels: enhancement

#1 - Feature/interface

Pull Request - State: closed - Opened by czoey almost 6 years ago

GitHub / memverge/splash issues and pull requests