G-Research/spark-extension issues and pull requests

#158 - [CI] upgrade spark versions

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#157 - Test publish action from fork

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#156 - Bump scalatest_2.13 from 3.2.14 to 3.3.0-SNAP4

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 6 comments
Labels: dependencies, java

#155 - Is there a way to handle comparison for Map<T, Struct>?

Issue - State: closed - Opened by asolimando over 1 year ago - 5 comments

#154 - Add reader for parquet metadata

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#153 - Add .Net DateTime.Ticks conversion functions

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#152 - Add whitespace comparator to Python API

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#151 - Build python package

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#150 - Bump maven-surefire-report-plugin from 3.0.0-M9 to 3.0.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 3 comments
Labels: dependencies, java

#149 - Bump maven-surefire-plugin from 3.0.0-M9 to 3.0.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 3 comments
Labels: dependencies, java

#148 - Add Spark 3.4.0 to CI

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#147 - Test Spark 3.3.2, 3.3.3-SNAPSHOT and 3.4.1-SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#146 - Add example source reporting partition and order

Pull Request - State: open - Opened by EnricoMi over 1 year ago - 1 comment

#145 - Bump scala-maven-plugin from 4.8.0 to 4.8.1

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies, java

#144 - Add final test result step to CI

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#143 - Upgrade Spark to 3.3.2 in CI

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#142 - Replace getSparkVersion with spark.version

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#141 - Link to partitioning blog

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#140 - Bump maven-surefire-report-plugin from 3.0.0-M8 to 3.0.0-M9

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies, java

#139 - Bump maven-surefire-plugin from 3.0.0-M8 to 3.0.0-M9

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies, java

#138 - Type the equiv comparator for its natural datatype

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#137 - Add whitespace agnostic diff comparator

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#136 - Note on null-aware diff equiv

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#135 - Test with Spark RC

Pull Request - State: open - Opened by EnricoMi over 1 year ago - 1 comment

#134 - Bump maven-surefire-report-plugin from 3.0.0-M7 to 3.0.0-M8

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#133 - Bump maven-surefire-plugin from 3.0.0-M7 to 3.0.0-M8

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#132 - Fix Python API calling into Scala code

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#131 - Test built package against all compatible Spark versions

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#130 - Add fix versions for partitioned write issue

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#129 - Split JVM and Python tests, test more Python versions

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#128 - Bump scala-maven-plugin from 4.7.2 to 4.8.0

Pull Request - State: closed - Opened by dependabot[bot] almost 2 years ago - 1 comment
Labels: dependencies, java

#127 - Allow for custom diff equality

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 4 comments

#126 - Fix missing python-version for testing

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#125 - Upgrade dependencies

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago

#124 - Cache before partitioned-write, provide unpersist handle

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#123 - Move UnpersistHandle into own file under same package path

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#122 - More snapshots in CI

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#121 - Test more snapshots in CI

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#120 - Adjust asserts for Spark 3.4.0 snapshot

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#119 - Move Java tests outside spark package

Pull Request - State: closed - Opened by EnricoMi almost 2 years ago - 1 comment

#118 - Bump scalatest-maven-plugin from 2.0.2 to 2.2.0

Pull Request - State: closed - Opened by dependabot[bot] almost 2 years ago - 1 comment
Labels: dependencies, java

#117 - NoSuchMethodError: scala.Product.$init$

Issue - State: closed - Opened by markmcwiggins almost 2 years ago - 3 comments

#116 - Get spark-extension work in Python dev environment

Issue - State: closed - Opened by EnricoMi almost 2 years ago
Labels: documentation, enhancement

#115 - the test-release.sh script is out of sync with repos from 2020

Issue - State: closed - Opened by markmcwiggins almost 2 years ago - 10 comments

#114 - Python test-release.py isn't working

Issue - State: closed - Opened by markmcwiggins almost 2 years ago - 6 comments

#113 - Bump scala-maven-plugin from 4.7.0 to 4.7.2

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 2 comments
Labels: dependencies, java

#112 - Adjust asserts for Spark 3.4.0 snapshot

Pull Request - State: closed - Opened by EnricoMi about 2 years ago - 1 comment

#111 - Diff similar DataFrames with ignoreColumns

Pull Request - State: closed - Opened by EnricoMi about 2 years ago - 2 comments

#110 - Bump pyspark from 3.2.0 to 3.2.2 in /python

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 2 comments
Labels: dependencies, python

#109 - Add diffwith to Scala, Java and Python Diff API

Pull Request - State: closed - Opened by EnricoMi about 2 years ago - 1 comment

#108 - Allow for extra columns on either side if they are ignored

Issue - State: closed - Opened by EnricoMi about 2 years ago

#107 - Replace usage of deprecated DataFrame.sql_ctx with sparkSession

Pull Request - State: closed - Opened by EnricoMi about 2 years ago - 1 comment

#106 - No matter what i do still "ModuleNotFoundError: No module named 'uk'"

Issue - State: closed - Opened by Mimezzz about 2 years ago - 8 comments

#105 - Diff Failure on AWS Glue

Issue - State: closed - Opened by bobhaffner about 2 years ago - 7 comments

#104 - Bump scala-maven-plugin from 4.7.0 to 4.7.1

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 2 comments
Labels: dependencies, java

#103 - How to use with AWS Glue?

Issue - State: closed - Opened by kasunnirmala about 2 years ago - 9 comments
Labels: question

#102 - Bump scala-maven-plugin from 4.6.3 to 4.7.0

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 3 comments
Labels: dependencies, java

#101 - Rename CI jobs and upgrade dependencies

Pull Request - State: closed - Opened by EnricoMi about 2 years ago - 1 comment

#100 - Bump scalatest-maven-plugin from 2.0.2 to 2.1.0-SNAP2

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 2 comments
Labels: dependencies, java

#99 - Bump scalatest-maven-plugin from 2.0.2 to 2.1.0-SNAP1

Pull Request - State: closed - Opened by dependabot[bot] about 2 years ago - 2 comments
Labels: dependencies, java

#98 - Bump actions/setup-python from 3 to 4

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, github_actions

#97 - Add withRowNumbers transformation to Scala, Java and Python API

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 2 comments

#96 - Bump scala-maven-plugin from 4.6.1 to 4.6.3

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, java

#95 - PySpark - how return only changed rows

Issue - State: closed - Opened by flxshk over 2 years ago - 1 comment

#94 - Bump maven-surefire-report-plugin from 3.0.0-M6 to 3.0.0-M7

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 3 comments
Labels: dependencies, java

#93 - Bump maven-surefire-plugin from 3.0.0-M6 to 3.0.0-M7

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 3 comments
Labels: dependencies, java

#92 - Bump scala-maven-plugin from 4.6.1 to 4.6.2

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, java

#91 - Bump actions/setup-java from 2 to 3

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, github_actions

#90 - Bump actions/download-artifact from 2 to 3

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, github_actions

#89 - Bump actions/upload-artifact from 2 to 3

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, github_actions

#88 - Bump nexus-staging-maven-plugin from 1.6.12 to 1.6.13

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 3 comments
Labels: dependencies, java

#87 - Cleaning up CHANGELOG.md

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 1 comment

#86 - Bump maven-surefire-report-plugin from 3.0.0-M5 to 3.0.0-M6

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, java

#85 - Bump maven-surefire-plugin from 3.0.0-M5 to 3.0.0-M6

Pull Request - State: closed - Opened by dependabot[bot] over 2 years ago - 2 comments
Labels: dependencies, java

#84 - Test against Spark snapshot versions

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 1 comment

#83 - Bump scalatest version to 3.2.11

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 1 comment

#78 - Allow for custom ordering-based diff equality

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 1 comment

#76 - Add sorted group methods to Dataset

Pull Request - State: closed - Opened by EnricoMi over 2 years ago - 1 comment

#64 - On AWS - after Diff, Insert columns are all null

Issue - State: closed - Opened by leewalter78 about 3 years ago - 10 comments

#59 - Comparisons between DFs with MapFields fail comparison

Issue - State: closed - Opened by rimerj about 3 years ago - 2 comments

#31 - Can we use this library in a Java project?

Issue - State: closed - Opened by gajendravaradhan over 3 years ago - 4 comments
Labels: question

#28 - Add Diff app

Issue - State: closed - Opened by EnricoMi almost 4 years ago
Labels: enhancement

#20 - Get added to spark-packages.org

Issue - State: closed - Opened by EnricoMi about 4 years ago - 1 comment

#14 - Introduce user defined and fuzzy comparators

Pull Request - State: closed - Opened by chile12 over 4 years ago - 2 comments

#7 - Un-pivot / melt: implement un-pivot / melt function

Issue - State: closed - Opened by EnricoMi over 4 years ago - 1 comment
Labels: enhancement

#2 - Diff: Add floating point precision comparison

Issue - State: closed - Opened by EnricoMi over 4 years ago - 2 comments
Labels: enhancement

#1 - Diff: Implement Dataset[T].diffWith(Dataset[T], ...): Dataset[(String,T, T)] similar to joinWith

Issue - State: closed - Opened by EnricoMi over 4 years ago
Labels: enhancement

GitHub / G-Research/spark-extension issues and pull requests