G-Research/spark-extension issues and pull requests

#262 - Use pagination for clearing caches

Pull Request - State: closed - Opened by EnricoMi 15 days ago

#261 - Only save caches on master (and not schedule event)

Pull Request - State: closed - Opened by EnricoMi 15 days ago - 1 comment

#260 - Release workflow fails on snapshot version test

Issue - State: open - Opened by EnricoMi 15 days ago - 1 comment

#259 - Upgrade to Spark 3.4.4 + 3.5.3 and upgrade Pyhon versions

Pull Request - State: closed - Opened by EnricoMi 26 days ago - 1 comment

#258 - Support diff for Spark Connect (Plugin)

Pull Request - State: open - Opened by EnricoMi 3 months ago - 1 comment

#257 - Make ` Column.expr` accessible

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#256 - Support Spark4 Column Node API

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#255 - Move to Spark 3.5.3-SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi 3 months ago

#254 - Simplify getDiffColumns logic

Pull Request - State: closed - Opened by EnricoMi 3 months ago

#253 - Improve test success job

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#252 - Add ignore columns to diff in Python API

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#251 - Support diff for Spark Connect (Dataset API)

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#250 - Check that the Java / Scala package is installed when needed

Pull Request - State: closed - Opened by EnricoMi 3 months ago - 1 comment

#249 - Support diff with ignore columns in Python

Issue - State: closed - Opened by EnricoMi 3 months ago
Labels: enhancement

#248 - Support Spark Connect server

Issue - State: open - Opened by EnricoMi 4 months ago - 4 comments

#247 - Detect and test Spark Connect server

Pull Request - State: closed - Opened by EnricoMi 4 months ago - 1 comment

#246 - Not able to use spark-extension package with Spark Connect server / Databricks 14.x runtime

Issue - State: open - Opened by shri-0509 4 months ago - 5 comments

#245 - Fix: Change env for publishing snapshot release

Pull Request - State: open - Opened by ljubon 4 months ago - 7 comments

#244 - Check that the Java / Scala package is installed when needed

Pull Request - State: closed - Opened by EnricoMi 5 months ago - 1 comment

#243 - Test Spark 4.0.0 preview

Pull Request - State: open - Opened by EnricoMi 6 months ago - 1 comment

#242 - Error: 'JavaPackage' object is not callable

Issue - State: open - Opened by rish-shar 6 months ago - 4 comments

#241 - CI: Fix warnings & deprecation messages in the workflows

Pull Request - State: closed - Opened by ljubon 6 months ago - 1 comment

#240 - Update changelog

Pull Request - State: closed - Opened by EnricoMi 7 months ago - 1 comment

#239 - Upgrade to Spark 3.4.4 snapshot

Pull Request - State: closed - Opened by EnricoMi 7 months ago - 1 comment

#238 - Changeset values should honor comparators

Pull Request - State: closed - Opened by ets 7 months ago - 5 comments

#237 - CI: Release workflow

Pull Request - State: closed - Opened by ljubon 8 months ago - 5 comments

#236 - Remove Scala 2.12 for Spark 4 snapshot

Pull Request - State: closed - Opened by EnricoMi 8 months ago - 1 comment

#235 - Reduce cache cardinality

Pull Request - State: closed - Opened by EnricoMi 8 months ago - 2 comments

#234 - Account for changed exception message in Spark 4

Pull Request - State: closed - Opened by EnricoMi 8 months ago - 1 comment

#233 - Spark 3.5.1 has been released, bump snapshot versions and pom.xml

Pull Request - State: closed - Opened by EnricoMi 8 months ago - 1 comment

#232 - Spark extension not working with 3.5.1

Issue - State: closed - Opened by datanikkthegreek 9 months ago - 3 comments

#231 - Pyspark - Import Error

Issue - State: closed - Opened by VinothKanna007 10 months ago - 5 comments

#230 - PySpark - Diff Epsilon Inclusive vs Exclusive

Issue - State: closed - Opened by VinothKanna007 10 months ago - 4 comments

#229 - Merge Differ.diff_with_options into Differ.diff and check input types

Issue - State: open - Opened by EnricoMi 10 months ago
Labels: enhancement, good first issue

#228 - Check Python input types

Issue - State: open - Opened by EnricoMi 10 months ago
Labels: enhancement, good first issue

#227 - Diff Issue

Issue - State: closed - Opened by VinothKanna007 10 months ago - 4 comments

#226 - Add map diff comparator to Python API

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#225 - Comparators error when using pyspark

Issue - State: closed - Opened by hbashary 10 months ago - 6 comments

#224 - Move python dependencies into tests_require

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#223 - Add install-python-deps example

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#222 - Make create_temporary_dir work with pyspark-extension only

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#221 - Sync python/README.md with README.md

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#220 - Upgrade dependencies

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 2 comments

#219 - Fix skipped poetry install tests

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#218 - Debug CI test results

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 2 comments

#217 - Upgrade Spark patch versions

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#216 - Add install_poetry_project

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#215 - Allow to install PIP packages into PySpark job

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#214 - CI upgrade to Spark 3.4.3 snapshot

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#213 - Provide groupByKey shortcuts for groupBy.as

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#212 - Apply scalafmt changes, check during compile phase

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#211 - Add columns, values and nulls count where possible

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#210 - Diff: diff method throws IllegalArgumentException when specifying multiple id columns

Issue - State: closed - Opened by mschawag 12 months ago - 2 comments

#209 - Add more columns to reading parquet metadata

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#208 - Support reading parquet schema

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#207 - Fix detection of python test failure

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#206 - Add count_null aggregate function

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#205 - dependabot: add python

Pull Request - State: closed - Opened by ljubon about 1 year ago - 4 comments