Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / G-Research/spark-extension issues and pull requests

#258 - Support diff for Spark Connect (Plugin)

Pull Request - State: open - Opened by EnricoMi about 1 month ago - 1 comment

#257 - Make ` Column.expr` accessible

Pull Request - State: closed - Opened by EnricoMi about 1 month ago - 1 comment

#256 - Support Spark4 Column Node API

Pull Request - State: closed - Opened by EnricoMi about 1 month ago - 1 comment

#255 - Move to Spark 3.5.3-SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi about 1 month ago

#254 - Simplify getDiffColumns logic

Pull Request - State: closed - Opened by EnricoMi about 1 month ago

#253 - Improve test success job

Pull Request - State: closed - Opened by EnricoMi about 2 months ago - 1 comment

#252 - Add ignore columns to diff in Python API

Pull Request - State: closed - Opened by EnricoMi about 2 months ago - 1 comment

#251 - Support diff for Spark Connect (Dataset API)

Pull Request - State: open - Opened by EnricoMi about 2 months ago - 1 comment

#250 - Check that the Java / Scala package is installed when needed

Pull Request - State: closed - Opened by EnricoMi about 2 months ago - 1 comment

#249 - Support diff with ignore columns in Python

Issue - State: closed - Opened by EnricoMi about 2 months ago
Labels: enhancement

#248 - Support Spark Connect server

Issue - State: open - Opened by EnricoMi 2 months ago - 4 comments

#247 - Detect and test Spark Connect server

Pull Request - State: closed - Opened by EnricoMi 2 months ago - 1 comment

#245 - Fix: Change env for publishing snapshot release

Pull Request - State: open - Opened by ljubon 3 months ago - 6 comments

#244 - Check that the Java / Scala package is installed when needed

Pull Request - State: closed - Opened by EnricoMi 4 months ago - 1 comment

#243 - Test Spark 4.0.0 preview

Pull Request - State: open - Opened by EnricoMi 4 months ago - 1 comment

#242 - Error: 'JavaPackage' object is not callable

Issue - State: open - Opened by rish-shar 4 months ago - 4 comments

#241 - CI: Fix warnings & deprecation messages in the workflows

Pull Request - State: closed - Opened by ljubon 5 months ago - 1 comment

#240 - Update changelog

Pull Request - State: closed - Opened by EnricoMi 5 months ago - 1 comment

#239 - Upgrade to Spark 3.4.4 snapshot

Pull Request - State: closed - Opened by EnricoMi 5 months ago - 1 comment

#238 - Changeset values should honor comparators

Pull Request - State: closed - Opened by ets 5 months ago - 5 comments

#237 - CI: Release workflow

Pull Request - State: closed - Opened by ljubon 6 months ago - 5 comments

#236 - Remove Scala 2.12 for Spark 4 snapshot

Pull Request - State: closed - Opened by EnricoMi 6 months ago - 1 comment

#235 - Reduce cache cardinality

Pull Request - State: closed - Opened by EnricoMi 6 months ago - 2 comments

#234 - Account for changed exception message in Spark 4

Pull Request - State: closed - Opened by EnricoMi 7 months ago - 1 comment

#233 - Spark 3.5.1 has been released, bump snapshot versions and pom.xml

Pull Request - State: closed - Opened by EnricoMi 7 months ago - 1 comment

#232 - Spark extension not working with 3.5.1

Issue - State: closed - Opened by datanikkthegreek 7 months ago - 3 comments

#231 - Pyspark - Import Error

Issue - State: closed - Opened by VinothKanna007 8 months ago - 5 comments

#230 - PySpark - Diff Epsilon Inclusive vs Exclusive

Issue - State: closed - Opened by VinothKanna007 8 months ago - 4 comments

#229 - Merge Differ.diff_with_options into Differ.diff and check input types

Issue - State: open - Opened by EnricoMi 8 months ago
Labels: enhancement, good first issue

#228 - Check Python input types

Issue - State: open - Opened by EnricoMi 8 months ago
Labels: enhancement, good first issue

#227 - Diff Issue

Issue - State: closed - Opened by VinothKanna007 8 months ago - 4 comments

#226 - Add map diff comparator to Python API

Pull Request - State: closed - Opened by EnricoMi 8 months ago - 1 comment

#225 - Comparators error when using pyspark

Issue - State: closed - Opened by hbashary 8 months ago - 6 comments

#224 - Move python dependencies into tests_require

Pull Request - State: closed - Opened by EnricoMi 9 months ago - 1 comment

#223 - Add install-python-deps example

Pull Request - State: closed - Opened by EnricoMi 9 months ago - 1 comment

#222 - Make create_temporary_dir work with pyspark-extension only

Pull Request - State: closed - Opened by EnricoMi 9 months ago - 1 comment

#221 - Sync python/README.md with README.md

Pull Request - State: closed - Opened by EnricoMi 9 months ago - 1 comment

#220 - Upgrade dependencies

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 2 comments

#219 - Fix skipped poetry install tests

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#218 - Debug CI test results

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 2 comments

#217 - Upgrade Spark patch versions

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#216 - Add install_poetry_project

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#215 - Allow to install PIP packages into PySpark job

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#214 - CI upgrade to Spark 3.4.3 snapshot

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#213 - Provide groupByKey shortcuts for groupBy.as

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#212 - Apply scalafmt changes, check during compile phase

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#211 - Add columns, values and nulls count where possible

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#209 - Add more columns to reading parquet metadata

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#208 - Support reading parquet schema

Pull Request - State: closed - Opened by EnricoMi 10 months ago - 1 comment

#207 - Fix detection of python test failure

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#206 - Add count_null aggregate function

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#205 - dependabot: add python

Pull Request - State: closed - Opened by ljubon 11 months ago - 4 comments

#204 - Fix version order to find latest version

Pull Request - State: closed - Opened by EnricoMi 11 months ago

#203 - Fix expected error message for spark 4.0.0-SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi 11 months ago - 1 comment

#202 - CI remove caches

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#201 - Adds scalafmt.conf

Pull Request - State: closed - Opened by EnricoMi 12 months ago - 1 comment

#200 - release spak-extension

Pull Request - State: closed - Opened by ljubon 12 months ago - 3 comments

#199 - CI: Add linter check for scala

Pull Request - State: closed - Opened by ljubon 12 months ago - 11 comments

#198 - Use Java 17 for Spark 4.0.0

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#197 - Remove --find-links for Spark 3.5.0, add to test-python matrix

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#196 - Remove check status, checks can fail on non-master branches only

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#195 - Relax status of check reports

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#194 - Install the specific pyspark version, not the dependency of the whl

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#193 - Check built whl file

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#192 - Add `gresearch.spark.parquet` to Python package in `setup.py`

Issue - State: closed - Opened by ezhou7 about 1 year ago - 2 comments

#191 - Fix `gresearch.spark.parquet` package missing from Python whl

Pull Request - State: closed - Opened by ezhou7 about 1 year ago - 2 comments

#190 - Add --filter to diff app

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#189 - Add --statistics to diff app

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#188 - Upgrade Spark to 3.3.3, 3.5.0 and 4.0.0-SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#187 - Key order sensitive map comparator

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#186 - Fix key-sensitivity in map comparator

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#185 - MapComparator is sensitive to key order

Issue - State: closed - Opened by EnricoMi about 1 year ago
Labels: bug

#184 - Best way to create a DiffComparator for json comparison of json strings

Issue - State: open - Opened by mattseburn about 1 year ago - 2 comments

#183 - Use dataset encoder rather than implicit value encoder

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 2 comments

#182 - Bump org.codehaus.mojo:properties-maven-plugin from 1.1.0 to 1.2.0

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago - 1 comment
Labels: dependencies, java

#181 - Create MAINTAINERS.md

Pull Request - State: closed - Opened by demarillacizere about 1 year ago - 1 comment

#180 - Use workflow call to structure CI workflows

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#179 - Upgrade to Spark 3.4.1

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#178 - Adjust expected error message for Spark 3.5

Pull Request - State: closed - Opened by EnricoMi about 1 year ago - 1 comment

#177 - Bump maven-surefire-plugin from 3.1.0 to 3.1.2

Pull Request - State: open - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies, java

#176 - Bump maven-surefire-report-plugin from 3.1.0 to 3.1.2

Pull Request - State: open - Opened by dependabot[bot] over 1 year ago
Labels: dependencies, java

#175 - Upgrade maven plugins

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#174 - Add licence to files missing licence header

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#173 - Bump maven-source-plugin from 3.2.1 to 3.3.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#172 - Add methods to set, append and unset Spark job description

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#171 - Bump build-helper-maven-plugin from 3.3.0 to 3.4.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#170 - Bump maven-gpg-plugin from 3.0.1 to 3.1.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#169 - Bump maven-surefire-plugin from 3.0.0 to 3.1.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#168 - Bump maven-surefire-report-plugin from 3.0.0 to 3.1.0

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 2 comments
Labels: dependencies, java

#167 - Properly fix inconsistent dependency of Spark 3.5 SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#166 - Fix inconsistent dependency of Spark 3.5 SNAPSHOT

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#165 - Bringing back Spark 3.0 and 3.1 support

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#164 - Add parallelism argument to parquet metadata read methods

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#163 - Make Spark 3.4 the main version, remove last traces of 3.0 and 3.1

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#162 - Extend reading parquet metadata

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#161 - Use Spark version instead of build version, get build versions from pom

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment

#160 - Add Spark Diff app

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 2 comments

#159 - Extend PARQUET.md from blog article

Pull Request - State: closed - Opened by EnricoMi over 1 year ago - 1 comment