delta-io/delta issues and pull requests

#3573 - [Delta] Block alter command from overriding or unsetting coordinated commits properties

Pull Request - State: closed - Opened by yumingxuanguo-db 3 months ago

#3572 - [TEST-ONLY + Refactor] Fix Vacuum test code to make use of artificial clock everywhere

Pull Request - State: closed - Opened by rajeshparangi 3 months ago

#3571 - [DO_NOT_MERGE] Run only the python tests to verify they work for #3510

Pull Request - State: open - Opened by allisonport-db 3 months ago

#3570 - [Spark] Do not use the same variable name in the coordinator registration code

Pull Request - State: closed - Opened by junlee-db 3 months ago

#3569 - [Spark] Enable Row Tracking Backfill by default in Delta

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3568 - [SPARK] Refactor IdentityColumnTestUtils

Pull Request - State: closed - Opened by zhipengmao-db 3 months ago

#3567 - [Kernel] Add Cast expression with extended upcasting support

Pull Request - State: open - Opened by johanl-db 3 months ago
Labels: kernel

#3566 - [SPARK] Add MERGE support for tables with Identity Columns

Pull Request - State: closed - Opened by zhipengmao-db 3 months ago - 1 comment

#3565 - [Feature Request] [Spark] OPTIMIZE DRY RUN

Issue - State: open - Opened by felipepessoto 3 months ago
Labels: enhancement

#3564 - Update delta-sharing-client to 1.2.0

Pull Request - State: closed - Opened by taiga-db 3 months ago - 2 comments

#3563 - 2024-08-15 infra test

Pull Request - State: open - Opened by scottsand-db 3 months ago - 1 comment

#3562 - [Spark] Use `encoderFor` to copy encoders for DeltaUDF

Pull Request - State: closed - Opened by allisonport-db 3 months ago - 1 comment

#3561 - [Delta] Centralize checks for coordinated commits property via different code paths

Pull Request - State: closed - Opened by yumingxuanguo-db 3 months ago

#3560 - [INFRA] Fix the pip version in the dockerfile as well to accomodate for versions with "-" in it

Pull Request - State: closed - Opened by allisonport-db 3 months ago

#3559 - [KERNEL] Validate readSchema is a subset of the table schema in ScanBuilder

Pull Request - State: open - Opened by ilicmarkodb 3 months ago - 1 comment
Labels: kernel

#3558 - [SPARK] Allow non-deterministic expressions in actions of merge

Pull Request - State: closed - Opened by zhipengmao-db 3 months ago - 1 comment

#3557 - [TEST-ONLY + Refactor] Fix Vacuum test code to make use of artificial clock everywhere

Pull Request - State: closed - Opened by rajeshparangi 3 months ago

#3556 - Upgrade AWS Java SDK to v2

Issue - State: open - Opened by achals 3 months ago
Labels: enhancement

#3555 - [Kernel] Add editable property to TableConfig

Pull Request - State: closed - Opened by ilicmarkodb 3 months ago - 2 comments
Labels: kernel

#3554 - [Feature Request]Support commit callback

Issue - State: open - Opened by melin 3 months ago
Labels: enhancement

#3553 - [Spark] Fix time travel utility method for ICT

Pull Request - State: closed - Opened by prakharjain09 3 months ago - 1 comment

#3552 - [Kernel] Ignore non-conforming commit files in Delta log directory

Pull Request - State: open - Opened by vkorukanti 3 months ago
Labels: kernel

#3551 - [Spark] Add RowTrackingBackfillBackfillConflictsSuite

Pull Request - State: closed - Opened by longvu-db 3 months ago - 1 comment

#3550 - [Spark] Add RowTrackingBackfill and Clone conflicts tests

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3549 - [SHARING][VARIANT] Add variant table feature to sharing client

Pull Request - State: closed - Opened by richardc-db 3 months ago

#3548 - Revert "Infra: Shard spark and spark master tests across 3 actions"

Pull Request - State: closed - Opened by scottsand-db 3 months ago - 1 comment

#3547 - [Spark] Remove duplicate unblockCommit to fix GitHub tests

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3546 - Revert "[Spark] Add RowTrackingBackfillConflictsSuite"

Pull Request - State: open - Opened by scottsand-db 3 months ago - 1 comment

#3545 - [Spark] Add Delta Connect Update/Delete Server and Scala Client

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3544 - [Docs] Refactor the docs

Pull Request - State: closed - Opened by vkorukanti 3 months ago

#3543 - [Docs] Minor update to docs build instructions

Pull Request - State: closed - Opened by vkorukanti 3 months ago

#3542 - [Infra] Separate out delta-spark python tests

Pull Request - State: closed - Opened by nastra 3 months ago - 4 comments

#3541 - [Kernel] Add widening type conversions to Kernel default parquet reader

Pull Request - State: closed - Opened by johanl-db 3 months ago - 1 comment
Labels: kernel

#3540 - [Spark] Add RowTrackingBackfillConflictsSuite

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3539 - [Kernel] Minor cleanup in delta-kernel

Pull Request - State: closed - Opened by ebyhr 3 months ago
Labels: kernel

#3538 - [BUG] Delta Table with special Characted in Column Name.

Issue - State: open - Opened by 0xbadidea 3 months ago
Labels: bug

#3537 - [Kernel] Fixed the issue with reading a list of JSON files when some files are empty

Pull Request - State: closed - Opened by ilicmarkodb 3 months ago - 2 comments
Labels: kernel

#3536 - always include CatalogTable#storage#properties into data source options

Pull Request - State: closed - Opened by cloud-fan 3 months ago - 2 comments

#3535 - [Spark] Delta Connect python client with tests running in CI

Pull Request - State: open - Opened by allisonport-db 3 months ago

#3534 - [Kernel][IcebergCompatV2] Validate `add` actions have `numRecords` in statistics

Pull Request - State: closed - Opened by vkorukanti 3 months ago
Labels: kernel

#3533 - [FLINK] [#2331] Add support to partition table by date type

Pull Request - State: closed - Opened by tuliocavalcanti 3 months ago - 6 comments
Labels: kernel

#3532 - [KERNEL] Additional Delta action support (Metadata, Protocol, CommitInfo and AddCDCFile) for LogImpl::getChangesByVersion

Pull Request - State: closed - Opened by allisonport-db 3 months ago - 1 comment

#3531 - [KERNEL] Initial support to read the Delta actions from the Delta log between two versions

Pull Request - State: closed - Opened by allisonport-db 3 months ago
Labels: kernel

#3530 - [Kernel][IcebergCompatV2] Include partition columns in physical data written to files

Pull Request - State: closed - Opened by vkorukanti 3 months ago
Labels: kernel

#3529 - Bump Spark version from 3.5.0 to 3.5.2

Pull Request - State: closed - Opened by nastra 3 months ago - 5 comments

#3528 - Separate test groups for Spark Master tests

Pull Request - State: open - Opened by nastra 3 months ago

#3527 - Separate test groups

Pull Request - State: open - Opened by nastra 3 months ago

#3526 - [SPARK][TEST-ONLY] Add more tests for Identity Column

Pull Request - State: closed - Opened by zhipengmao-db 3 months ago - 5 comments

#3525 - [SPARK] Relax metadata conflict for identity column

Pull Request - State: closed - Opened by zhipengmao-db 3 months ago

#3524 - [Infra] Increase test parallelism

Pull Request - State: closed - Opened by nastra 3 months ago - 4 comments

#3523 - [Spark] Always enable all legacy preceding features when a legacy feature is enabled

Pull Request - State: closed - Opened by andreaschat-db 3 months ago

#3522 - Larger GH runners

Pull Request - State: closed - Opened by nastra 3 months ago

#3521 - Update GH runners to latest Ubuntu version

Pull Request - State: closed - Opened by nastra 3 months ago

#3520 - [Delta][WIP] FSCK With missing DVs support

Pull Request - State: open - Opened by Sovima 3 months ago - 1 comment

#3519 - [WIP][SHARING] Support variant in sharing

Pull Request - State: closed - Opened by richardc-db 3 months ago

#3518 - (do not merge) enable test parallelism for spark

Pull Request - State: open - Opened by amogh-jahagirdar 3 months ago

#3517 - Infra: Shard spark and spark master tests across 3 actions

Pull Request - State: closed - Opened by amogh-jahagirdar 3 months ago - 2 comments

#3516 - [KERNEL][VARIANT] Basic read variant

Pull Request - State: open - Opened by richardc-db 3 months ago

#3515 - [Kernel][Default Parquet Writer] Always return the row count in returned file statistics

Pull Request - State: closed - Opened by vkorukanti 3 months ago
Labels: kernel

#3514 - Fix possible stability issues when reusing deltaLog of delta sharing queries

Pull Request - State: closed - Opened by linzhou-db 3 months ago

#3513 - docs: fix punctuation typo

Pull Request - State: open - Opened by avriiil 3 months ago

#3512 - [Spark] Remove -preview from Coordinated Commits configs

Pull Request - State: open - Opened by dhruvarya-db 3 months ago

#3511 - [Kernel][Defaults] Add field id validation when writing data to Parquet files

Pull Request - State: closed - Opened by vkorukanti 3 months ago
Labels: kernel

#3510 - [Infra][Spark] Run python tests in Spark master CI using Spark 4.0 preview release

Pull Request - State: open - Opened by allisonport-db 3 months ago

#3509 - [Kernel] Always read the checkpoint manifest for CPV2 if sidecar files are present

Pull Request - State: closed - Opened by chirag-s-db 3 months ago
Labels: kernel

#3508 - Testing if increasing parallelism helps as a baseline

Pull Request - State: open - Opened by amogh-jahagirdar 3 months ago

#3507 - [Spark][UniForm] Add `UniversalFormatSuite` and utils for `IcebergCompatV1/V2` & `REORG UPGRADE UNIFORM`

Pull Request - State: closed - Opened by xzhseh 3 months ago

#3506 - [Spark] Makes LogSegment comparison aware of coordinated commits

Pull Request - State: closed - Opened by LukasRupprecht 3 months ago - 1 comment

#3505 - [Spark] Pass scala version as optional argument to configure_spark_with_delta_pip

Pull Request - State: open - Opened by ppkgtmm 3 months ago

#3504 - [Kernel][IcebergCompatV2] Support writing nested field ids in schema to Parquet file

Pull Request - State: closed - Opened by vkorukanti 3 months ago
Labels: kernel

#3503 - [Kernel] Add demo for integration test of coordinated commit with Spark using DynamoDBCommitCoordinatorClient

Pull Request - State: open - Opened by EstherBear 3 months ago

#3502 - [Kernel] Add vacuumProtocolCheck to writer feature list

Pull Request - State: open - Opened by EstherBear 3 months ago

#3501 - [Spark] Writing of UUID commits should not use put-if-absent semantics

Pull Request - State: closed - Opened by LukasRupprecht 3 months ago

#3500 - [DO_NOT_MERGE] Delta Connect Python client with pypi packaging

Pull Request - State: open - Opened by allisonport-db 3 months ago

#3499 - [Delta][WIP] Add FSCK implementation

Pull Request - State: open - Opened by Sovima 3 months ago - 1 comment

#3498 - Revert "[Spark] Allow stale reads when commit coordinator has not been registered"

Pull Request - State: closed - Opened by dhruvarya-db 3 months ago

#3497 - invoke Spark catalog plugin API to create tables

Pull Request - State: closed - Opened by cloud-fan 3 months ago - 9 comments

#3496 - [BUG] [Spark] Permission issues in reading _delta_log

Issue - State: open - Opened by sid-habu 3 months ago - 2 comments
Labels: bug

#3495 - [Feature Request] Bucketing implementation in Delta Lake

Issue - State: open - Opened by wudanzy 3 months ago - 3 comments
Labels: enhancement

#3494 - [Infra] Make `checkstyle` fail when there are style errors

Pull Request - State: open - Opened by allisonport-db 3 months ago - 1 comment

#3493 - [3.2][Sharing] Upgrade delta-sharing-client to version 1.1.1

Pull Request - State: closed - Opened by charlenelyu-db 3 months ago

#3492 - [Sharing] Upgrade delta-sharing-client to 1.1.1

Pull Request - State: closed - Opened by charlenelyu-db 3 months ago

#3491 - [Kernel] Always read the checkpoint manifest for CPV2 if sidecar files are present

Pull Request - State: closed - Opened by chirag-s-db 3 months ago - 1 comment

#3490 - [Spark] Add Delta Connect Describe History/Detail, Convert To Delta, Restore and isDeltaTable Server and Scala Client

Pull Request - State: closed - Opened by longvu-db 3 months ago

#3489 - [Kernel] build/sbt javafmtAll

Pull Request - State: closed - Opened by nastra 3 months ago

#3488 - Fix style in PROTOCOL

Pull Request - State: closed - Opened by ebyhr 3 months ago - 1 comment

#3487 - Do not leak column mapping metadata during streaming read

Pull Request - State: closed - Opened by jackierwzhang 3 months ago

#3486 - [UNIFORM] Remove timestamp partition patch as it is not effective currently

Pull Request - State: closed - Opened by amogh-jahagirdar 3 months ago - 1 comment

#3485 - [Spark] Fix compilation of Spark master build

Pull Request - State: closed - Opened by tdas 3 months ago - 1 comment

#3484 - [Spark] Pin the `pip` version to `24.0` to get around the version format requirement

Pull Request - State: closed - Opened by vkorukanti 3 months ago

#3483 - [Spark] Pin the `pip` version to `24.0` to get around the version format requirement

Pull Request - State: closed - Opened by vkorukanti 3 months ago

#3482 - [3.2][Spark] Pin the `pip` version to `24.0` to get around the version format requirement

Pull Request - State: closed - Opened by vkorukanti 3 months ago

#3481 - [Kernel][Expressions] Add TIMEADD expression to handle TIMESTAMP mill…

Pull Request - State: closed - Opened by raveeram-db 3 months ago

#3480 - [INFRA] Cleanup build.sbt build output

Pull Request - State: closed - Opened by scottsand-db 3 months ago

#3479 - Uniform: Bump Iceberg version to 1.6

Pull Request - State: open - Opened by amogh-jahagirdar 3 months ago

#3478 - [BUG] mac idea write_uniform_hudi.py failed

Issue - State: open - Opened by liujinhui1994 3 months ago
Labels: bug

#3477 - [Kernel] Update the Kernel to support coordinated commit downgrade

Pull Request - State: open - Opened by EstherBear 3 months ago

#3476 - [Kernel] Update Kernel checkpoint with coordinated commit support

Pull Request - State: open - Opened by EstherBear 3 months ago

#3475 - [WIP][Kernel] Support for writing into column mapping enabled tables

Pull Request - State: open - Opened by vkorukanti 3 months ago
Labels: kernel

#3474 - [Spark][Backport 3.2] Fix the semantic of `shouldRewriteToBeIcebergCompatible` in REORG UPGRADE UNIFORM

Pull Request - State: closed - Opened by xzhseh 3 months ago - 1 comment

GitHub / delta-io/delta issues and pull requests