delta-io/delta issues and pull requests

#3261 - [Doc][4.0] Add doc for the coordinated commits writer feature

Pull Request - State: open - Opened by dhruvarya-db 5 months ago

#3260 - [4.0 Preview][INFRA] Update integration tests to clear all possible ivy caches + minor update to S3LogStoreUtilIntegrationTest

Pull Request - State: open - Opened by allisonport-db 5 months ago

#3259 - [DELTA] Removes asInstanceOf[Metadata] casts

Pull Request - State: closed - Opened by yumingxuanguo-db 5 months ago

#3258 - [INFRA] Update integration tests to clear all possible ivy caches + minor update to S3LogStoreUtilIntegrationTest

Pull Request - State: open - Opened by allisonport-db 5 months ago

#3257 - [Tests] Some tests in `DeletionVectorsSuite` are flakey (timing out)

Issue - State: closed - Opened by vkorukanti 5 months ago

#3256 - [kernel][Feature Request] Row based Parquet Writer with flush capability

Issue - State: open - Opened by Sandy3094 5 months ago - 1 comment
Labels: enhancement, kernel

#3255 - [BUG][Spark] `DeltaTable.forName` method fails to parse fully qualified table name with catalog in Python API

Issue - State: open - Opened by donielix 5 months ago - 1 comment
Labels: bug

#3254 - [Feature Request] Support `failOnDataLoss` property for Delta source schema tracking

Issue - State: open - Opened by jackierwzhang 5 months ago
Labels: enhancement, good medium issue

#3253 - [Feature Request] Support additive change speculation for Delta source schema tracking

Issue - State: open - Opened by jackierwzhang 5 months ago
Labels: enhancement, good medium issue

#3252 - [Feature Request][Spark] Update UniForm related comment to include description to Hudi

Issue - State: open - Opened by harperjiang 5 months ago - 2 comments
Labels: enhancement, good first issue

#3251 - [Spark] Support create external table for clustered table

Pull Request - State: closed - Opened by zedtang 5 months ago

#3250 - [Feature Request][Spark] Relax check for generated columns and CHECK constraints depending on nested struct fields

Issue - State: closed - Opened by johanl-db 5 months ago - 2 comments
Labels: enhancement, good first issue

#3249 - [Feature Request] Consolidate optimizer suite logic across compaction and z-order

Issue - State: open - Opened by Kimahriman 5 months ago
Labels: enhancement

#3248 - [Feature Request][Spark] Add test coverage for clustering on generated columns

Issue - State: open - Opened by zedtang 5 months ago - 4 comments
Labels: enhancement, good first issue

#3247 - [Feature Request][SPARK] Make Vacuum DRY RUN list all the files along with the the size of the files

Issue - State: open - Opened by rajeshparangi 5 months ago - 1 comment
Labels: enhancement, good first issue

#3246 - [DELTA] Add logs around managed commits

Pull Request - State: closed - Opened by prakharjain09 5 months ago - 1 comment

#3245 - [BUG][Spark] Logs are not compacted

Issue - State: closed - Opened by Minashraf 5 months ago - 2 comments
Labels: bug

#3244 - [Spark] Throw exception when additional listing also doesn't reconcile the listing gap in testing

Pull Request - State: closed - Opened by sumeet-db 5 months ago

#3243 - [Spark][4.0] Add an integration test for DynamoDB Commit Coordinator

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3242 - [Spark] Use SQL expression in VARCHAR violation error

Pull Request - State: closed - Opened by tomvanbussel 5 months ago

#3241 - [Spark][4.0] Rename managed commit to coordinated commits

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3240 - [Feature Request] Support for Spark Connect (aka Delta Connect)

Issue - State: open - Opened by allisonport-db 5 months ago
Labels: enhancement

#3239 - [Spark][3.2] Support RESTORE for clustered table

Pull Request - State: closed - Opened by zedtang 5 months ago - 1 comment

#3238 - Fix the issue of not using the correct refreshToken

Pull Request - State: closed - Opened by linzhou-db 5 months ago - 1 comment

#3237 - [Spark] Rename managed commit to coordinated commits

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3236 - [Spark] use map lookup in createPhysicalSchema

Pull Request - State: closed - Opened by andrewxue-db 5 months ago

#3235 - [4.0preview][Kernel][Parquet Writer] Fix an issue with writing decimal as binary (#3233)

Pull Request - State: closed - Opened by vkorukanti 5 months ago

#3234 - [Delta Uniform] Compute correct MAX_ID in column mapping on a schema with nested fields and already have IDs assigned

Pull Request - State: closed - Opened by ChengJi-db 5 months ago

#3233 - [Kernel][Parquet Writer] Fix an issue with writing decimal as binary

Pull Request - State: closed - Opened by vkorukanti 5 months ago
Labels: kernel

#3232 - [Doc] Delta 4.0 - Delta Connect documentation

Pull Request - State: closed - Opened by longvu-db 5 months ago

#3231 - [Feature Request] Spark Connect support for the Python API

Issue - State: open - Opened by sebastiandaberdaku 5 months ago - 1 comment
Labels: enhancement

#3230 - [Kernel][Expressions] Add IS NOT DISTINCT FROM support

Pull Request - State: closed - Opened by zzl-7 5 months ago - 3 comments
Labels: kernel

#3229 - [BUG] Error on import with new release

Issue - State: closed - Opened by stinodego 5 months ago - 1 comment
Labels: bug

#3228 - [Feature Request][Spark] Remove dropped columns from Parquet files in REORG TABLE (PURGE)

Issue - State: closed - Opened by johanl-db 5 months ago - 2 comments
Labels: enhancement, good first issue

#3227 - [BUG][Spark] INSERT INTO struct evolution in map/arrays breaks when a column is renamed

Issue - State: open - Opened by johanl-db 5 months ago - 10 comments
Labels: bug, good medium issue

#3226 - [Delta] Simplify DeltaHistoryManagerSuite

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3225 - [Spark] Make naming of manage commit consistent

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3224 - [Spark] Make naming of manage commit consistent

Pull Request - State: closed - Opened by dhruvarya-db 5 months ago

#3223 - [Spark][Test-only] Split type widening tests in multiple suites

Pull Request - State: closed - Opened by johanl-db 5 months ago

#3222 - [Spark] Append the tieBreaker unicode max character only if we actually truncated the string

Pull Request - State: closed - Opened by sumeet-db 5 months ago

#3221 - [Spark] Use checkError in MERGE tests instead of checking error messages

Pull Request - State: open - Opened by johanl-db 5 months ago

#3220 - [VARIANT] Add variant integration tests

Pull Request - State: closed - Opened by richardc-db 6 months ago

#3219 - [Feature Request] Reverse UniForm

Issue - State: open - Opened by lzlfred 6 months ago - 1 comment
Labels: enhancement

#3218 - [Kernel] Add the InCommitTimestamp table feature

Pull Request - State: closed - Opened by EstherBear 6 months ago

#3217 - [QUESTION] How can one generate Uniform data using local standalone spark?

Issue - State: open - Opened by frosforever 6 months ago - 2 comments
Labels: bug

#3216 - [Kernel] Support table config delta.appendOnly.

Pull Request - State: open - Opened by horizonzy 6 months ago - 1 comment

#3215 - [BUG] [Kernel] Support reading 2-level arrays in the default Parquet handler

Issue - State: open - Opened by horizonzy 6 months ago - 1 comment
Labels: bug, kernel

#3214 - [Spark][WIP] Leverage type widening in Delta streaming source

Pull Request - State: closed - Opened by johanl-db 6 months ago

#3213 - [Kernel] Support update table schema and configs.

Pull Request - State: open - Opened by horizonzy 6 months ago

#3212 - [Kernel] Support clean expired log

Pull Request - State: closed - Opened by horizonzy 6 months ago - 4 comments
Labels: kernel

#3211 - [Kernel] Introduce more configs.

Pull Request - State: closed - Opened by horizonzy 6 months ago

#3210 - [Kernel] Introduce more operations: UPGRADE SCHEMA, COMPACT, UPGRADE SCHEMA.

Pull Request - State: open - Opened by horizonzy 6 months ago

#3209 - [Kernel] Add `remove` schema to the single action schema

Pull Request - State: closed - Opened by horizonzy 6 months ago
Labels: kernel

#3208 - [Kernel][Test] Add test for reading a shallow cloned table

Pull Request - State: closed - Opened by vkorukanti 6 months ago
Labels: kernel

#3207 - [VARIANT] Add variant integration tests

Pull Request - State: closed - Opened by richardc-db 6 months ago

#3206 - [VARIANT][DOCS] Add Variant type (Preview) to docs

Pull Request - State: closed - Opened by richardc-db 6 months ago - 1 comment

#3205 - [Kernel] Add Delta log replay metrics in active file list construction

Pull Request - State: closed - Opened by vkorukanti 6 months ago
Labels: kernel

#3204 - [SPARK] [DELTA_UNIFORM] Expose Iceberg Shallow Clone through Delta Uniform SQL syntax

Pull Request - State: open - Opened by xzhseh 6 months ago

#3203 - [Doc] Add doc for the coordinated commits writer feature

Pull Request - State: closed - Opened by dhruvarya-db 6 months ago

#3202 - [INFRA] Move unidoc settings from `crossSparkSettings()` to the spark project definition since they are the same

Pull Request - State: closed - Opened by allisonport-db 6 months ago

#3201 - [Spark] Add Delta Connect Python Client for First 4.0 Preview branch

Pull Request - State: closed - Opened by longvu-db 6 months ago

#3200 - [Spark] Add Delta Connect Scala Client for First 4.0 Preview branch

Pull Request - State: closed - Opened by longvu-db 6 months ago

#3199 - [Spark] Widen outer references in subqueries during conflict checking

Pull Request - State: closed - Opened by cstavr 6 months ago

#3198 - [Spark] Add Delta Connect Server Lib for First 4.0 Preview branch

Pull Request - State: closed - Opened by longvu-db 6 months ago

#3197 - [4.0-preview1][INFRA] Use public Spark release for PyPI and Maven

Pull Request - State: closed - Opened by allisonport-db 6 months ago

#3196 - [Kernel] Throw `KernelException` when `VOID` type is encountered

Pull Request - State: closed - Opened by vkorukanti 6 months ago

#3195 - [Spark][3.2] Fix replacing clustered table with non-clustered table

Pull Request - State: closed - Opened by zedtang 6 months ago

#3194 - [Spark] Support RESTORE for clustered table

Pull Request - State: closed - Opened by zedtang 6 months ago

#3193 - [Spark] Add a Delta config to enable/disable the workaround of handling colons ':' in paths

Pull Request - State: closed - Opened by xupefei 6 months ago - 1 comment

#3192 - [DOCS] vacuum duration check: add note for pyspark users

Pull Request - State: open - Opened by avriiil 6 months ago

#3191 - [SPARK][MINOR] Code cleanup (reformatting and typo hunting)

Pull Request - State: closed - Opened by jaceklaskowski 6 months ago - 1 comment

#3190 - [DO_NOT_MERGE][4.0-preview1][Spark] Add Delta Connect Scala Client

Pull Request - State: open - Opened by allisonport-db 6 months ago

#3189 - [Spark] Fix race condition in Uniform conversion

Pull Request - State: closed - Opened by harperjiang 6 months ago

#3188 - [Kernel] Write a new inCommitTimestamp in CommitInfo on every commit

Pull Request - State: open - Opened by EstherBear 6 months ago - 1 comment

#3187 - [DO_NOT_MERGE][4.0-preview1][Spark] Add Delta Connect Python Client

Pull Request - State: open - Opened by allisonport-db 6 months ago

#3186 - [Kernel] Handle long values in `FieldMetadata` parsing

Pull Request - State: closed - Opened by vkorukanti 6 months ago
Labels: kernel

#3185 - [Kernel][Expression] - Performance Optimization for LIKE expression evaluation

Pull Request - State: closed - Opened by krishnanravi 6 months ago - 2 comments
Labels: kernel

#3184 - [DO_NOT_MERGE] CI test

Pull Request - State: open - Opened by allisonport-db 6 months ago

#3183 - [Delta Uniform] Compute correct MAX_ID in column mapping on a schema with nested fields and already have IDs assigned

Pull Request - State: closed - Opened by ChengJi-db 6 months ago

#3182 - [INFRA] Build changes to use the Spark 4.0 preview RC and prepare for Delta 4.0 preview release using Spark 4.0 preview

Pull Request - State: closed - Opened by allisonport-db 6 months ago

#3176 - [Kernel] Make some non-breaking changes to support Scala 2.13 in the future and avoid unidoc failures for missing internal packages

Pull Request - State: closed - Opened by allisonport-db 6 months ago

GitHub / delta-io/delta issues and pull requests