apache/hudi issues and pull requests

#11832 - [HUDI-8124] Allow reuse of meta client in Meta Sync path

Pull Request - State: closed - Opened by the-other-tim-brown about 1 month ago - 2 comments
Labels: meta-sync, code-refactor, size:L

#11816 - [HUDI-8034] Support custom key generator with HoodieCatalogTable

Pull Request - State: closed - Opened by lokeshj1703 about 1 month ago - 1 comment
Labels: size:L

#11788 - [HUDI-8089] Remove support for Spark2 and Scala2.11 from Hudi

Pull Request - State: open - Opened by jonvex about 2 months ago - 1 comment
Labels: size:XL

#11785 - Unable to merge cdc records to Hudi snapshot.

Issue - State: open - Opened by rcc1101 about 2 months ago - 6 comments
Labels: writer-core

#11783 - [SUPPORT] Failed to use Bloom filter Indexing

Issue - State: open - Opened by Gatsby-Lee about 2 months ago - 8 comments
Labels: index, version-compatibility

#11781 - [SUPPORT]create hive table on top of existing hudi table

Issue - State: closed - Opened by ayan2k21 about 2 months ago - 4 comments
Labels: meta-sync

#11780 - [HUDI-8066] cherry pick Flink 1.18 into 0.14.2

Pull Request - State: closed - Opened by HuangZhenQiu about 2 months ago - 3 comments
Labels: size:XL

#11752 - [HUDI-8068] Hook up source partitions to s3 incr source

Pull Request - State: closed - Opened by rmahindra123 about 2 months ago - 1 comment
Labels: size:S

#11733 - [WIP]: [HUDI-7947][RFC-80] Support column families (configuration part)

Pull Request - State: open - Opened by wombatu-kun about 2 months ago
Labels: size:L

#11727 - [HUDI-7930] Flink Support for Array of Row and Map of Row value

Pull Request - State: open - Opened by David-N-Perkins about 2 months ago - 8 comments
Labels: size:XL

#11708 - [SUPPORT] Data corruption in parquet file in Hudi table (ParquetDecodingException)

Issue - State: open - Opened by mzheng-plaid 2 months ago - 7 comments
Labels: priority:critical, data-corruption

#11706 - [HUDI-8024] Test secondary index updates and rollback

Pull Request - State: open - Opened by codope 2 months ago - 1 comment
Labels: size:M

#11702 - [HUDI-8032] Fix MockStateInitializationContext#getKeyedStateStore to use non-KeyedStateStore

Pull Request - State: closed - Opened by usberkeley 2 months ago - 15 comments
Labels: size:L

#11639 - [HUDI-7919] docker demo spark3.5 support for IT

Pull Request - State: open - Opened by jonvex 3 months ago - 1 comment
Labels: size:XL

#11606 - [HUDI-7975] Provide an API to create empty commit

Pull Request - State: open - Opened by suryaprasanna 3 months ago - 3 comments
Labels: size:M

#11605 - [HUDI-7974] Create empty clean commit at a cadence and make it configurable

Pull Request - State: open - Opened by suryaprasanna 3 months ago - 1 comment
Labels: size:M

#11580 - [HUDI-7507] Adding timestamp ordering validation before creating requested instant

Pull Request - State: open - Opened by nsivabalan 3 months ago - 12 comments
Labels: size:L

#11555 - [RFC-79] Improving reliability of concurrent table service executions and rollbacks

Pull Request - State: open - Opened by kbuci 3 months ago
Labels: size:M

#11541 - [HUDI-7940] Pass HoodieIngestionMetrics to Error Table Writer to be able to emit metrics for Error Table Writer

Pull Request - State: closed - Opened by rmahindra123 3 months ago - 2 comments
Labels: size:M

#11481 - [SUPPORT] - Performance Variation in Hudi 0.14

Issue - State: open - Opened by RuyRoaV 3 months ago - 7 comments
Labels: performance

#11396 - [HUDI-7830] Add predicate filter pruning for snapshot queries in hudi related sources

Pull Request - State: closed - Opened by vinishjail97 4 months ago - 1 comment
Labels: size:M

#11377 - [HUDI-7825]Support Report pending clustering and compaction plan metric

Pull Request - State: open - Opened by LXin96 4 months ago - 1 comment
Labels: size:S

#11344 - [HUDI-7507] Adding timestamp ordering validation before creating requested instant

Pull Request - State: closed - Opened by nsivabalan 4 months ago - 8 comments
Labels: size:M

#11335 - org.apache.hudi.exception.HoodieException: org.apache.avro.AvroTypeException: Cannot encode decimal with precision 14 as max precision 13

Issue - State: open - Opened by KSubramanyaH 4 months ago - 11 comments
Labels: schema-and-data-types, priority:critical

#11175 - [HUDI-7718] Use source profile in HoodieIncrSource

Pull Request - State: closed - Opened by vinishjail97 5 months ago - 1 comment
Labels: size:M

#11157 - [HUDI-7716] Adding more logs around index lookup, kafka sources.

Pull Request - State: closed - Opened by nsivabalan 5 months ago - 1 comment
Labels: size:S

#11149 - [HUDI-7711] Fix MultiTableStreamer to read per-stream properties file

Pull Request - State: closed - Opened by hwani3142 5 months ago - 7 comments
Labels: size:M

#11122 - [SUPPORT] java.lang.OutOfMemoryError: Requested array size exceeds VM limit on data ingestion to COW table

Issue - State: open - Opened by TarunMootala 5 months ago - 11 comments
Labels: priority:major, spark, spark-streaming

#11073 - [HUDI-7596] Enable Jacoco code coverage report across multiple modules

Pull Request - State: closed - Opened by danny0405 5 months ago - 5 comments
Labels: project-build, size:S

#11031 - [HUDI-7622] Optimize HoodieTableSource's sanity check

Pull Request - State: closed - Opened by zhuanshenbsj1 6 months ago - 4 comments
Labels: size:L

#11006 - [HUDI-7609] Support array field type whose element type can be nullable

Pull Request - State: closed - Opened by empcl 6 months ago - 1 comment
Labels: size:S

#10998 - [HUDI-7604] Automatically set hoodie.table.name based on write config in Spark

Pull Request - State: closed - Opened by jonvex 6 months ago - 1 comment
Labels: size:S

#10983 - [SUPPORT] org.apache.avro.SchemaParseException: Can't redefine decimal field

Issue - State: open - Opened by junkri 6 months ago - 4 comments
Labels: schema-and-data-types, priority:major, on-call-triaged

#10969 - [HUDI-8236] Upgrade maven-surefire-plugin and maven-failsafe-plugin to 3.5.0

Pull Request - State: open - Opened by sullis 6 months ago - 2 comments
Labels: size:M

#10897 - [HUDI-7466] Add tests to AWSGlueCatalogSyncClient

Pull Request - State: closed - Opened by VitoMakarevich 7 months ago - 15 comments
Labels: size:S

#10835 - [HUDI-8218] Change the properties to be loaded from both default path and environment variable

Pull Request - State: closed - Opened by Amar1404 7 months ago - 6 comments
Labels: configs, size:S

#10646 - [HUDI-7596] Enable code coverage in Azure

Pull Request - State: closed - Opened by linliu-code 8 months ago - 2 comments
Labels: size:L

#10643 - [MINOR] Remove a dependency from hadoop-common

Pull Request - State: open - Opened by linliu-code 8 months ago - 1 comment

#10642 - [MINOR] Update func index RFC

Pull Request - State: open - Opened by codope 8 months ago - 1 comment

#10641 - [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics

Pull Request - State: open - Opened by lokeshj1703 8 months ago - 2 comments

#10640 - [HUDI-7394] Fix run script of hudi-timeline-server-bundle

Pull Request - State: closed - Opened by voonhous 8 months ago - 1 comment

#10639 - [SUPPORT] Error upsetting bucketType UPDATE for partition :20240119

Issue - State: open - Opened by zjq888 8 months ago

#10638 - [SUPPORT]Problems with Hudi's version using LSM-tree

Issue - State: open - Opened by yekebit 8 months ago - 1 comment
Labels: feature-enquiry

#10637 - [MINOR] Docker push

Pull Request - State: open - Opened by linliu-code 8 months ago - 1 comment

#10636 - [HUDI-7392] Fix connection leak causing lingering CLOSE_WAIT

Pull Request - State: closed - Opened by voonhous 8 months ago - 1 comment
Labels: resource-leak

#10635 - [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry

Pull Request - State: open - Opened by lokeshj1703 8 months ago - 1 comment

#10634 - [MINOR] fix typo

Pull Request - State: closed - Opened by lxliyou001 8 months ago - 7 comments

#10633 - [HUDI-7387] Serializable Class need contains serialVersionUID to keep compatibility in upgrade

Pull Request - State: closed - Opened by xuzifu666 8 months ago - 1 comment

#10632 - [HUDI-7341] Support unmerged record read

Pull Request - State: open - Opened by linliu-code 8 months ago - 1 comment

#10631 - [HUDI-7341] Support unmerged record read

Pull Request - State: open - Opened by linliu-code 8 months ago - 1 comment

#10630 - [HUDI-7386] add builder to filegroup reader

Pull Request - State: open - Opened by jonvex 8 months ago - 1 comment

#10629 - [SUPPORT] BQ synch tool not working with HUDI bundle jar

Issue - State: open - Opened by masthanmca 8 months ago - 1 comment
Labels: meta-sync, priority:major, gcp-support

#10628 - File not found while using metadata table for insert_overwrite table

Issue - State: open - Opened by Shubham21k 8 months ago - 6 comments
Labels: priority:major, writer-core

#10627 - [HUDI-7375] Disable a test method failure caused by MiniHdfs

Pull Request - State: closed - Opened by linliu-code 8 months ago - 1 comment

#10626 - Inconsistency in Hudi Table Configuration between Initial Insert and Subsequent Merges

Issue - State: open - Opened by prashant462 8 months ago - 7 comments
Labels: priority:critical, on-call-triaged, dbt

#10625 - [HUDI-7384][secondary-index][write] build secondary index on the keys…

Pull Request - State: open - Opened by bhat-vinay 8 months ago - 1 comment

#10624 - [DOCS] Add a new Hudi Architectural Stack page

Pull Request - State: closed - Opened by dipankarmazumdar 8 months ago - 1 comment

#10623 - [HUDI-7351] Handle case when glue expression larger than 2048 limit

Pull Request - State: closed - Opened by parisni 8 months ago - 2 comments

#10622 - Apache Hudi Auto-Size During Writes is not Working for Flink SQL

Issue - State: closed - Opened by vkhoroshko 8 months ago - 4 comments
Labels: flink, feature-enquiry

#10621 - [HUDI-7382] Get partitions from active timeline instead of listing when building clustering plan

Pull Request - State: open - Opened by fhan688 8 months ago - 1 comment

#10620 - [HUDI-7382] Get partitions from active timeline instead of listing when building clustering plan

Pull Request - State: closed - Opened by fhan688 8 months ago

#10619 - [HUDI-7381] Fix compaction write stats and metrics for create and upsert time

Pull Request - State: open - Opened by rmahindra123 8 months ago - 1 comment

#10618 - [HUDI-7379] Exclude jackson-module-afterburner from hudi-aws module

Pull Request - State: open - Opened by PrabhuJoseph 8 months ago - 2 comments

#10617 - [HUDI-7146] Handle duplicate keys in HFile

Pull Request - State: closed - Opened by bhat-vinay 8 months ago - 1 comment

#10616 - [SUPPORT] Using MRO table and synchronizing to hive, Flink checkpoint failed, resulting in log files being unable to scroll to parquet files

Issue - State: open - Opened by Toroidals 8 months ago - 4 comments
Labels: priority:major, flink

#10615 - [HUDI-7378] Fix Spark SQL DML with custom key generator

Pull Request - State: open - Opened by yihua 8 months ago - 1 comment

#10614 - POC hudi-aws integration testing w/ moto

Pull Request - State: open - Opened by parisni 8 months ago - 1 comment

#10613 - [HUDI-7284] Fix bad method name getLastPendingClusterCommit to getLastPendingClusterInstant

Pull Request - State: closed - Opened by jonvex 8 months ago - 1 comment

#10612 - [HUDI-7373] Make schema evolution doc and config correct

Pull Request - State: open - Opened by jonvex 8 months ago

#10611 - [HUDI-7373] revert config hoodie.write.handle.missing.cols.with.lossless.type.promotion

Pull Request - State: closed - Opened by jonvex 8 months ago - 2 comments

#10610 - [HUDI-6902] Fix string format of the timestamp in a test

Pull Request - State: closed - Opened by linliu-code 8 months ago - 2 comments

#10609 - RLI Spark Hudi Error occurs when executing map

Issue - State: open - Opened by maheshguptags 8 months ago - 5 comments
Labels: priority:critical, index

#10608 - [SUPPORT] Executor executes action [commits the instant 20240202161708414] error

Issue - State: open - Opened by Toroidals 8 months ago - 6 comments

#10607 - [HUDI-7367] Add makeQualified APIs

Pull Request - State: closed - Opened by yihua 8 months ago - 2 comments
Labels: release-1.0.0

#10606 - [HUDI-7365] Fix a flaky test for timestamp output format

Pull Request - State: closed - Opened by linliu-code 8 months ago - 2 comments

#10605 - [MINOR] Add UT org.apache.hudi.io.TestHoodieTimelineArchiver#testRetryArchivalAfterPreviousFailedDeletion

Pull Request - State: open - Opened by kbuci 8 months ago - 1 comment
Labels: release-0.14.2

#10604 - [HUDI-7351] Implement partition pushdown for glue

Pull Request - State: closed - Opened by parisni 8 months ago - 6 comments

#10603 - [DOCS][WEBSITE] Update team page for pmc and committer list

Pull Request - State: closed - Opened by nfarah86 8 months ago - 2 comments

#10602 - [HUDI-7366] Fix HoodieLocation with encoded paths

Pull Request - State: closed - Opened by yihua 8 months ago - 2 comments
Labels: release-1.0.0

#10601 - [SUPPORT] FileNotFoundException when clustering

Issue - State: open - Opened by echisan 8 months ago - 1 comment

#10600 - [MINOR] Reorder Azure CI test modules

Pull Request - State: closed - Opened by linliu-code 8 months ago - 1 comment

#10599 - [HUDI-7364] Move InLineFs classes to hudi-hadoop-common module

Pull Request - State: open - Opened by yihua 8 months ago - 1 comment
Labels: release-1.0.0

#10598 - [HUDI-7385] Add config for custom write support for parquet row writer

Pull Request - State: open - Opened by jonvex 8 months ago - 1 comment

#10597 - [Docs] Added known regression note for 0.14.1 release related to ComplexKeyGen

Pull Request - State: closed - Opened by ad1happy2go 8 months ago

#10596 - [HUDI-7362] Fix hudi partition base path scheme to s3

Pull Request - State: open - Opened by parisni 8 months ago - 1 comment

#10595 - [SUPPORT] Athena does not support s3a partition scheme anymore leading to missing data

Issue - State: open - Opened by parisni 8 months ago
Labels: aws-support, priority:critical, release-0.14.2

#10594 - [HUDI-9424]Support using local timezone when writing flink TIMESTAMP data

Pull Request - State: open - Opened by cmmp6 8 months ago - 5 comments

#10593 - [HUDI-7361] Fix a concurrency issue caused by rollbackFailedWrites

Pull Request - State: closed - Opened by eric9204 8 months ago - 1 comment

#10592 - [MINOR] Add serialVersionUID to HoodieRecord class

Pull Request - State: closed - Opened by KingdomWG 8 months ago - 1 comment

#10591 - [HUDI-6497] Replace FileSystem, Path, and FileStatus usage in `hudi-common` module

Pull Request - State: open - Opened by yihua 8 months ago - 2 comments
Labels: release-1.0.0

#10590 - [SUPPORT] java.lang.NoClassDefFoundError: org/apache/hudi/com/fasterxml/jackson/module/scala/DefaultScalaModule$ when doing an Incremental CDC Query in 0.14.1

Issue - State: open - Opened by johnwilsonabartys 8 months ago - 2 comments
Labels: priority:critical, on-call-triaged, reader-core, class-not-found

#10589 - [hudi bucket prune]

Issue - State: closed - Opened by lookingUpAtTheSky 8 months ago - 2 comments
Labels: performance, spark, optimizer

#10588 - [HUDI-7340] Use spillable map for cached log records in HoodieBaseFileGroupRecordBuffer

Pull Request - State: closed - Opened by danny0405 8 months ago - 2 comments

#10587 - Upsert operation not working and job is running longer while using "Record level index" in Apache Hudi 0.14 in EMR 6.15

Issue - State: open - Opened by SudhirSaxena 8 months ago - 23 comments
Labels: performance, priority:critical, priority:major

GitHub / apache/hudi issues and pull requests