Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / GoogleCloudDataproc/spark-bigquery-connector issues and pull requests

#1009 - 'UnknownException' in 0.31.1 (Works fine in 0.28.0)

Issue - State: closed - Opened by david-royo-db over 1 year ago - 3 comments

#1009 - 'UnknownException' in 0.31.1 (Works fine in 0.28.0)

Issue - State: closed - Opened by david-royo-db over 1 year ago - 3 comments

#1008 - publish metrics using spark custom metrics API

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 6 comments

#1007 - Implement at-least-once option that utilizes default stream

Pull Request - State: closed - Opened by agrawal-siddharth over 1 year ago - 7 comments

#1005 - Not able to apply push down filtering to ingestion time partitioned table

Issue - State: closed - Opened by pricemg over 1 year ago - 7 comments

#1003 - Cannot write to BigQuery from within Databricks Spark (read is ok)

Issue - State: closed - Opened by vasco-abreu over 1 year ago - 9 comments

#1002 - StatusRuntimeException: INTERNAL: request failed: internal error

Issue - State: closed - Opened by Ahaberling over 1 year ago - 2 comments

#1001 - Partitioning updates (WIP)

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#1000 - Support Spark 3.4 for BigQuery Pushdown

Issue - State: closed - Opened by idc101 over 1 year ago - 5 comments

#997 - Changing Schema fails if newly added field is Numeric / BigNumeric

Issue - State: closed - Opened by bbarodia over 1 year ago - 3 comments

#996 - Custom Connector - AWS Glue for latest version of spark-bigquery connector

Issue - State: closed - Opened by kaa125 over 1 year ago - 5 comments
Labels: waiting for information

#995 - Indirect writer does not respect `gcpImpersonationServiceAccount` option

Issue - State: closed - Opened by pPanda-beta over 1 year ago - 5 comments

#993 - Spark ML vector does not work on Serverless

Issue - State: closed - Opened by karthikt16 over 1 year ago - 1 comment

#991 - Fix to support allowFieldAddition for columns with nested fields

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 5 comments

#990 - Adding new columns in struct type columns no longer possible in BQ

Issue - State: closed - Opened by pricemg over 1 year ago - 1 comment

#985 - failed to read bigquery table in internal network

Issue - State: closed - Opened by stallapalli20 over 1 year ago - 4 comments
Labels: waiting for information

#984 - spark-3.1-bigquery-0.30.0.jar - writes in Parquet not AVRO

Issue - State: closed - Opened by annacielas over 1 year ago - 1 comment

#982 - Datetime fix 2.4.x-read support (write still WIP)

Pull Request - State: closed - Opened by kmalikgoogle over 1 year ago

#981 - Fixed BigNumeric acceptance tests

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 1 comment

#979 - Create spark table from BQ table fix

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 1 comment

#978 - reverting PR#951

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 1 comment

#977 - Support for BQ DATETIME / Spark 3.4.0 TimestampNTZType

Issue - State: closed - Opened by sixdimensionalarray over 1 year ago - 3 comments
Labels: enhancement

#976 - Invalid dataset error while having dataset starting with number

Issue - State: open - Opened by saugat-joshi over 1 year ago - 1 comment

#974 - Applying DPP to Hive partitioned BigLake tables

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 7 comments

#974 - Applying DPP to Hive partitioned BigLake tables

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 7 comments

#973 - Added support for writing to CMEK managed tables

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 8 comments

#972 - Incremental load within glue etl job utilizing this connector

Issue - State: open - Opened by Ahaberling over 1 year ago - 1 comment

#971 - Fix wrong results or schema error when spark nested schema pruning is on for datasource v2

Pull Request - State: closed - Opened by yigress over 1 year ago - 8 comments

#970 - Support for partition overwrite of integer-based partitions

Issue - State: open - Opened by ngsalmon over 1 year ago - 1 comment
Labels: enhancement

#968 - Biqquery changes not reflecting in the Streaming Pipeline

Issue - State: closed - Opened by nishant-neo over 1 year ago - 3 comments

#967 - Spark Structured Streaming Query is failing with the following stracktrace

Issue - State: closed - Opened by nishant-neo over 1 year ago - 4 comments

#966 - Error due to `schema` mismatch while writing to `Big Query`

Issue - State: closed - Opened by gomrinal over 1 year ago - 3 comments

#965 - changing the hash from BigQueryConfig to BigQueryClientFactoryConfig

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 5 comments

#964 - BadRequest when performing merge after write in the same Spark session

Issue - State: open - Opened by khaledh over 1 year ago - 2 comments

#962 - Issue with timezones

Issue - State: closed - Opened by nikolay-voskresenskiy-db over 1 year ago - 1 comment

#961 - Switch off V1_BATCH_WRITE for dsv2

Pull Request - State: closed - Opened by yigress over 1 year ago - 1 comment

#960 - Add a configuration option to specify a static channel pool size

Pull Request - State: open - Opened by kmjung over 1 year ago - 2 comments

#959 - Issue with BigNumeric support - Serverless

Issue - State: open - Opened by karthikt16 over 1 year ago - 6 comments

#958 - fix for allowFieldAddition not working in indirect mode

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 3 comments

#957 - Direct write with subset field list support fix

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 11 comments

#955 - [BREAKING CHANGE] Numeric/Decimal updates

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 11 comments

#953 - Behavior difference in BigQuery vs Hive when changing schema of a table with saveMode=overwrite

Issue - State: closed - Opened by bbarodia over 1 year ago - 1 comment
Labels: duplicate

#952 - Issue #845: Upgrading auth library to version 1.16.1

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 1 comment

#951 - Creating BigQueryReadClient with UnboundedScheduledExecutorService

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 8 comments

#950 - Add support for service account impersonation

Pull Request - State: closed - Opened by jphalip over 1 year ago - 4 comments

#949 - Multiple sources found for bigquery

Issue - State: closed - Opened by wangyu0305 over 1 year ago - 8 comments

#948 - Removing Scala 2.11 connector from README

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#947 - Fix README-template

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#946 - Declaring DIRECT write method and DSv2 connectors as GA

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 5 comments

#945 - New Column addition in Spark Bigquery Connector 2.12_0.29.0 vs 2.12_0.20.0

Issue - State: closed - Opened by bbarodia over 1 year ago - 23 comments

#944 - Adding support to set query job priority

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 2 comments

#943 - Spark is always producing 1000 partitions when reading from GBQ

Issue - State: closed - Opened by mkvenkatesh over 1 year ago - 12 comments

#942 - Improving BigQuery's DateTime type support

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 1 comment

#941 - Updated dependency versions

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 2 comments

#939 - Using RangePartitioning possible?

Issue - State: open - Opened by dirk-gasser-db over 1 year ago - 1 comment

#937 - Any plan to support Spark 3.3 with Scala 2.12?

Issue - State: closed - Opened by aajisaka over 1 year ago - 2 comments

#935 - DIRECT write to partitioned table clarification

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#934 - Adding Spotless check GitHub workflow

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#933 - additional scan perf tuning.

Pull Request - State: closed - Opened by emkornfield over 1 year ago - 3 comments

#932 - Add support for table snapshots

Pull Request - State: closed - Opened by jphalip over 1 year ago - 5 comments

#931 - help: Writing data from databricks to google bigquery

Issue - State: closed - Opened by jeeva-98 over 1 year ago - 5 comments

#930 - Cannot write partitioned table with Orc intermediete files

Issue - State: open - Opened by mathnut02 over 1 year ago - 9 comments

#929 - Cannot change destination table schema when in overwrite mode

Issue - State: closed - Opened by mdesnoyer over 1 year ago - 3 comments
Labels: question

#928 - Create ReadSession asynchronously

Pull Request - State: closed - Opened by vishalkarve15 over 1 year ago - 2 comments

#927 - Cache ReadSession response

Pull Request - State: closed - Opened by vishalkarve15 over 1 year ago - 20 comments

#926 - How to get the cost of BQ query

Issue - State: closed - Opened by waltyou over 1 year ago - 1 comment

#925 - Add support for writing BigNumeric

Pull Request - State: closed - Opened by HaeSe0ng over 1 year ago - 3 comments

#924 - Allow maxParallelism to be specified via spark properties

Pull Request - State: closed - Opened by anoopj over 1 year ago - 2 comments

#923 - Adding SchemaConvertersConfiguration preparing for the datetime update.

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 3 comments

#922 - README clarification on preferredMinParallelism default behavior

Pull Request - State: closed - Opened by kristopherkane over 1 year ago - 4 comments

#921 - Remove one level of indirection in ArrowSchemaConverter.

Pull Request - State: closed - Opened by emkornfield over 1 year ago - 8 comments

#920 - Collapse ArrowSchemaConverter hierarchy

Issue - State: closed - Opened by emkornfield over 1 year ago

#919 - Adding Map type support documentation

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#918 - Making Spotbugs to fail on error

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 1 comment

#917 - Adding map type write support

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 11 comments

#915 - Issue #908: setting the preferred min parallelism to be not more than max parallelism

Pull Request - State: closed - Opened by suryasoma over 1 year ago - 4 comments

#914 - Adding map type read support

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago - 10 comments

#913 - Add support for passing configuration to custom AccessTokenProvider

Pull Request - State: closed - Opened by tom-s-powell over 1 year ago - 11 comments

#912 - Allow triggering bloom filter

Pull Request - State: closed - Opened by vishalkarve15 over 1 year ago - 4 comments

#911 - CHANGES update from release 0.28.1

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#910 - Allow spark DSV1 connector to reuse exchange for same table and pushdown filters

Pull Request - State: closed - Opened by vishalkarve15 over 1 year ago - 3 comments

#909 - Connector can silently fail to update table content

Issue - State: closed - Opened by dtraviszus over 1 year ago - 3 comments

#908 - GRPC error reading from Spark 3.1/Dataproc 2.0-debian10

Issue - State: closed - Opened by kristopherkane over 1 year ago - 11 comments

#907 - Fix runtime filtering by returning only partition,clustering fields

Pull Request - State: closed - Opened by vishalkarve15 over 1 year ago - 1 comment

#906 - Removing row number stats

Pull Request - State: closed - Opened by abmodi over 1 year ago - 1 comment

#905 - upcoming release documentation changes

Pull Request - State: closed - Opened by davidrabinowitz over 1 year ago

#904 - Fixing premature client closing in certain cases

Pull Request - State: closed - Opened by davidrabinowitz almost 2 years ago - 3 comments

#903 - Dependency upgrades

Pull Request - State: closed - Opened by davidrabinowitz almost 2 years ago - 3 comments

#901 - spark write fails on gcp-bq table save

Issue - State: closed - Opened by vidhyaguru almost 2 years ago - 9 comments