oap-project/raydp issues and pull requests

#417 - fix CVE

Pull Request - State: open - Opened by minmingzhu 6 days ago

#416 - add support of init_spark from existing SparkSession?

Issue - State: open - Opened by zhh210 7 days ago

#415 - Slack Group for the commiters & maintainers

Issue - State: open - Opened by mehulbatra-d11 13 days ago - 4 comments

#414 - Can you pass spark.driver.extraClassPath directly into raydp.init()?

Issue - State: open - Opened by tanderson10 26 days ago

#413 - Add Instructions for Ray Jobs with PyModules

Pull Request - State: open - Opened by noahshpak 2 months ago

#412 - RayDP support for spark-connect

Issue - State: open - Opened by aawasthid11 3 months ago - 1 comment

#411 - Fix Spark 3.5.x support + support Spark 3.4.3, deprecate python 3.7+support python 3.10

Pull Request - State: closed - Opened by pang-wu 4 months ago

#410 - Samsara/1.6.0

Pull Request - State: closed - Opened by pang-wu 4 months ago

#410 - Samsara/1.6.0

Pull Request - State: closed - Opened by pang-wu 4 months ago

#409 - incompatible with spark 3.4.3 and 3.5.0

Issue - State: open - Opened by wuxiaocheng0506 4 months ago - 4 comments

#408 - how to init raydp resource

Issue - State: open - Opened by yasaorder 4 months ago - 1 comment

#407 - moved the requirements for python from setup.py to requirements.txt

Pull Request - State: open - Opened by akashd11 5 months ago

#406 - moved the requirements for python from setup.py to requirements.txt

Pull Request - State: closed - Opened by akashd11 5 months ago

#405 - How to develop in intellij?

Issue - State: open - Opened by akashd11 5 months ago

#404 - raydp.init_spark fails

Issue - State: open - Opened by Mayurji 6 months ago - 1 comment

#403 - Fix CI with latest ray

Pull Request - State: closed - Opened by kira-lin 6 months ago

#402 - install raydp==1.6.0 with pip rasied the "More than one SparkShimProvider found" error

Issue - State: open - Opened by evilpapa 7 months ago

#401 - fix CVE issue

Pull Request - State: closed - Opened by minmingzhu 7 months ago

#400 - fix openssf issue

Pull Request - State: closed - Opened by minmingzhu 7 months ago

#399 - Fix CVE issues

Issue - State: closed - Opened by minmingzhu 7 months ago

#398 - Fix OpenSSF issues

Issue - State: closed - Opened by minmingzhu 7 months ago

#397 - Support Spark 3.4.2, lift pyspark dependency range

Pull Request - State: closed - Opened by pang-wu 10 months ago - 2 comments

#396 - dynamic allocation executor pending for addition flood fix

Pull Request - State: closed - Opened by raviranak 10 months ago - 4 comments

#395 - Spark 3.5.0 support on raydp

Pull Request - State: closed - Opened by raviranak 10 months ago - 1 comment

#394 - Is it possible for raydp to utilize a Spark session created by Apache Livy?

Issue - State: open - Opened by Taurus-Le 10 months ago - 1 comment

#393 - [stream] does raydp support spark stream data?

Issue - State: open - Opened by fwensen 10 months ago - 1 comment

#392 - Update raydp-submit

Pull Request - State: closed - Opened by gptbert 10 months ago - 1 comment

#391 - Changes for executor actor recovery using ray's fault tolerance.

Pull Request - State: closed - Opened by KiranP-d11 10 months ago - 2 comments

#390 - RayDP Docker Image with Python 3.10 Failing to Build

Issue - State: closed - Opened by dattasoumya92 11 months ago - 2 comments

#389 - Recovery for lost executors 364

Pull Request - State: closed - Opened by KiranP-d11 11 months ago - 9 comments

#388 - upgrade Checkpoint usage to fit ray 3.0

Pull Request - State: closed - Opened by harborn 11 months ago - 1 comment

#387 - Feat/recover lost executors

Pull Request - State: closed - Opened by KiranP-d11 11 months ago - 2 comments

#387 - Feat/recover lost executors

Pull Request - State: closed - Opened by KiranP-d11 11 months ago - 2 comments

#386 - Issue with Running Multiple Spark Jobs Concurrently on Ray Cluster

Issue - State: closed - Opened by DimitarSirakov 12 months ago - 1 comment

#385 - import IPEX in torch config

Pull Request - State: closed - Opened by harborn 12 months ago - 1 comment

#384 - Upgrade snappy version to fix CVE issue

Pull Request - State: closed - Opened by Deegue 12 months ago - 1 comment

#384 - Upgrade snappy version to fix CVE issue

Pull Request - State: closed - Opened by Deegue 12 months ago - 1 comment

#383 - Getting Java gateway process exited before sending its port number during init_spark()

Issue - State: open - Opened by swapkh91 12 months ago - 11 comments

#383 - Getting Java gateway process exited before sending its port number during init_spark()

Issue - State: open - Opened by swapkh91 12 months ago - 9 comments

#382 - Support Spark 3.4.1 and 3.3.3

Pull Request - State: closed - Opened by pang-wu 12 months ago - 1 comment

#382 - Support Spark 3.4.1 and 3.3.3

Pull Request - State: closed - Opened by pang-wu 12 months ago - 1 comment

#381 - Bump org.xerial.snappy:snappy-java from 1.1.10.1 to 1.1.10.4 in /core

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies, java

#381 - Bump org.xerial.snappy:snappy-java from 1.1.10.1 to 1.1.10.4 in /core

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies, java

#380 - Runtime env_vars cannot be passed down to executors

Issue - State: open - Opened by Deegue about 1 year ago

#380 - Runtime env_vars cannot be passed down to executors

Issue - State: open - Opened by Deegue about 1 year ago - 1 comment

#379 - Update ivy and spark base version

Pull Request - State: closed - Opened by Deegue about 1 year ago

#378 - Bump org.apache.ivy:ivy from 2.5.1 to 2.5.2 in /core

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies, java

#377 - Bump org.apache.spark:spark-core_2.12 from 3.2.2 to 3.3.3 in /core

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies, java

#376 - Added possibility to set custom actor owner when convert spark dataframe to ray dataset

Pull Request - State: closed - Opened by max-509 about 1 year ago - 3 comments

#375 - Add the ability to set the owner of objects when calling _save_spark_df_to_object_store()

Issue - State: closed - Opened by max-509 about 1 year ago - 1 comment

#374 - Bump to 1.7.0

Pull Request - State: closed - Opened by kira-lin about 1 year ago

#373 - Fix Jackson and Scala vulnerabilities

Pull Request - State: closed - Opened by Deegue about 1 year ago

#372 - When submit a script another process start locally in the head node

Issue - State: open - Opened by gbraes about 1 year ago - 1 comment

#371 - Support ray 2.3.1

Issue - State: open - Opened by fengsp about 1 year ago - 3 comments

#370 - [Follow-up] Use OutputStreamWriter instead of FileWriter

Pull Request - State: closed - Opened by Deegue about 1 year ago

#369 - [Follow-up] Add charset to getBytes()

Pull Request - State: closed - Opened by Deegue about 1 year ago

#368 - [Follow-up] Fix encoding in Agent.java

Pull Request - State: closed - Opened by Deegue about 1 year ago

#367 - Fix static code scan

Pull Request - State: closed - Opened by Deegue about 1 year ago

#366 - Implement Spark executor node affinity using custom resource

Pull Request - State: closed - Opened by pang-wu about 1 year ago - 7 comments

#365 - Use environment variable in maven to build

Pull Request - State: open - Opened by kira-lin about 1 year ago

#364 - Executor lost triggered by JVM OOM can't recover and cause subsequent steps stuck

Issue - State: closed - Opened by pang-wu about 1 year ago - 12 comments

#363 - [Follow-up] Fix log4j and some dependencies for `shims-common` (#362)

Pull Request - State: closed - Opened by kira-lin about 1 year ago

#362 - [Follow-up] Fix log4j and some dependencies for `shims-common`

Pull Request - State: closed - Opened by Deegue about 1 year ago - 2 comments

#361 - Merge master branch into 1.6 release

Pull Request - State: closed - Opened by kira-lin about 1 year ago

#360 - [Follow-up] Fix maven dependency and prepare for release

Pull Request - State: closed - Opened by Deegue about 1 year ago

#359 - Merge changes to branch 1.6

Pull Request - State: closed - Opened by kira-lin about 1 year ago

#358 - Use LoggerFactory for agent and add pydantic

Pull Request - State: closed - Opened by Deegue about 1 year ago

#357 - raydp start occur null error

Issue - State: open - Opened by sydt2014 over 1 year ago - 1 comment

#356 - Fix maven dependency and prepare for release

Pull Request - State: closed - Opened by Deegue over 1 year ago

#355 - Fix unbound error in _SparkContext for int memory

Pull Request - State: closed - Opened by guseggert over 1 year ago - 1 comment

#354 - Update RayDP to be compatible with Ray 2.5.0

Pull Request - State: open - Opened by kira-lin over 1 year ago - 1 comment

#353 - fixed some UT errors

Pull Request - State: closed - Opened by jiafuzha over 1 year ago

#352 - Issues with running on Ray Client

Issue - State: open - Opened by wctmanager over 1 year ago - 3 comments

#351 - Fix: Ray nightly now uses its own schema type

Pull Request - State: closed - Opened by kira-lin over 1 year ago

#350 - raydp.init_spark fails

Issue - State: open - Opened by wctmanager over 1 year ago - 3 comments

#349 - Fix bug: to_spark will fail if number of executors > 1 when block format is bytes

Pull Request - State: closed - Opened by kira-lin over 1 year ago

#348 - Threads hang on getting from object store when calling `to_spark`

Issue - State: open - Opened by Deegue over 1 year ago

#347 - enable_hive not working with raydp

Issue - State: closed - Opened by raviranak over 1 year ago - 24 comments

#346 - Wanted to configure spark to use mysql as metastore instead of derby hive metastore which is default

Issue - State: closed - Opened by raviranak over 1 year ago - 5 comments

#345 - Update k8s related instructions

Pull Request - State: closed - Opened by kira-lin over 1 year ago

#344 - Skip ownership transfer test when using ray client to be compatible with Ray 2.4.0

Pull Request - State: closed - Opened by kira-lin over 1 year ago

#343 - Data ownership transfer tests fail in Ray 2.4.0

Issue - State: open - Opened by kira-lin over 1 year ago

#342 - Add shim layer for Spark 3.4.0 and 3.2.4

Pull Request - State: closed - Opened by kira-lin over 1 year ago - 2 comments

#341 - Support arrow zerocopy for reader and writer in object store

Pull Request - State: open - Opened by Deegue over 1 year ago

#340 - Performance benchmark on RayDP v.s. Spark

Issue - State: open - Opened by chenya-zhang over 1 year ago - 3 comments

#339 - Support/Validate Ray 2.4.0 Release

Issue - State: open - Opened by carsonwang over 1 year ago

#338 - Add shim layer for Spark 3.4.x

Issue - State: closed - Opened by kira-lin over 1 year ago

#337 - Bump spark-core_2.12 from 3.2.1 to 3.4.0 in /core

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies, java

GitHub / oap-project/raydp issues and pull requests