Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / Yelp/service_configuration_lib issues and pull requests

#144 - Update spark-run to no longer override --aws-profile

Pull Request - State: closed - Opened by choww 28 days ago - 2 comments

#143 - Remove Spark SignalFx dashboard URLs

Pull Request - State: closed - Opened by edingroot 4 months ago

#141 - Add unit tests for utils

Pull Request - State: closed - Opened by edingroot 4 months ago - 2 comments

#140 - Fix default user name label

Pull Request - State: closed - Opened by edingroot 4 months ago

#139 - MLCOMPUTE-1160 | consolidate tron and paasta logic to service config lib

Pull Request - State: closed - Opened by CaptainSame 5 months ago - 1 comment

#137 - Allow get_aws_credentials to assume_role_with_web_identity

Pull Request - State: closed - Opened by Qmando 5 months ago - 5 comments

#136 - Remove debian targets from Makefile

Pull Request - State: open - Opened by nemacysts 6 months ago

#135 - Limiting spark appname to 63 characters

Pull Request - State: closed - Opened by 88manpreet 8 months ago

#134 - Bump urllib3 from 1.26.15 to 1.26.18

Pull Request - State: open - Opened by dependabot[bot] 9 months ago
Labels: dependencies

#133 - MLCOMPUTE-1056 | Append a random postfix to the app name

Pull Request - State: closed - Opened by edingroot 9 months ago

#132 - Update spark job monitoring dashboard url

Pull Request - State: closed - Opened by edingroot 9 months ago

#131 - Pin types-requests to resolve dependency resolution error

Pull Request - State: closed - Opened by edingroot 9 months ago

#130 - Bump urllib3 from 1.26.15 to 1.26.17

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies

#129 - Show warning message when spark app id is explicitly specified

Pull Request - State: closed - Opened by edingroot 9 months ago

#128 - MLCOMPUTE-949 | Pick spark ui port from a preferred port range

Pull Request - State: closed - Opened by edingroot 10 months ago

#127 - MLCOMPUTE-746 | Remove the option for explicitly specifying app id

Pull Request - State: closed - Opened by edingroot 10 months ago

#126 - Support explicitly setting spark app id

Pull Request - State: closed - Opened by edingroot 10 months ago

#125 - Update spark app id naming convention and limit length

Pull Request - State: closed - Opened by edingroot 11 months ago

#124 - Setting spark.app.id to a more intuitive name

Pull Request - State: closed - Opened by edingroot 11 months ago

#123 - Add driver ui port executor pod label for prometheus metrics

Pull Request - State: closed - Opened by edingroot 11 months ago - 1 comment

#122 - Pick spark ui port with preference and add user pod label

Pull Request - State: closed - Opened by edingroot 11 months ago

#121 - Fix spark master assignment

Pull Request - State: closed - Opened by edingroot 11 months ago - 1 comment

#120 - Add error message when using spark-run with assume-aws-role

Pull Request - State: closed - Opened by Qmando 12 months ago

#119 - Disable decommission for non-spot pools

Pull Request - State: open - Opened by 88manpreet 12 months ago - 1 comment

#118 - MLCOMPUTE-951 | Refactor and make the spark_config modular

Pull Request - State: closed - Opened by CaptainSame 12 months ago - 1 comment

#116 - Enable prometheus metrics export on driver's endpoint

Pull Request - State: closed - Opened by edingroot about 1 year ago - 1 comment

#115 - Add support for --use-eks in PaaSTA

Pull Request - State: closed - Opened by nemacysts about 1 year ago

#114 - Add grafana spark dashboard link

Pull Request - State: closed - Opened by edingroot about 1 year ago

#113 - Fix service_configuration_lib dependency conflict bug

Pull Request - State: closed - Opened by 88manpreet about 1 year ago - 1 comment

#112 - MLCOMPUTE-699 | Mandatory spark srv config defaults

Pull Request - State: closed - Opened by CaptainSame about 1 year ago

#111 - Change assume_aws_role to use user credentials rather than web identity token

Pull Request - State: closed - Opened by Qmando about 1 year ago - 2 comments

#110 - MLCOMPUTE-707 | fix unreadable formatting in costs log

Pull Request - State: closed - Opened by CaptainSame about 1 year ago - 1 comment

#109 - Remove showConsole flag

Pull Request - State: closed - Opened by 88manpreet over 1 year ago

#107 - Update spark.cores.max after resource adjustment

Pull Request - State: closed - Opened by edingroot over 1 year ago

#106 - Update logic to include 56g executors into 56g bucket

Pull Request - State: closed - Opened by edingroot over 1 year ago

#105 - Add ability to do an assume-role-with-web-identity call for spark

Pull Request - State: closed - Opened by Qmando over 1 year ago - 1 comment

#104 - Load spark constants from srv-configs

Pull Request - State: closed - Opened by edingroot over 1 year ago

#103 - MLCOMPUTE-595 | remove DRA disable suggestion for notebooks

Pull Request - State: closed - Opened by CaptainSame over 1 year ago - 1 comment

#102 - Release adjust spark executor resources for mem up to 14g

Pull Request - State: closed - Opened by edingroot over 1 year ago

#101 - Fix k8s adjusted executor cores limit

Pull Request - State: closed - Opened by edingroot over 1 year ago

#100 - MLCOMPUTE-548 | enabled DRA by default for Jupyter Spark sessions

Pull Request - State: closed - Opened by CaptainSame over 1 year ago

#99 - Upgrade from py36 to py37 to fix CI test error

Pull Request - State: closed - Opened by edingroot over 1 year ago - 3 comments

#98 - Change executor memory cap to 110g

Pull Request - State: closed - Opened by edingroot over 1 year ago

#97 - Cap executor resource with instance count adjustment

Pull Request - State: closed - Opened by edingroot over 1 year ago - 1 comment

#96 - Fix pre-commit config

Pull Request - State: closed - Opened by edingroot over 1 year ago - 1 comment

#95 - Cap spark executor resource limits - Part1

Pull Request - State: closed - Opened by edingroot over 1 year ago - 1 comment

#93 - Remove console flag to reduce logging noise

Pull Request - State: open - Opened by 88manpreet over 1 year ago

#92 - Remove showConsoleProgress flag

Pull Request - State: open - Opened by 88manpreet over 1 year ago - 1 comment

#90 - CLUSTERMAN-710, bump version service_configuration_lib

Pull Request - State: closed - Opened by akshaysharma096 over 1 year ago

#89 - COMPINFRA-1592, Add labels to spark pods

Pull Request - State: closed - Opened by akshaysharma096 over 1 year ago - 1 comment

#87 - adding PAASTA_SOA_DIR env var to set custom yelpsoa-configs directory

Pull Request - State: open - Opened by cuza almost 2 years ago

#86 - Update signalfx link to use y/spark-metrics

Pull Request - State: closed - Opened by 88manpreet almost 2 years ago

#85 - make Spark on Mesos use old spark shuffle fetch protocol

Pull Request - State: closed - Opened by nanflasted almost 2 years ago

#83 - Add cluster-manager=local support.

Pull Request - State: closed - Opened by jsleight almost 2 years ago - 2 comments

#81 - MLCOMPUTE-23 | Enable DRA using recommended configs

Pull Request - State: closed - Opened by CaptainSame about 2 years ago - 1 comment

#79 - Coreml 3238 | set k8s allocation batch size accurately

Pull Request - State: closed - Opened by CaptainSame about 2 years ago

#78 - Set AWS creds via Spark Config

Pull Request - State: closed - Opened by jsleight about 2 years ago - 3 comments

#77 - Make api for spark hourly cost public

Pull Request - State: closed - Opened by 88manpreet about 2 years ago

#75 - COREML-3036 | Add Spark conf log and console progress config

Pull Request - State: closed - Opened by CaptainSame over 2 years ago - 1 comment

#74 - [COREML-2697] handle missing kubernetes config values for mass migration

Pull Request - State: closed - Opened by gcollura over 2 years ago - 2 comments

#73 - Add func to read soa-configs metadata file

Pull Request - State: closed - Opened by kawaiwanyelp over 2 years ago

#71 - [COREML-2694] automatically compute and set k8s executor limit cores and batch size

Pull Request - State: closed - Opened by gcollura over 2 years ago - 1 comment

#70 - [COREML-2591] Fix issue with long label names in k8s

Pull Request - State: closed - Opened by gcollura almost 3 years ago - 1 comment

#69 - Creating TypeDict for ServiceInfo

Pull Request - State: closed - Opened by cuza almost 3 years ago - 1 comment

#68 - [COREML-2570] fix k8s volumes being mounted twice

Pull Request - State: closed - Opened by gcollura almost 3 years ago - 2 comments

#67 - Release v2.5.7 to trigger new build on GHA

Pull Request - State: closed - Opened by gcollura almost 3 years ago

#66 - Add GitHub actions

Pull Request - State: closed - Opened by gcollura almost 3 years ago

#65 - Change url for k8s cluster to be used for spark master

Pull Request - State: closed - Opened by 88manpreet almost 3 years ago - 1 comment

#64 - Update spark_config to handle k8s container required volumes for paasta spark-run

Pull Request - State: closed - Opened by gcollura almost 3 years ago - 3 comments

#63 - Include 'autotuned_defaults' in the list of folders to watch

Pull Request - State: closed - Opened by thejoycekung almost 3 years ago

#62 - use spark.mesos.executor.memoryOverhead for memory calculation as well

Pull Request - State: closed - Opened by talalryz over 3 years ago - 1 comment

#61 - Bump to Version 2.5.4

Pull Request - State: closed - Opened by smrojas over 3 years ago - 1 comment

#59 - Remove production target

Pull Request - State: closed - Opened by nemacysts over 3 years ago

#58 - Pass session.token When Using TemporaryAWSCredentialsProvider

Pull Request - State: closed - Opened by smrojas over 3 years ago

#57 - Do not respond to the production target

Pull Request - State: open - Opened by tzhu-yelp over 3 years ago

#56 - ensure user_spark_opts value is str type

Pull Request - State: closed - Opened by lydian over 3 years ago

#55 - seperate find_spark_master function

Pull Request - State: closed - Opened by lydian over 3 years ago

#54 - combined spark functions from paasta and spark_tools

Pull Request - State: closed - Opened by lydian over 3 years ago - 1 comment

#52 - release 2.4.6

Pull Request - State: closed - Opened by chrislan815 about 4 years ago

#51 - u/clan/BATCHPROC-92-update-spark-k8s-namespace

Pull Request - State: closed - Opened by chrislan815 about 4 years ago

#50 - Set label and node-selector for kubernetes pods for spark jobs

Pull Request - State: closed - Opened by 88manpreet about 4 years ago - 1 comment

#48 - Add parameter to read_extra_service_information that allows the user …

Pull Request - State: closed - Opened by EvanKrall about 4 years ago

#47 - Add new style labels to Spark executors in K8s

Pull Request - State: closed - Opened by nisan1101 about 4 years ago

#46 - Release 2.4.2

Pull Request - State: closed - Opened by nisan1101 about 4 years ago

#45 - Add labels for spark executors running in K8s

Pull Request - State: closed - Opened by nisan1101 about 4 years ago - 4 comments

#44 - Fix kubernetes volume name format

Pull Request - State: closed - Opened by jangsutsr over 4 years ago - 1 comment