substratusai/kubeai issues and pull requests

#333 - Cache optimized routing ("PrefixHash" load balancing - i.e. CHWBL)

Pull Request - State: closed - Opened by nstogner 3 months ago

#332 - Bump github.com/onsi/ginkgo/v2 from 2.17.1 to 2.22.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#331 - Bump go.opentelemetry.io/otel/exporters/prometheus from 0.52.0 to 0.54.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#330 - Bump gocloud.dev/pubsub/natspubsub from 0.39.0 to 0.40.0

Pull Request - State: open - Opened by dependabot[bot] 3 months ago
Labels: dependencies, go

#329 - Bump k8s.io/client-go from 0.30.1 to 0.31.3

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#328 - bump openai python client in e2e test

Pull Request - State: closed - Opened by samos123 3 months ago

#327 - Bump actions/checkout from 2 to 4 in the actions-all group

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, github_actions

#326 - add nvidia-gpu-rtx4070-8gb and qwen2.5 models

Pull Request - State: closed - Opened by samos123 3 months ago - 1 comment

#325 - Fix formatting for docs

Pull Request - State: closed - Opened by samos123 3 months ago

#324 - Bump golang from 1.22 to 1.23

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 2 comments
Labels: dependencies, docker

#323 - Bump k8s.io/apimachinery from 0.30.1 to 0.31.3

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#322 - Bump gocloud.dev/pubsub/rabbitpubsub from 0.39.0 to 0.40.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#321 - Bump go.opentelemetry.io/otel/exporters/stdout/stdoutlog from 0.6.0 to 0.8.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#320 - Bump github.com/onsi/gomega from 1.32.0 to 1.36.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#319 - Bump gocloud.dev/pubsub/kafkapubsub from 0.39.0 to 0.40.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies, go

#318 - Bump gocloud.dev from 0.39.0 to 0.40.0

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 3 comments
Labels: dependencies, go

#317 - Add initial dependabot config

Pull Request - State: closed - Opened by alpe 3 months ago - 1 comment

#316 - Add SECURITY.md file

Issue - State: open - Opened by alpe 3 months ago

#315 - update appVersion to v0.11.0 and bump chart versions

Pull Request - State: closed - Opened by samos123 3 months ago

#315 - update appVersion to v0.11.0 and bump chart versions

Pull Request - State: closed - Opened by samos123 3 months ago

#314 - Proposal: Cache-optimized routing

Pull Request - State: closed - Opened by nstogner 3 months ago - 2 comments

#313 - Add Configure Text Generation Models guide

Pull Request - State: closed - Opened by samos123 3 months ago

#313 - Add Configure Text Generation Models guide

Pull Request - State: closed - Opened by samos123 3 months ago

#312 - add a generic K8s install guide

Pull Request - State: closed - Opened by samos123 3 months ago

#312 - add a generic K8s install guide

Pull Request - State: closed - Opened by samos123 3 months ago

#311 - Proposal: Mount a PVC in ReadManyOnly mode as model storage

Issue - State: open - Opened by samos123 3 months ago - 6 comments

#310 - update vllm image for GPU and TPU to v0.6.4.post1

Pull Request - State: closed - Opened by samos123 3 months ago - 2 comments

#310 - update vllm image for GPU and TPU to v0.6.4.post1

Pull Request - State: closed - Opened by samos123 3 months ago - 2 comments

#309 - Add Lambda's tutorial and video to the README's table of adopters

Pull Request - State: closed - Opened by cbrownstein-lambda 3 months ago - 1 comment

#308 - add k8s device plugin / GPU operator values file

Pull Request - State: closed - Opened by samos123 4 months ago

#307 - Llama 3.1 70b with pipeline parallelism

Pull Request - State: closed - Opened by samos123 4 months ago

#306 - is kubeai support multimodal model？

Issue - State: closed - Opened by qiankunli 4 months ago - 2 comments

#305 - Update README.md

Pull Request - State: closed - Opened by samos123 4 months ago

#304 - LoRA Adapters for vLLM & support for s3, gs, oss for pulling adapters and models (to cache) from buckets

Pull Request - State: closed - Opened by nstogner 4 months ago - 10 comments

#303 - how to access model files by pvc?

Issue - State: closed - Opened by qiankunli 4 months ago - 9 comments

#302 - add llama 3.1 70b fp8 model on 1 x gh200

Pull Request - State: closed - Opened by samos123 4 months ago

#301 - Consider supporting a reranker model?

Issue - State: open - Opened by qiankunli 4 months ago - 7 comments

#300 - Add gh200 support and model

Pull Request - State: closed - Opened by happytreees 4 months ago

#299 - Is there a way to have performance metrics when running kubeai?

Issue - State: open - Opened by strus38 4 months ago - 3 comments
Labels: good first issue

#298 - Dynamically calculate PVC request sizes or remove, or document if it can be ignored

Issue - State: open - Opened by nstogner 4 months ago - 2 comments

#297 - add ollama caching

Issue - State: open - Opened by kaiehrhardt 4 months ago - 4 comments

#296 - update README

Pull Request - State: closed - Opened by samos123 4 months ago

#295 - improve caching docs

Pull Request - State: closed - Opened by samos123 4 months ago

#294 - Deep Chat integration

Pull Request - State: closed - Opened by nstogner 4 months ago

#293 - Add guide for using DeepChat

Issue - State: closed - Opened by nstogner 4 months ago

#291 - helm: bump chartVersions and appVersion to v0.10.0

Pull Request - State: closed - Opened by samos123 4 months ago

#290 - Update kubernetes api reference

Pull Request - State: closed - Opened by samos123 4 months ago

#289 - add caching models with EFS guide

Pull Request - State: closed - Opened by samos123 4 months ago

#288 - increase caching e2e test timeout

Pull Request - State: closed - Opened by samos123 4 months ago

#287 - Add EKS Installation Guide

Pull Request - State: closed - Opened by samos123 4 months ago

#286 - KubeAI stuck when configuring model caching with missing storageclass

Issue - State: open - Opened by samos123 4 months ago - 1 comment

#285 - Look into decommissioning embedded Open WebUI chart in favor of the upstream chart

Issue - State: closed - Opened by nstogner 4 months ago

#284 - add kubeai metrics service endpoint

Pull Request - State: closed - Opened by kaiehrhardt 4 months ago

#283 - OpenAI Server incompatible with OpenAI API when streaming and requesting usage

Issue - State: closed - Opened by tpfau 4 months ago - 9 comments

#282 - Add support for HTTP X-Label-Selector headers to support Multitenancy

Pull Request - State: closed - Opened by nstogner 4 months ago

#281 - Adding Build WF timeout to address stuck WF's

Pull Request - State: closed - Opened by Sudhamsh 4 months ago - 2 comments

#280 - helm: update appVersion to v0.9.0

Pull Request - State: closed - Opened by samos123 4 months ago

#279 - add manual test of vLLM on GPU and TPU

Pull Request - State: closed - Opened by samos123 4 months ago

#278 - Security context not set on Infinity engine

Issue - State: open - Opened by nstogner 4 months ago

#277 - Optimize ollama startup probe

Issue - State: open - Opened by nstogner 4 months ago

#276 - Update the huggingface model loader image to use official built version

Issue - State: open - Opened by nstogner 4 months ago

#275 - Add e2e test for Infinity engine

Issue - State: open - Opened by nstogner 4 months ago

#274 - Make model cache loading more efficient

Issue - State: open - Opened by nstogner 4 months ago

#273 - update vllm images to 0.6.3

Pull Request - State: closed - Opened by samos123 4 months ago

#272 - Shared filesystem caching

Pull Request - State: closed - Opened by nstogner 4 months ago - 2 comments
Labels: enhancement

#271 - add tpu quota to GKE install guide and use values-gke.yaml

Pull Request - State: closed - Opened by samos123 4 months ago

#270 - Failed to aggregate stats: failed to parse metrics: text format parsing error in line 1: invalid metric name

Issue - State: closed - Opened by kaiehrhardt 4 months ago - 3 comments

#269 - vllm CPU incorporate performance tips in reference models

Issue - State: open - Opened by samos123 5 months ago

#268 - Add Autoscaler State ConfigMap

Pull Request - State: closed - Opened by nstogner 5 months ago - 1 comment

#267 - Support distributed inference

Issue - State: open - Opened by nstogner 5 months ago - 2 comments

#266 - Route requests to take advantage of prefix caching

Issue - State: open - Opened by nstogner 5 months ago - 6 comments

#265 - Autoscaler should maintain state across KubeAI restarts & crashes

Issue - State: closed - Opened by nstogner 5 months ago

#264 - add resourceProfiles and 405b on A100 80GB

Pull Request - State: closed - Opened by samos123 5 months ago

#263 - Refactor e2e tests

Pull Request - State: closed - Opened by nstogner 5 months ago - 3 comments

#262 - Remove request body logging

Issue - State: closed - Opened by nstogner 5 months ago

#261 - Autoscale based on KubeAI OpenTelemetry active requests metrics

Pull Request - State: closed - Opened by nstogner 5 months ago

#260 - helm: update charts and kubeai appVersion v0.8.0

Pull Request - State: closed - Opened by samos123 5 months ago

#259 - Support for multiple slash

Pull Request - State: open - Opened by samos123 5 months ago - 3 comments

#258 - Llama 3.2 11B Instruct vision on 1 x L4 GPU

Pull Request - State: closed - Opened by samos123 5 months ago

#257 - issue with parsing model from json when using multiple / in the path

Issue - State: open - Opened by samos123 5 months ago - 10 comments
Labels: good first issue

#256 - Support for backing a single model with multiple models

Issue - State: open - Opened by samos123 5 months ago

#255 - Add example: python Models client

Pull Request - State: closed - Opened by nstogner 5 months ago - 1 comment

#254 - add llama 3.1 405b model

Pull Request - State: closed - Opened by samos123 5 months ago

#253 - Add runtimeClassName as optional field in resource profile

Pull Request - State: closed - Opened by nstogner 5 months ago

#252 - Add guide on how to expose KubeAI Server and OpenWebUI via Ingress

Issue - State: open - Opened by nstogner 5 months ago - 5 comments

#251 - Write guide on how to benchmark vLLM / OpenAI compatible LLM engines

Issue - State: open - Opened by samos123 5 months ago

#250 - Some CPU models broken on GKE Autopilot - missing ephemeral-storage reqs/limits

Issue - State: open - Opened by nstogner 5 months ago
Labels: bug

#249 - Initial TPU support

Pull Request - State: closed - Opened by nstogner 5 months ago

#248 - rough min model replicas

Pull Request - State: closed - Opened by sam-huang1223 5 months ago - 1 comment

#247 - provide way to specify minimum model replicas available

Issue - State: open - Opened by sam-huang1223 5 months ago - 1 comment

#246 - fix huggingface secret helm template issue

Pull Request - State: closed - Opened by samos123 5 months ago

#245 - helm: bump models chart version

Pull Request - State: closed - Opened by samos123 5 months ago

#244 - helm: bump appVersion v0.7.0

Pull Request - State: closed - Opened by samos123 5 months ago

#243 - Ability to provide chat templates to vLLM

Issue - State: closed - Opened by samos123 5 months ago - 3 comments
Labels: enhancement

#242 - Provide tutorial and example for using Vision model like Pixtral

Issue - State: closed - Opened by samos123 5 months ago

#241 - Improved Pod Managment

Pull Request - State: closed - Opened by nstogner 5 months ago

#240 - fix #235 utilize standard k8s labels

Pull Request - State: closed - Opened by samos123 5 months ago

#239 - fix: #231 Print target requests value

Pull Request - State: closed - Opened by benz9527 5 months ago - 1 comment

#238 - make test occasionally flaky on local machine

Issue - State: open - Opened by samos123 5 months ago

#237 - autoscaling support for Infinity

Issue - State: closed - Opened by samos123 5 months ago
Labels: enhancement

GitHub / substratusai/kubeai issues and pull requests