kserve/kserve issues and pull requests

#3964 - bump to vllm0.6.2

Pull Request - State: open - Opened by hustxiayang 3 days ago - 1 comment

#3963 - Add Support or Documentation for Running Hugging Face/vLLM Server on AMD GPUs

Issue - State: open - Opened by boris-toninski-prime 3 days ago

#3962 - how to change containruntime image

Issue - State: closed - Opened by SZ789 3 days ago

#3961 - install lmcache_vllm

Pull Request - State: closed - Opened by wengyao04 5 days ago

#3960 - Install lmcache_vllm

Pull Request - State: closed - Opened by wengyao04 5 days ago

#3959 - install llmcache-vllm

Pull Request - State: closed - Opened by wengyao04 5 days ago

#3956 - It failed to perform inference of sklearn using raw deployment

Issue - State: open - Opened by houshengbo 6 days ago - 6 comments

#3955 - Add NodeDownloadPending status to ClusterLocalModel

Pull Request - State: closed - Opened by greenmoon55 6 days ago - 1 comment
Labels: lgtm, approved

#3954 - add tags to rest server timing logs to differentiate cpu and wall time

Pull Request - State: closed - Opened by gfkeith 6 days ago - 2 comments
Labels: lgtm, approved

#3952 - Gateway API Migration - Raw Deployment

Pull Request - State: open - Opened by sivanantha321 7 days ago

#3948 - bump to vLLM0.6.1post2

Pull Request - State: open - Opened by hustxiayang 10 days ago

#3947 - Allow for unbound resource requests

Issue - State: open - Opened by michaeltinsley 10 days ago - 2 comments

#3946 - Deploying Llama3 model using Huggingface predictor fails

Issue - State: open - Opened by surajssd 10 days ago

#3945 - Support serving multiple models with Triton In process API

Issue - State: open - Opened by sivanantha321 11 days ago - 1 comment

#3944 - Fix ingress configuration for path based routing and update go mod

Pull Request - State: closed - Opened by sivanantha321 11 days ago - 1 comment
Labels: lgtm, approved

#3943 - Add HostIPC field to ServingRuntimePodSpec

Pull Request - State: open - Opened by greenmoon55 13 days ago - 5 comments

#3943 - Add HostIPC field to ServingRuntimePodSpec

Pull Request - State: open - Opened by greenmoon55 13 days ago - 5 comments

#3942 - Direct PVC mount does not support pvc path for a single file

Issue - State: open - Opened by greenmoon55 13 days ago

#3941 - remove conversion wehbook part from self-signed-ca.sh

Pull Request - State: open - Opened by Jooho 14 days ago - 3 comments

#3940 - Support configured metadata for logs

Issue - State: open - Opened by benkawecki 14 days ago

#3939 - Log Sampling

Issue - State: open - Opened by benkawecki 14 days ago

#3938 - Combined Request/Response Logging

Issue - State: open - Opened by benkawecki 14 days ago

#3937 - Push example modelcar image to a public registry

Issue - State: open - Opened by rhuss 14 days ago

#3936 - Getting 404 while invoking a sample Inference svc based on modelformat tensorflow

Issue - State: open - Opened by sonu1311 18 days ago - 1 comment

#3935 - Error: kserve with transformer pull image from private habor

Issue - State: closed - Opened by johnzheng1975 18 days ago - 2 comments

#3934 - bump to vllm 0.6.0

Pull Request - State: closed - Opened by hustxiayang 20 days ago - 5 comments
Labels: lgtm, approved

#3933 - For vllm users, our parser should be able to support both - and _

Pull Request - State: open - Opened by hustxiayang 20 days ago - 2 comments

#3932 - Modelcar race condition mitigation with an init container

Pull Request - State: open - Opened by israel-hdez 20 days ago - 6 comments

#3931 - /v2/health/ready endpoint does not work as expected on multiple models

Issue - State: open - Opened by beratturan 21 days ago

#3930 - fix: orphan cluster role kserve-addressable-resolver

Pull Request - State: open - Opened by zhangyue110026 22 days ago - 4 comments

#3929 - Orphan cluster role kserve-addressable-resolver deployed

Issue - State: open - Opened by zhangyue110026 22 days ago

#3928 - Follow security best practice for kserve serving runtime pods

Issue - State: closed - Opened by yuzisun 23 days ago
Labels: kserve/security

#3927 - Modelcar: Add no-op init-container to pre-pull model image

Issue - State: open - Opened by rhuss 24 days ago - 3 comments

#3926 - Minikube pod keeps restarting

Issue - State: open - Opened by serdar-akyol 24 days ago - 2 comments

#3925 - Enhance documentation: modelserving/storage/s3/s3

Issue - State: open - Opened by SeraphicRav 24 days ago - 2 comments

#3924 - Fix update-openapigen.sh that can be executed from kserve dir

Pull Request - State: open - Opened by Jooho 25 days ago - 3 comments

#3923 - Fail to execute make generate on local

Issue - State: open - Opened by Jooho 25 days ago - 1 comment

#3922 - "Failed to start model server: integer division or modulo by zero" even when allocating GPUs for InferenceService

Issue - State: closed - Opened by vanakema 25 days ago - 2 comments

#3921 - pin gosec to 2.20.0

Pull Request - State: closed - Opened by greenmoon55 25 days ago - 1 comment
Labels: lgtm, approved

#3920 - go-security-scan fails

Issue - State: closed - Opened by greenmoon55 25 days ago

#3919 - Kserve Logging Discrepancy

Issue - State: open - Opened by u6yuvi 25 days ago

#3918 - Add ppc64le platform support to Storage Initializer

Pull Request - State: open - Opened by bhagyashrigai 25 days ago - 4 comments

#3917 - Add security best practices for inferenceservice, inferencegraph, servingruntimes

Pull Request - State: closed - Opened by sivanantha321 25 days ago - 2 comments
Labels: lgtm, approved

#3916 - SCA / Dependency Management

Issue - State: open - Opened by scinerio 27 days ago - 1 comment

#3915 - Create dependabot.yml

Pull Request - State: closed - Opened by scinerio 27 days ago

#3914 - Diffusion models with Hugging Face runtime

Issue - State: open - Opened by ka2222222 27 days ago

#3913 - What are the best practices for using `ClusterServingRuntime` and `ServingRuntime`?

Issue - State: open - Opened by fyuan1316 27 days ago - 2 comments

#3912 - Bump Go to 1.22

Pull Request - State: closed - Opened by sivanantha321 28 days ago - 2 comments
Labels: lgtm, approved

#3911 - bump to vllm 0.5.5

Pull Request - State: closed - Opened by lizzzcai 28 days ago - 3 comments
Labels: lgtm, approved

#3910 - mount /dev/shm volume to huggingfaceserver

Pull Request - State: closed - Opened by lizzzcai 29 days ago - 6 comments
Labels: lgtm, approved

#3909 - Kserve agent exec probe failed

Issue - State: open - Opened by beratturan 29 days ago - 3 comments

#3908 - Support OCI image volume mounts

Issue - State: open - Opened by rhuss about 1 month ago - 3 comments

#3907 - update fluid kserve sample to use huggingface servingruntime

Pull Request - State: open - Opened by lizzzcai about 1 month ago - 4 comments

#3906 - Allow other engine builders other than docker

Pull Request - State: open - Opened by spolti about 1 month ago - 1 comment

#3905 - Kserve Storage-initializer Optimization

Issue - State: open - Opened by npanpaliya about 1 month ago - 3 comments

#3904 - Knative- queue proxy resource setting is not correct.

Issue - State: open - Opened by SANJAY-NT about 1 month ago - 1 comment

#3903 - For vllm users, our parser should be able to support both - and _

Pull Request - State: closed - Opened by hustxiayang about 1 month ago

#3902 - storage initializer segregation

Pull Request - State: open - Opened by spolti about 1 month ago - 2 comments

#3901 - aiffairness image dependency issue

Issue - State: open - Opened by spolti about 1 month ago - 4 comments

#3900 - Revision create failed - Duplicate value: "kserve-provision-location"

Issue - State: open - Opened by nagar-ajay about 1 month ago

#3899 - MLflow model cannot be served if storageUri doesn't end in "/"

Issue - State: open - Opened by hakuro95 about 1 month ago

#3898 - Add Security Context and Resources to RBAC Proxy

Pull Request - State: closed - Opened by HotsauceLee about 1 month ago
Labels: lgtm, approved

#3897 - RBAC permission for secret is too broad

Issue - State: closed - Opened by wqlparallel about 1 month ago - 2 comments
Labels: kind/bug

#3896 - Use API token for publishing package to PyPI

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago
Labels: lgtm, approved

#3895 - added http headers to inbound request

Pull Request - State: closed - Opened by andyi2it about 1 month ago - 2 comments
Labels: lgtm, approved

#3894 - CRDS not installing first when using kserve-crd chart as a dependency

Issue - State: open - Opened by Calcagiara about 1 month ago
Labels: kserve/install

#3893 - Remove unwanted cluster scope secret permissions

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 4 comments
Labels: lgtm, approved

#3892 - Deploying GGUF models

Issue - State: open - Opened by Zeashh about 1 month ago - 1 comment

#3891 - SDLC is broken when KServe is installed with helm

Issue - State: closed - Opened by sivanantha321 about 1 month ago

#3890 - Fix sdlc broken when kserve installed using helm

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 4 comments
Labels: lgtm, approved

#3889 - Fix permission error in snyk scan

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 1 comment
Labels: lgtm, approved

#3888 - Add prow-github-action

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 2 comments

#3887 - Consistent Error Responses between Huggingface and vLLM Backends in huggingfaceserver

Issue - State: open - Opened by saileshd1402 about 1 month ago
Labels: kind/bug, kserve/llm

#3886 - Fix the 'tokens exceeding model limit' error response in vllm server

Pull Request - State: closed - Opened by saileshd1402 about 1 month ago
Labels: lgtm, approved

#3885 - Set the volume mount's readonly annotation based on the ISVC annotation

Pull Request - State: closed - Opened by hdefazio about 1 month ago - 8 comments
Labels: lgtm, approved

#3883 - integrating the kserve hosted model with a third party application

Issue - State: open - Opened by tarunsk1998 about 1 month ago

#3882 - An attempt to run microservice on Windows fails with NotImplementedException

Issue - State: open - Opened by drassokhin about 1 month ago

#3881 - Protobuf version upgrade 4.25.4

Pull Request - State: closed - Opened by andyi2it about 1 month ago - 3 comments
Labels: lgtm, approved

#3880 - Fix: Add workaround for snyk image scan failure

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 1 comment
Labels: lgtm, approved

#3879 - Fix trust_remote_code not working with huggingface backend

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 1 comment
Labels: lgtm, approved

#3878 - add a new doc 'common issues and solutions'

Pull Request - State: closed - Opened by Jooho about 1 month ago - 9 comments
Labels: lgtm, approved

#3877 - Enable Server-Side Apply for Kustomize Overlays in Test Environment

Pull Request - State: closed - Opened by Jooho about 1 month ago - 10 comments
Labels: lgtm, approved

#3876 - Can kserve support eks pod download s3 model from another aws account?

Issue - State: closed - Opened by johnzheng1975 about 1 month ago - 3 comments

#3875 - when use huggingface backend get `trust_remote_code=True` error

Issue - State: closed - Opened by jaffe-fly about 1 month ago

#3874 - Bump vLLM to 0.5.4

Pull Request - State: closed - Opened by sivanantha321 about 1 month ago - 3 comments
Labels: lgtm, approved

#3873 - Storage initializer missing

Issue - State: closed - Opened by Zeashh about 1 month ago - 5 comments

#3872 - Release 0.14 Tracking

Issue - State: open - Opened by yuzisun about 1 month ago - 1 comment

#3871 - add a new API for multi-node/multi-gpu

Pull Request - State: open - Opened by Jooho about 1 month ago - 31 comments

#3870 - Support Multi-Node Inference and Serving.

Issue - State: open - Opened by Jooho about 1 month ago
Labels: kind/feature, kserve/llm

#3869 - Support automatic model caching using PV/PVC

Issue - State: open - Opened by yuzisun about 1 month ago
Labels: kserve/llm

#3868 - Provide comprehensive inference benchmarking for LLM backends (vLLM, TensorRT-LLM, TGI)

Issue - State: open - Opened by yuzisun about 1 month ago
Labels: kserve/llm

#3867 - Publish 0.14.0-rc0 release

Pull Request - State: closed - Opened by yuzisun about 1 month ago - 4 comments
Labels: lgtm, approved

#3866 - Fix failed build for knativeLocalGatewayService

Pull Request - State: closed - Opened by yuzisun about 1 month ago - 2 comments
Labels: lgtm, approved

#3865 - Prow GitHub action

Pull Request - State: closed - Opened by sivanantha321 about 2 months ago

#3864 - Serverless. Auto-generated Service for Deployment with enabled metrics aggregation missing port "aggr-metric"

Issue - State: open - Opened by dkondidatov about 2 months ago

#3863 - Add TLS support for Inference Loggers

Pull Request - State: closed - Opened by ruivieira about 2 months ago - 9 comments
Labels: lgtm, approved

#3862 - KServe controller missing SecurityContext in helm chart installation

Issue - State: closed - Opened by yuzisun about 2 months ago - 1 comment
Labels: kind/bug

#3861 - Updated the Makefile to correctly build the image on mac env

Pull Request - State: open - Opened by mholder6 about 2 months ago - 5 comments

#3860 - Cluster local model controller

Pull Request - State: open - Opened by greenmoon55 about 2 months ago - 2 comments

#3859 - InferenceService in version "v1beta1" cannot be handled as a InferenceService: strict decoding error: unknown field "spec.predictor.podSpec"

Issue - State: open - Opened by zhouxinlun about 2 months ago - 8 comments

GitHub / kserve/kserve issues and pull requests