kserve/kserve issues and pull requests

#3647 - Support ollama server

Issue - State: open - Opened by skonto 5 months ago - 2 comments
Labels: kind/feature, kserve/llm

#3646 - Add Modelcars as initContainer with restartPolicy == Always (optional)

Issue - State: open - Opened by rhuss 5 months ago
Labels: kind/feature

#3645 - Add python 3.12 support and remove python 3.8 support

Pull Request - State: open - Opened by sivanantha321 5 months ago - 4 comments

#3644 - Add vLLM backend e2e test

Issue - State: open - Opened by yuzisun 5 months ago - 1 comment
Labels: kind/feature, kserve/llm

#3643 - Inference gRPC/Rest client to support FP16

Issue - State: closed - Opened by yuzisun 5 months ago
Labels: kind/feature

#3642 - chore: Bump ModelMesh version to v0.12.0-rc0 in Helm chart

Pull Request - State: closed - Opened by terrytangyuan 5 months ago - 2 comments
Labels: lgtm, approved

#3641 - build: Remove misleading logs from minimal-crdgen.sh

Pull Request - State: closed - Opened by terrytangyuan 5 months ago - 2 comments
Labels: lgtm, approved

#3640 - Test ci

Pull Request - State: closed - Opened by houshengbo 5 months ago - 1 comment

#3639 - Merge responses from InferenceGraph Sequence node steps

Issue - State: open - Opened by asd981256 5 months ago - 2 comments
Labels: kind/feature, kserve/inference_graph

#3638 - Autoscaling with multiple metrics does not work

Issue - State: open - Opened by shazinahmed 5 months ago - 3 comments
Labels: kind/feature

#3637 - logger not surfacing the error when failed to send cloud event

Issue - State: open - Opened by yuzisun 5 months ago
Labels: kind/bug

#3636 - Record ownerReferences on managed ingress

Issue - State: closed - Opened by backjo 5 months ago
Labels: kind/feature

#3635 - adding metadata on requests

Pull Request - State: closed - Opened by gcemaj 5 months ago - 12 comments
Labels: lgtm, approved

#3634 - Add metadata to logger system

Issue - State: closed - Opened by gcemaj 5 months ago
Labels: kind/feature

#3633 - add re-run info in the PR templates

Pull Request - State: closed - Opened by spolti 5 months ago - 2 comments
Labels: lgtm, approved

#3632 - Custom ClusterServingRuntime not being selected based on modelFormat Name/Version without runtime specification

Issue - State: open - Opened by supertetelman 5 months ago - 2 comments
Labels: kind/bug

#3631 - Allow re-running of failed workflows

Issue - State: closed - Opened by andyi2it 5 months ago
Labels: kind/feature

#3630 - Fix docker build failures for arm64 and restrict ray serve to ~2.10.0

Pull Request - State: closed - Opened by sivanantha321 5 months ago - 1 comment

#3629 - Test image builds for ARM64 arch in CI

Pull Request - State: closed - Opened by sivanantha321 5 months ago - 6 comments
Labels: lgtm, approved

#3628 - WIP: test purpose Check ci

Pull Request - State: closed - Opened by houshengbo 5 months ago - 1 comment
Labels: do-not-merge/work-in-progress

#3627 - [release-0.12] Fix docker build failure for ARM64

Pull Request - State: closed - Opened by sivanantha321 5 months ago - 4 comments
Labels: lgtm, approved

#3626 - aiohttp package upgrade

Pull Request - State: closed - Opened by andyi2it 5 months ago - 4 comments
Labels: do-not-merge/work-in-progress

#3625 - Discuss the future of models-webapp

Issue - State: open - Opened by rimolive 5 months ago - 8 comments
Labels: kind/feature

#3624 - WIP:Check ci

Pull Request - State: closed - Opened by houshengbo 5 months ago - 1 comment
Labels: do-not-merge/work-in-progress

#3623 - Any specific optimization did in kserve to support LLM inference?

Issue - State: open - Opened by Jeffwan 5 months ago - 2 comments
Labels: kind/question

#3622 - Fix Pydantic 2 warnings

Pull Request - State: closed - Opened by cmaddalozzo 5 months ago - 2 comments
Labels: lgtm, approved

#3621 - Add base model for proxying request to an OpenAI API enabled model server

Pull Request - State: closed - Opened by cmaddalozzo 5 months ago - 2 comments
Labels: lgtm, approved

#3620 - build: Fix CRD copying in generate-install.sh

Pull Request - State: closed - Opened by terrytangyuan 5 months ago - 2 comments
Labels: lgtm, approved

#3619 - chore: v0.12.1 install files

Pull Request - State: closed - Opened by terrytangyuan 5 months ago - 3 comments
Labels: lgtm, approved

#3618 - Fix CVE-2023-45288 for qpext

Pull Request - State: closed - Opened by sivanantha321 5 months ago - 2 comments
Labels: lgtm, approved

#3617 - Fix make deploy-dev-storage-initializer not working

Pull Request - State: closed - Opened by sivanantha321 5 months ago - 2 comments
Labels: lgtm, approved

#3616 - release-0.12 pin back ray to 2.10

Pull Request - State: closed - Opened by yuzisun 5 months ago - 3 comments
Labels: lgtm, approved

#3615 - Enabled the multiple domains support on an inference service

Pull Request - State: closed - Opened by houshengbo 5 months ago - 21 comments
Labels: lgtm, approved

#3614 - [release-0.12] Pydantic 2 support

Pull Request - State: closed - Opened by cmaddalozzo 5 months ago - 2 comments
Labels: lgtm, approved

#3613 - Enable dtype support for huggingface server

Pull Request - State: closed - Opened by Datta0 5 months ago - 9 comments
Labels: lgtm, approved

#3612 - [release-0.12] Make the modelcar injection idempotent

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: lgtm, approved

#3611 - Upgrade from v0.10 to v0.11 breaks the InferenceService at routing level

Issue - State: open - Opened by bgalvao 6 months ago - 1 comment
Labels: kind/bug

#3610 - Prepare for release 0.12.1

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: lgtm, approved

#3609 - [release-0.12] Update fastapi to 0.109.1 and Support ray 2.10

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: lgtm, approved

#3608 - Update exclusion list for go lint and Fix Go lint errors

Issue - State: closed - Opened by andyi2it 6 months ago
Labels: kind/bug

#3607 - Add option for returning probabilities in huggingface server

Pull Request - State: closed - Opened by andyi2it 6 months ago - 2 comments
Labels: lgtm, approved

#3606 - Support overriding model mount path in model server container

Issue - State: open - Opened by cmaddalozzo 6 months ago - 3 comments
Labels: kind/feature, kserve/llm

#3605 - Added secure gRPC server option to kserve

Pull Request - State: open - Opened by cbalbera-imaige 6 months ago - 2 comments

#3604 - build: Allow pre-commit to keep changes in reformatted code

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 5 comments
Labels: lgtm, approved

#3603 - feat: Support customizable deployment strategy for RawDeployment mode. Fixes #3452

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 11 comments
Labels: lgtm, approved

#3602 - Fix new Golang security vulnerability - CVE-2024-24786

Issue - State: closed - Opened by andyi2it 6 months ago
Labels: kind/bug

#3601 - Integrate triton with huggingface runtime

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 3 comments
Labels: do-not-merge/work-in-progress

#3600 - Add e2e tests for huggingface

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 3 comments
Labels: lgtm, approved

#3599 - golang.org/x/net Allocation of Resources Without Limits or Throttling

Pull Request - State: closed - Opened by spolti 6 months ago - 2 comments

#3598 - pillow - Buffer Overflow

Pull Request - State: closed - Opened by spolti 6 months ago - 11 comments
Labels: lgtm, approved

#3597 - Scale pods based on a cron schedule

Issue - State: open - Opened by vukor 6 months ago
Labels: kind/feature, kserve/autoscaling

#3596 - golang.org/x/net Allocation of Resources Without Limits or Throttling

Pull Request - State: closed - Opened by spolti 6 months ago - 2 comments
Labels: lgtm, approved

#3595 - Parallel Model Inference with Ray Serving not working due to deprecated API

Issue - State: closed - Opened by ajstewart 6 months ago - 3 comments
Labels: kind/bug

#3594 - set default SAFETENSORS_FAST_GPU and HF_HUB_DISABLE_TELEMETRY in HF Server

Pull Request - State: closed - Opened by lizzzcai 6 months ago - 4 comments
Labels: lgtm, approved

#3593 - add dockerfile

Pull Request - State: closed - Opened by kevinmingtarja 6 months ago - 2 comments

#3591 - Old Revisions of Inference Service not Scaled Down

Issue - State: open - Opened by ksgnextuple 6 months ago - 3 comments
Labels: kind/bug

#3590 - Support OpenAIModel in ModelRepository

Pull Request - State: closed - Opened by grandbora 6 months ago - 7 comments
Labels: lgtm, approved

#3589 - VLLM support for OpenAI Completions in HF server

Pull Request - State: closed - Opened by gavrishp 6 months ago - 6 comments
Labels: lgtm, approved

#3587 - Fix model unload in server stop method

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 5 comments
Labels: lgtm, approved

#3586 - -

Pull Request - State: closed - Opened by kevinmingtarja 6 months ago - 2 comments

#3585 - Security fix - CVE 2024 24786

Pull Request - State: closed - Opened by andyi2it 6 months ago - 3 comments
Labels: lgtm, approved

#3584 - Implement Huggingface model download in storage initializer

Pull Request - State: open - Opened by andyi2it 6 months ago - 7 comments

#3583 - Remove x/crypto replace as it was updated to 0.19

Pull Request - State: closed - Opened by spolti 6 months ago - 1 comment

#3582 - Add OpenAI API support to Huggingfaceserver

Pull Request - State: closed - Opened by cmaddalozzo 6 months ago - 4 comments
Labels: lgtm, approved

#3581 - Add a user friendly error message for http exceptions

Pull Request - State: closed - Opened by grandbora 6 months ago - 19 comments
Labels: lgtm, approved

#3580 - Allow remote code execution on huggingfaceserver

Issue - State: closed - Opened by isaranto 6 months ago - 3 comments
Labels: kind/feature, kserve/llm

#3579 - docs: Move Alibi explainer to docs

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 4 comments
Labels: lgtm, approved

#3578 - build: Add flake8 and black to pre-commit hooks

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 7 comments
Labels: lgtm, approved

#3577 - Unify the log configuration using kserve logger

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: lgtm, approved

#3576 - fix: Set writable cache folder to avoid permission issue. Fixes #3562

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 4 comments
Labels: lgtm, approved

#3575 - Fix typo in README.md

Pull Request - State: closed - Opened by terrytangyuan 6 months ago - 2 comments
Labels: lgtm, approved

#3574 - Add parameter in ModelMetadataResponse in v2 (aka open inference) protocol

Issue - State: open - Opened by harshita-meena 6 months ago
Labels: kind/feature

#3572 - Support text embedding task in huggingface server

Issue - State: closed - Opened by kevinmingtarja 6 months ago - 6 comments
Labels: kind/feature, kserve/llm

#3571 - Fix python linter

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 5 comments
Labels: lgtm, approved

#3570 - Add option to skip postprocess in huggingface server

Issue - State: closed - Opened by kevinmingtarja 6 months ago - 1 comment
Labels: kind/feature, kserve/huggingface

#3569 - fix lint typo on Makefile

Pull Request - State: closed - Opened by spolti 6 months ago - 2 comments
Labels: lgtm, approved

#3568 - Black lint

Pull Request - State: closed - Opened by cmaddalozzo 6 months ago - 4 comments
Labels: lgtm, approved

#3567 - Use black to auto-format Python code

Issue - State: closed - Opened by cmaddalozzo 6 months ago
Labels: kind/feature

#3565 - Explicitly specify pad token id when generating tokens

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 10 comments
Labels: lgtm, approved

#3564 - Add support for pluggable explainer runtimes

Pull Request - State: open - Opened by TimKleinloog 6 months ago - 7 comments

#3563 - Go Coverage action is failing in CI

Issue - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: kind/bug

#3562 - Huggingface server fails to start on OpenShift cluster

Issue - State: closed - Opened by eyalcha 6 months ago - 3 comments
Labels: kind/bug

#3561 - Native integration with KEDA for LLM inference autoscaling

Issue - State: open - Opened by yuzisun 6 months ago - 4 comments
Labels: kind/feature, kserve/llm

#3560 - Unified log format in KServe runtime application

Issue - State: closed - Opened by lizzzcai 6 months ago - 2 comments
Labels: kind/feature, kserve/sdk

#3559 - Only load from model repository if model binary is not found under model_dir

Pull Request - State: closed - Opened by sivanantha321 6 months ago - 2 comments
Labels: lgtm, approved

#3558 - Support model revision and tokenizer revision in huggingface server

Pull Request - State: closed - Opened by lizzzcai 6 months ago - 4 comments
Labels: lgtm, approved

#3557 - Support model revision and tokenizer revision in huggingface server

Issue - State: closed - Opened by lizzzcai 6 months ago
Labels: kind/feature

#3556 - CVE-2024-24762 - update fastapi to 0.109.1

Pull Request - State: closed - Opened by spolti 6 months ago - 8 comments
Labels: lgtm, approved

#3555 - Support mlflow direct integration through storage initializer

Issue - State: open - Opened by Ryojikn 6 months ago
Labels: kind/feature, kfserving/storage

#3554 - Add ability to InferenceGraph to passTrough custom annotations

Issue - State: open - Opened by trojaond 6 months ago - 1 comment
Labels: kind/bug, kserve/inference_graph

#3553 - Transformers don't deploy with ModelMesh InferenceService in RawDeployment mode

Issue - State: open - Opened by slik13 6 months ago - 1 comment
Labels: kind/bug, kserve/raw_deployment, kserve/modelmesh

#3552 - Fix golint errors

Pull Request - State: closed - Opened by andyi2it 6 months ago - 2 comments
Labels: lgtm, approved

#3551 - updated xgboost to support json and ubj models

Pull Request - State: closed - Opened by andyi2it 6 months ago - 2 comments
Labels: lgtm, approved

#3550 - Allow rerunning failed workflows by comment

Pull Request - State: closed - Opened by andyi2it 6 months ago - 4 comments
Labels: lgtm, approved

#3549 - Closed

Issue - State: closed - Opened by SNagappan 6 months ago - 7 comments

#3548 - Update codeQL to v3

Pull Request - State: closed - Opened by spolti 6 months ago - 2 comments
Labels: lgtm, approved

#3547 - Expose Initial LLM metrics

Pull Request - State: open - Opened by sivanantha321 6 months ago - 1 comment
Labels: do-not-merge/work-in-progress

#3546 - Support .json and .ubj model format for XGBoost server image

Issue - State: closed - Opened by hvanu 6 months ago
Labels: kind/feature

#3545 - Support download huggingface models in storage initializer

Issue - State: open - Opened by lizzzcai 6 months ago - 2 comments
Labels: kind/feature, kserve/huggingface

#3544 - fix a typo in OPENSHIFT_GUIDE.md

Pull Request - State: closed - Opened by marek-veber 6 months ago - 4 comments
Labels: lgtm, approved

GitHub / kserve/kserve issues and pull requests