llmariner/inference-manager issues and pull requests

#543 - feat(engine): add ollama tempalte for Phi-4

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: enhancement

#542 - fix(engine): ignore ErrRequestCanceled from Preloader.pullModel

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: bug

#541 - fix(server): fix a data race in TestSendAndProcessTask

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: bug

#540 - chore(engine): add a vlog on model pull error

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: skip-changelog

#539 - feat(engine): add a template for phi-4

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: enhancement

#538 - feat: bump vllm version to support Phi-4

Pull Request - State: closed - Opened by kkaneda about 1 month ago - 1 comment
Labels: enhancement

#537 - chore: fix a comment in findLeastLoadedEngine

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: skip-changelog

#536 - feat: route request to another engine when the runtime fail to schedule

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: enhancement

#535 - feat: cancel an inference request when an error occurs

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: enhancement

#534 - Release v1.12.0

Pull Request - State: closed - Opened by github-actions[bot] about 2 months ago
Labels: skip-changelog

#533 - fix(engine): make special_tokens_map.json optional

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: bug

#532 - feat(engine): make --chat-template optional

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: enhancement

#531 - Release v1.11.0

Pull Request - State: closed - Opened by github-actions[bot] about 2 months ago
Labels: skip-changelog

#530 - fix(engine): set the default context length for deepseek

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: bug

#529 - fix(engine): add an Ollama template for deepseek-ai-DeepSeek-Coder-V2-Lite-Instruct

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: bug

#528 - chore: remove the legacyContent field

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: skip-changelog

#527 - feat(engine): be able to configure runtime class from Helm values

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: enhancement

#526 - ci(local): disable component-status-sender in the local dev env

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: skip-changelog

#525 - feat(engine): be able to specify a scheduler name

Pull Request - State: closed - Opened by kkaneda about 2 months ago
Labels: enhancement

#524 - fix(engine): add a retry delay for the reachable check to the runtime

Pull Request - State: closed - Opened by Ladicle about 2 months ago

#523 - feat(engine): update ScaledObjects during startup

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: enhancement

#522 - feat(engine): expose inference engine metrics

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: enhancement

#521 - fix(script): fix helmfile selecting flag

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: bug

#520 - refactor(engine): rename metrics files to client

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: skip-changelog

#519 - fix(engine): disable leader election when the autoscaler type is keda

Pull Request - State: closed - Opened by Ladicle about 2 months ago
Labels: bug

#518 - feat(engine): set --tensor-paralle-size for Inferentia nodes

Pull Request - State: closed - Opened by kkaneda 2 months ago
Labels: enhancement

#517 - feat(engine)!: add KEDA integration for vLLM

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: enhancement, breaking-change

#516 - refactor(engine): move scaler registerer from runtime to autoscaler pkg

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: skip-changelog

#515 - refactor(engine): rename files for the builtin scaler

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: skip-changelog

#514 - refactor(engine): rename Multiautoscaler to BuiltinScaler

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: skip-changelog

#513 - feat: add a template for Llama3.3

Pull Request - State: closed - Opened by kkaneda 2 months ago
Labels: enhancement

#512 - chore(chart): remove unnecessary file

Pull Request - State: closed - Opened by Ladicle 2 months ago - 1 comment
Labels: skip-changelog

#511 - Release v1.10.1

Pull Request - State: closed - Opened by github-actions[bot] 2 months ago
Labels: skip-changelog

#510 - chore: create a PodStatusSender only when enabled

Pull Request - State: closed - Opened by kkaneda 2 months ago
Labels: skip-changelog

#509 - fix: bump cluster-manager dep to fix health reporting

Pull Request - State: closed - Opened by kkaneda 2 months ago
Labels: bug

#508 - Release v1.10.0

Pull Request - State: closed - Opened by github-actions[bot] 2 months ago
Labels: skip-changelog

#507 - ci(apply-dep): support for installing extra dependency apps

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: skip-changelog

#506 - feat(runtime): set runtime/model annotations to the pod template

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: enhancement

#505 - feat(engine): send component status message

Pull Request - State: closed - Opened by guangrui-cloudnatix 2 months ago
Labels: enhancement

#504 - refactor(chart): wrap optional values with `with`

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: skip-changelog

#503 - feat: support vLLM on macOS for development

Pull Request - State: closed - Opened by Ladicle 2 months ago
Labels: enhancement

#502 - ci: use memory as a rate limit backend in the local dev env

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#502 - ci: use memory as a rate limit backend in the local dev env

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#501 - chore(hack): implicitly specify the dependency components for test

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#501 - chore(hack): implicitly specify the dependency components for test

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#500 - Release v1.9.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#500 - Release v1.9.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#499 - ci(release): add bump option

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#499 - ci(release): add bump option

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#498 - fix(chart): unset the default value of `enable`

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: bug

#498 - fix(chart): unset the default value of `enable`

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: bug

#497 - feat: support request-based rate-limiter

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#497 - feat: support request-based rate-limiter

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#496 - fix(engine): unset legacyContent

Pull Request - State: closed - Opened by guangrui-cloudnatix 3 months ago - 4 comments
Labels: bug

#496 - fix(engine): unset legacyContent

Pull Request - State: closed - Opened by guangrui-cloudnatix 3 months ago - 4 comments
Labels: bug

#495 - fix(engine): correctly set pod annotations for runtimes

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#495 - fix(engine): correctly set pod annotations for runtimes

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#494 - test(server): fix the flaky integration test

Pull Request - State: closed - Opened by kkaneda 3 months ago

#494 - test(server): fix the flaky integration test

Pull Request - State: closed - Opened by kkaneda 3 months ago

#493 - fix(trition-proxy): remove an unneeded char in the prompt template

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#493 - fix(trition-proxy): remove an unneeded char in the prompt template

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#492 - feat(api): support both array and string for the "content" field

Pull Request - State: closed - Opened by kkaneda 3 months ago - 1 comment
Labels: enhancement

#492 - feat(api): support both array and string for the "content" field

Pull Request - State: closed - Opened by kkaneda 3 months ago - 1 comment
Labels: enhancement

#491 - ci(chart): cleanup LLMariner Chart.lock for test

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#491 - ci(chart): cleanup LLMariner Chart.lock for test

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#490 - feat(server): support an enable flag for the usage sender

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#490 - feat(server): support an enable flag for the usage sender

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#489 - Release v1.8.1

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#489 - Release v1.8.1

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#488 - Release v1.9.0

Pull Request - State: closed - Opened by kkaneda 3 months ago

#488 - Release v1.9.0

Pull Request - State: closed - Opened by kkaneda 3 months ago

#487 - Release v1.9.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#487 - Release v1.9.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#486 - fix(chart): update the schema of the engine chart

Pull Request - State: closed - Opened by kkaneda 3 months ago

#486 - fix(chart): update the schema of the engine chart

Pull Request - State: closed - Opened by kkaneda 3 months ago

#485 - feat(chart): add an enable option for the dependency condition

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#484 - Release v1.8.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#483 - fix(engine/config): restore an unreferenced volume validation

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: bug

#482 - feat(runtime): set an engine Deployment as an owner of runtime StatefulSet

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#481 - ci(make): add target to re-apply and restarts components

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#480 - fix(engine): fix leader-election and runtime deletion handling

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: bug

#479 - feat(api/engine): support audio chat completion

Pull Request - State: closed - Opened by guangrui-cloudnatix 3 months ago - 1 comment
Labels: enhancement

#478 - fix(server): set StreamOptions only for streaming requests

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#477 - ci: add scripts for testing local inference server/engine

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#476 - Release v1.7.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#475 - refactor(log): remove unused struct

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#474 - ci(make): fix git-clean-check to validate the preceding target results

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#473 - feat(engine): propagate pod annotations to runtime pods

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: enhancement

#472 - fix(chart): do not set the default value of s3.endpoinrUrl

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: bug

#471 - feat(chart): add cronjob for regular runtime shutdown

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#470 - fix(values.yaml): fix typos

Pull Request - State: closed - Opened by takeshi-cloudnatix 3 months ago - 2 comments

#469 - Release v1.6.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#468 - feat(engine): change the vLLM version back to 0.6.2

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: enhancement

#467 - feat(engine): set VLLM_RPC_TIMEOUT to a larger value

Pull Request - State: closed - Opened by kkaneda 3 months ago
Labels: enhancement

#466 - chore(chart): fix a comment for PrometheusRule

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#465 - feat(server): add an option to enable/disable dynamic model loading

Pull Request - State: closed - Opened by kkaneda 3 months ago - 3 comments
Labels: enhancement

#464 - Release v1.5.0

Pull Request - State: closed - Opened by github-actions[bot] 3 months ago
Labels: skip-changelog

#463 - feat(chart): productionize helm chart

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: enhancement

#462 - ci(pr-labeler): change trigger to pull_request_target

Pull Request - State: closed - Opened by Ladicle 3 months ago
Labels: skip-changelog

#461 - feat(api): support multiple content types in chat completion request

Pull Request - State: closed - Opened by guangrui-cloudnatix 3 months ago
Labels: enhancement

GitHub / llmariner/inference-manager issues and pull requests