Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / llmariner/inference-manager issues and pull requests

#485 - feat(chart): add an enable option for the dependency condition

Pull Request - State: closed - Opened by Ladicle 18 days ago
Labels: enhancement

#484 - Release v1.8.0

Pull Request - State: closed - Opened by github-actions[bot] 18 days ago
Labels: skip-changelog

#483 - fix(engine/config): restore an unreferenced volume validation

Pull Request - State: closed - Opened by Ladicle 19 days ago
Labels: bug

#482 - feat(runtime): set an engine Deployment as an owner of runtime StatefulSet

Pull Request - State: closed - Opened by Ladicle 19 days ago
Labels: enhancement

#481 - ci(make): add target to re-apply and restarts components

Pull Request - State: closed - Opened by Ladicle 19 days ago
Labels: skip-changelog

#480 - fix(engine): fix leader-election and runtime deletion handling

Pull Request - State: closed - Opened by Ladicle 19 days ago
Labels: bug

#479 - feat(api/engine): support audio chat completion

Pull Request - State: open - Opened by guangrui-cloudnatix 20 days ago
Labels: enhancement

#478 - fix(server): set StreamOptions only for streaming requests

Pull Request - State: closed - Opened by kkaneda 20 days ago
Labels: bug

#477 - ci: add scripts for testing local inference server/engine

Pull Request - State: closed - Opened by Ladicle 21 days ago
Labels: skip-changelog

#476 - Release v1.7.0

Pull Request - State: closed - Opened by github-actions[bot] 24 days ago
Labels: skip-changelog

#475 - refactor(log): remove unused struct

Pull Request - State: closed - Opened by Ladicle 24 days ago
Labels: skip-changelog

#474 - ci(make): fix git-clean-check to validate the preceding target results

Pull Request - State: closed - Opened by Ladicle 25 days ago
Labels: skip-changelog

#473 - feat(engine): propagate pod annotations to runtime pods

Pull Request - State: closed - Opened by kkaneda 25 days ago
Labels: enhancement

#472 - fix(chart): do not set the default value of s3.endpoinrUrl

Pull Request - State: closed - Opened by kkaneda 25 days ago
Labels: bug

#471 - feat(chart): add cronjob for regular runtime shutdown

Pull Request - State: closed - Opened by Ladicle 25 days ago
Labels: enhancement

#470 - fix(values.yaml): fix typos

Pull Request - State: closed - Opened by takeshi-cloudnatix 25 days ago - 2 comments

#469 - Release v1.6.0

Pull Request - State: closed - Opened by github-actions[bot] 26 days ago
Labels: skip-changelog

#468 - feat(engine): change the vLLM version back to 0.6.2

Pull Request - State: closed - Opened by kkaneda 26 days ago
Labels: enhancement

#467 - feat(engine): set VLLM_RPC_TIMEOUT to a larger value

Pull Request - State: closed - Opened by kkaneda 26 days ago
Labels: enhancement

#466 - chore(chart): fix a comment for PrometheusRule

Pull Request - State: closed - Opened by Ladicle 26 days ago
Labels: skip-changelog

#465 - feat(server): add an option to enable/disable dynamic model loading

Pull Request - State: closed - Opened by kkaneda 27 days ago - 3 comments
Labels: enhancement

#464 - Release v1.5.0

Pull Request - State: closed - Opened by github-actions[bot] 28 days ago
Labels: skip-changelog

#463 - feat(chart): productionize helm chart

Pull Request - State: closed - Opened by Ladicle 28 days ago
Labels: enhancement

#462 - ci(pr-labeler): change trigger to pull_request_target

Pull Request - State: closed - Opened by Ladicle 28 days ago
Labels: skip-changelog

#461 - feat(api): support multiple content types in chat completion request

Pull Request - State: closed - Opened by guangrui-cloudnatix about 1 month ago
Labels: enhancement

#460 - chore(api): remove legacy/inference_server_worker.swagger.json

Pull Request - State: closed - Opened by kkaneda about 1 month ago
Labels: skip-changelog

#459 - Release v1.4.0

Pull Request - State: closed - Opened by github-actions[bot] about 1 month ago
Labels: skip-changelog

#458 - test: add integration for multi-server instances

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#457 - ci: setup labeler for changelog

Pull Request - State: closed - Opened by Ladicle about 1 month ago
Labels: skip-changelog

#456 - chore(server): update Engines() to LocalEngines()

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#455 - fix(server): fix a nil-pointer-dereference in SendAndProcessTask

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#454 - feat(server): show IsLocal in the admin page

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#453 - chore(server): embed processMessagesFromEngine

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#452 - feat(engine): remove legacy runtime config

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#451 - feat(api): remove legacy gRPC service

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#450 - feat(api): remove deprecatedChatCompletionRequest from proto

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#449 - feat(server): remove enableEngineReadinessCheck

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#448 - chore(server): rename processTaskResult

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#447 - fix(common): fix a race condition in NewTestLogger

Pull Request - State: closed - Opened by kkaneda about 1 month ago - 1 comment

#446 - feat(server): make the task scheduler prefer a local engine

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#445 - fix(server): handle a case when an engine is removed just before task…

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#444 - feat(server): add SendAndProcessTask to infprocessor.P

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#443 - feat(server): implement task exchanger

Pull Request - State: closed - Opened by kkaneda about 1 month ago - 1 comment

#442 - chore(engine): remove the 'ctx' arg from sendEngineStatus

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#441 - refactor(server): refactor the logic for processing tasks results

Pull Request - State: closed - Opened by kkaneda about 1 month ago - 1 comment

#440 - chore(server): unexpose fields of task

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#439 - feat(server): define internal server

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#438 - feat(server): add Engines() to infprocessor.P

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#437 - refactor(server): drop Recv from engineCommunicator

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#436 - refactor(server): remove the use of clusterInfo from infprocessor

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#435 - Release v1.3.1

Pull Request - State: closed - Opened by github-actions[bot] about 1 month ago

#434 - fix(server): remove database from config

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#433 - chore(engine): fix a log message in vllmClient

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#432 - chore: run 'go mod tidy'

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#431 - Release v1.3.0

Pull Request - State: closed - Opened by github-actions[bot] about 1 month ago

#430 - feat(server): revert the DB wiring

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#429 - chore(server): factor out isTaskCompleted

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#428 - chore(server): add a small comment to writeTaskResultToChan

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#427 - Release v1.2.0

Pull Request - State: closed - Opened by github-actions[bot] about 1 month ago

#426 - feat(server): use engine status in DB to route requests

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#425 - feat(server): tracke engine status in DB

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#424 - fix(server): fix a bug in AddOrUpdateEngineStatus

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#423 - feat(server): Wire up database

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#422 - fix(server): re-enable the engine readiness check

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#421 - fix(server): Correctly acquire a lock in scheduleTask

Pull Request - State: closed - Opened by kkaneda about 1 month ago

#420 - docs: Update Copyright in LICENSE

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#419 - Release v1.1.0

Pull Request - State: closed - Opened by github-actions[bot] about 2 months ago

#418 - fix(runtime): Add back missing volumes for /dev/shm

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#417 - feat(runtime): Upgrade the vLLM container version to 0.6.3

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#415 - feat(runtime): support bitsandbytes quantization in vLLM

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#414 - feat(runtime): Be able to specify extra flags for vLLM

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#413 - feat(runtime): explicitly set the medium to "Memory" for /dev/shm

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#412 - feat(runtime): support affinity and per-pod volume settings

Pull Request - State: closed - Opened by Ladicle about 2 months ago

#411 - Support nvidia-Llama-3.1-Nemotron

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#410 - chore(Makefile): fix non-working go-fmt target by invalid indents

Pull Request - State: closed - Opened by Ladicle about 2 months ago

#409 - Release v1.0.0

Pull Request - State: closed - Opened by github-actions[bot] about 2 months ago

#408 - Configure release workflows

Pull Request - State: closed - Opened by Ladicle about 2 months ago

#407 - Make a fine-tuned model use the same runtime config as its base model

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#406 - Remove "ft:" prefix when passing model IDs to the vLLM runtime

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#405 - Do not mention a tenant in an error message

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#404 - Update the model file for Ollama fine-tuning

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#403 - Do not skip a hidden file when downloading model files

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#402 - Set some values to ensembleGenerateRequest

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#401 - Use port 9000 for triton proxy

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#400 - Include the model ID in the triton model registry path

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#399 - Append repo/llama3 to the triton model registry path

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#398 - Correctly download hierarchical files DownloadAllModelFiles

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#397 - Add an experimental support of Trition Inference Server

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#396 - triton-proxy: Remove config

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#395 - Add triton-proxy

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#394 - Factor out common functions for upcoming change to support Triton

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#393 - Make small changes to improve the code readability

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#392 - Refactor the HuggingFace download logic to explicitly take an adapter…

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#391 - Bump rbac-server dep

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#390 - Track API key usage

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#389 - enable lora-based fine tuning model in vllm

Pull Request - State: closed - Opened by guangrui-cloudnatix about 2 months ago

#388 - Include usages in streaming by default

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#387 - Support StreamOption

Pull Request - State: closed - Opened by kkaneda about 2 months ago

#386 - Fix the sleep login in preloader

Pull Request - State: closed - Opened by kkaneda about 2 months ago