llmariner/inference-manager issues and pull requests

#100 - Remove "ft:" only from Ollama model names

Pull Request - State: closed - Opened by kkaneda 6 months ago

#99 - Check auth before pulling model to engine

Pull Request - State: closed - Opened by guangrui-cloudnatix 6 months ago

#98 - Remove the logic for removing "ft:" from model name

Pull Request - State: closed - Opened by kkaneda 6 months ago

#97 - Remove --issues-exit-code=0 from golangci-lint

Pull Request - State: closed - Opened by kkaneda 6 months ago

#96 - track request latency in server

Pull Request - State: closed - Opened by guangrui-cloudnatix 6 months ago

#95 - Make a minor fix to logging

Pull Request - State: closed - Opened by kkaneda 6 months ago

#94 - Rename route.go to route_map.go

Pull Request - State: closed - Opened by kkaneda 6 months ago

#93 - Add a unit test to completions.go

Pull Request - State: closed - Opened by kkaneda 6 months ago

#92 - Define the engineGetter interface

Pull Request - State: closed - Opened by kkaneda 6 months ago

#91 - Validate if a model exists for a user

Pull Request - State: closed - Opened by kkaneda 6 months ago - 1 comment

#90 - remove preloading models to engines

Pull Request - State: closed - Opened by guangrui-cloudnatix 6 months ago

#89 - Remove redudant log message on informer

Pull Request - State: closed - Opened by kkaneda 6 months ago

#88 - Use fake.NewSimpleClientset to inject pods

Pull Request - State: closed - Opened by kkaneda 6 months ago

#87 - Call findLeastLoadedEngine from GetEngineForModel

Pull Request - State: closed - Opened by kkaneda 6 months ago

#86 - Remove unneeded call of printRoute

Pull Request - State: closed - Opened by kkaneda 6 months ago

#85 - Bump model-manager dep

Pull Request - State: closed - Opened by kkaneda 6 months ago

#84 - Bump dep to rbac-server

Pull Request - State: closed - Opened by kkaneda 6 months ago

#83 - Set header matcher

Pull Request - State: closed - Opened by kkaneda 6 months ago

#82 - Bump rbac-manager dep

Pull Request - State: closed - Opened by kkaneda 7 months ago

#81 - Remove go-staticcheck from common.mk

Pull Request - State: closed - Opened by kkaneda 7 months ago

#80 - Set EmitDefaultValues to true for json marshaling

Pull Request - State: closed - Opened by kkaneda 7 months ago

#79 - Move pkg/k8s to server/internal/k8s

Pull Request - State: closed - Opened by kkaneda 7 months ago

#78 - Remove "llm-operator-" prefix from addrs

Pull Request - State: closed - Opened by kkaneda 7 months ago

#77 - Merge functions in pkg/k8s/informer.go

Pull Request - State: closed - Opened by kkaneda 7 months ago

#76 - dynamically load models per request

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago - 1 comment

#75 - Use interceptor to log

Pull Request - State: closed - Opened by kkaneda 7 months ago

#74 - Make a minor change to the Role definition

Pull Request - State: closed - Opened by kkaneda 7 months ago

#73 - add k8s pkg

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#72 - implement internal grpc functions for engine

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#71 - Be able to specify service annotations

Pull Request - State: closed - Opened by kkaneda 7 months ago

#70 - Support llama3 for Ollama model file

Pull Request - State: closed - Opened by kkaneda 7 months ago

#69 - Remove a server for gRPC

Pull Request - State: closed - Opened by kkaneda 7 months ago

#68 - Upgrade rbac-manager mod

Pull Request - State: closed - Opened by kkaneda 7 months ago

#67 - Support google-gemma-7b

Pull Request - State: closed - Opened by kkaneda 7 months ago

#66 - Make the stop field repeated string

Pull Request - State: closed - Opened by kkaneda 7 months ago

#65 - Remove db password env var from docker-compose.yaml

Pull Request - State: closed - Opened by kkaneda 7 months ago

#64 - allow inference-manager-server to access k8s object

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#63 - Add pull/delete/listModels methods for engine

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#62 - Check if a model exists

Pull Request - State: closed - Opened by kkaneda 7 months ago

#61 - Enable authorization to CreateChatCompletion

Pull Request - State: closed - Opened by kkaneda 7 months ago

#60 - Support streaming with server-side events

Pull Request - State: closed - Opened by kkaneda 7 months ago

#59 - Replace gRPC with HTTP

Pull Request - State: closed - Opened by kkaneda 7 months ago

#58 - Add docker-compose.yaml

Pull Request - State: closed - Opened by kkaneda 7 months ago

#57 - Change Ollma modelfile based on models

Pull Request - State: closed - Opened by kkaneda 7 months ago - 1 comment

#56 - Fix helm chart

Pull Request - State: closed - Opened by kkaneda 7 months ago

#55 - Remove ingress from engine

Pull Request - State: closed - Opened by kkaneda 7 months ago - 1 comment

#54 - clean up

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#53 - Make small changes to Helm charts

Pull Request - State: closed - Opened by kkaneda 7 months ago

#52 - Do not use the camel case for JSON fields to follow OpenAI API

Pull Request - State: closed - Opened by kkaneda 7 months ago

#51 - Use model internal client to list models

Pull Request - State: closed - Opened by kkaneda 7 months ago

#50 - setup keda scaledobject for engine

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#49 - Remove base name conversion

Pull Request - State: closed - Opened by kkaneda 7 months ago

#48 - Add a note on how to import Ollama models to locally running engine

Pull Request - State: closed - Opened by kkaneda 7 months ago

#47 - setup inference-manager-server deployment

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago - 2 comments

#46 - Make ollama_client using Ollama OpenAI compatible API

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#45 - Make minor fix to ollama_client.go

Pull Request - State: closed - Opened by kkaneda 7 months ago

#44 - Authorize token by rbac-server

Pull Request - State: closed - Opened by Ladicle 7 months ago

#43 - Fix the duplicate imported module

Pull Request - State: closed - Opened by Ladicle 7 months ago

#42 - fix build

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#41 - make server bypass to ollama

Pull Request - State: closed - Opened by guangrui-cloudnatix 7 months ago

#40 - Update internalGrpcPort to 8082

Pull Request - State: closed - Opened by kkaneda 7 months ago

#39 - Change Modelfile creation

Pull Request - State: closed - Opened by kkaneda 7 months ago

#38 - Pull base models from model manager

Pull Request - State: closed - Opened by kkaneda 7 months ago

#37 - Add checksum of values as annotation

Pull Request - State: closed - Opened by jmuk 7 months ago

#36 - Sync base models from model manager

Pull Request - State: closed - Opened by kkaneda 7 months ago

#35 - Ignore system models

Pull Request - State: closed - Opened by kkaneda 7 months ago

#34 - Switch to setup-go to v5.

Pull Request - State: closed - Opened by jmuk 7 months ago

#33 - Set default values for the service addr

Pull Request - State: closed - Opened by kkaneda 7 months ago

#32 - Remove the 'v' prefix from docker image tag and helm versions

Pull Request - State: closed - Opened by kkaneda 7 months ago

#31 - Replace tabs with spaces

Pull Request - State: closed - Opened by kkaneda 7 months ago

#30 - Use HuggingFace model names consistency

Pull Request - State: closed - Opened by kkaneda 7 months ago

#29 - Be able to specify S3 region

Pull Request - State: closed - Opened by kkaneda 7 months ago

#28 - Use global helm values

Pull Request - State: closed - Opened by kkaneda 7 months ago

#27 - Make Ollama pull images at startup time

Pull Request - State: closed - Opened by kkaneda 7 months ago

#26 - Update README.md

Pull Request - State: closed - Opened by kkaneda 7 months ago

#25 - Set --platform in build/engine/Dockerfile

Pull Request - State: closed - Opened by kkaneda 7 months ago

#24 - fix typos: replace tab by spaces.

Pull Request - State: closed - Opened by jmuk 7 months ago

#23 - Replace tabs with spaces

Pull Request - State: closed - Opened by kkaneda 7 months ago

#22 - Add arm64 support and latest tag

Pull Request - State: closed - Opened by kkaneda 7 months ago - 2 comments

#21 - Fix ECR repository names.

Pull Request - State: closed - Opened by jmuk 7 months ago

#20 - Add CI/CD for inference manager.

Pull Request - State: closed - Opened by jmuk 8 months ago

#19 - Convert Hugging Face model name to Ollama

Pull Request - State: closed - Opened by kkaneda 8 months ago

#18 - Log an error message on Ollama model creation

Pull Request - State: closed - Opened by kkaneda 8 months ago

#17 - Change the format of the generated models

Pull Request - State: closed - Opened by kkaneda 8 months ago

#16 - Exit(1) on error

Pull Request - State: closed - Opened by kkaneda 8 months ago

#15 - Remove the model name conversion for Ollama

Pull Request - State: closed - Opened by kkaneda 8 months ago

#14 - Update model registration flow

Pull Request - State: closed - Opened by kkaneda 8 months ago

#13 - Make lint fixes

Pull Request - State: closed - Opened by kkaneda 8 months ago

#12 - Add ingress

Pull Request - State: closed - Opened by kkaneda 8 months ago

#11 - Mount model store

Pull Request - State: closed - Opened by kkaneda 8 months ago

#10 - engine: Add helm chart

Pull Request - State: closed - Opened by kkaneda 8 months ago

#9 - engine: Initial impl

Pull Request - State: closed - Opened by kkaneda 8 months ago

#8 - Add an example of using LoRA

Pull Request - State: closed - Opened by kkaneda 8 months ago

#7 - Update README.md

Pull Request - State: closed - Opened by kkaneda 8 months ago

#6 - Fix the cache id in Dockerfile

Pull Request - State: closed - Opened by kkaneda 8 months ago

#5 - Add a note on Kaito and LocalAI

Pull Request - State: closed - Opened by kkaneda 8 months ago

#4 - Update dockerfile

Pull Request - State: closed - Opened by kkaneda 8 months ago

#3 - Add a note on available opensource

Pull Request - State: closed - Opened by kkaneda 8 months ago

#2 - Add Initial API impl

Pull Request - State: closed - Opened by kkaneda 8 months ago

#1 - Add an initial stub

Pull Request - State: closed - Opened by kkaneda 8 months ago

GitHub / llmariner/inference-manager issues and pull requests