Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / deepjavalibrary/djl-serving issues and pull requests

#2377 - Transformers NeuronX continuous batching support for Mistal 7b Instruct V3

Issue - State: open - Opened by CoolFish88 2 months ago
Labels: enhancement

#2374 - [unittest] Remove assert and add self.assertEqual

Pull Request - State: closed - Opened by sindhuvahinis 2 months ago

#2371 - [awscurl] Prints inter token latency

Pull Request - State: closed - Opened by frankfliu 2 months ago

#2369 - [fix] Format input text to avoid error

Pull Request - State: closed - Opened by xyang16 2 months ago

#2368 - [ci][fix] LCNC model tagging and accelerator count

Pull Request - State: closed - Opened by tosterberg 2 months ago

#2367 - [ci] Neuron LCNC tests small models

Pull Request - State: closed - Opened by tosterberg 2 months ago

#2365 - Model conversion process failed. Unable to find bin files

Issue - State: open - Opened by joshight 2 months ago - 1 comment
Labels: bug

#2364 - [CI] Add llama-3.1 lmi-dist test, with secure mode enabled

Pull Request - State: closed - Opened by ethnzhng 2 months ago

#2363 - [serving] add request id logging on invocations/predictions path

Pull Request - State: closed - Opened by siddvenk 2 months ago

#2361 - [serving] Updates dependencies version to latest

Pull Request - State: closed - Opened by frankfliu 2 months ago

#2359 - [ci] Reformat shell script with shfmt

Pull Request - State: closed - Opened by frankfliu 2 months ago

#2358 - [ci] minor fixes in multi-node integration test

Pull Request - State: closed - Opened by sindhuvahinis 2 months ago

#2357 - [serving] Print PIPELINE_PARALLEL_DEGREE env var

Pull Request - State: closed - Opened by xyang16 3 months ago

#2355 - Token metrics no longer computed when specifying a json query

Issue - State: closed - Opened by CoolFish88 3 months ago - 2 comments
Labels: bug

#2354 - Strange generation with Llama-3.1-70B on ml.inf2.48xlarge

Issue - State: closed - Opened by juliensimon 3 months ago - 4 comments
Labels: bug

#2353 - [serving] Updates onnxruntime to 1.19.0

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2351 - [serving] Minor code improvement

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2350 - [test][neuron] Add gpt2 test case and infinite loop guard

Pull Request - State: closed - Opened by tosterberg 3 months ago

#2349 - [serving] Removes form data size limit

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2348 - [docker][lmi] fix torch and flashattention dependency versions

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2346 - [ci] use g6 for llm integration due to capacity issues with g5

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2345 - [lmi][neuron] Add smart defaults to LMI Neuron

Pull Request - State: closed - Opened by tosterberg 3 months ago

#2343 - [docker] update vllm wheel for version required by lmi-dist

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2341 - [awscurl] Allows set max length by env var

Pull Request - State: closed - Opened by frankfliu 3 months ago - 2 comments

#2340 - awscurl: Missing token metrics when -t option specified

Issue - State: open - Opened by CoolFish88 3 months ago - 7 comments
Labels: bug

#2339 - awscurl: WARN maxLength is not explicitly specified, use modelMaxLength: 512

Issue - State: open - Opened by CoolFish88 3 months ago - 2 comments
Labels: bug

#2336 - Add simulated multi-node test

Pull Request - State: closed - Opened by nikhil-sk 3 months ago

#2335 - [Draft] Add simulated multi-node test

Pull Request - State: closed - Opened by nikhil-sk 3 months ago

#2334 - [Draft] Add EKS+LWS simulated multi-node test

Pull Request - State: closed - Opened by nikhil-sk 3 months ago

#2333 - [cherry-pick] allow list enable streaming

Pull Request - State: closed - Opened by ydm-amazon 3 months ago

#2332 - allowlist enable streaming

Pull Request - State: closed - Opened by ydm-amazon 3 months ago

#2331 - [ci] Fix awscurl run headers

Pull Request - State: closed - Opened by xyang16 3 months ago

#2330 - [Docs] Add a few missing TRT-LLM options

Pull Request - State: closed - Opened by ethnzhng 3 months ago

#2329 - LMI release notes

Pull Request - State: closed - Opened by ydm-amazon 3 months ago

#2328 - [ci] fix device_map auto change in hf handler

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2327 - [ci] fix benchmark nightly concurrency

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2326 - [awscurl] Loads AWS creadentials from EKS metadata

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2324 - [serving] Change default retry_threshold to 0

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2323 - awscurl loading aws credentials in SageMaker Studio

Issue - State: closed - Opened by acere 3 months ago
Labels: enhancement

#2322 - fix: allow chat template for non batch

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2321 - [docker] LMI Neuron bump optimum-neuron version

Pull Request - State: closed - Opened by tosterberg 3 months ago

#2320 - [ci] Add models in text embedding integration

Pull Request - State: closed - Opened by xyang16 3 months ago

#2319 - [ci] Add models in text embedding integration

Pull Request - State: closed - Opened by xyang16 3 months ago

#2318 - [feat] lmi neuronx add smart defaults context length estimates

Pull Request - State: closed - Opened by tosterberg 3 months ago

#2316 - [feat] lmi neuronx add smart defaults n_positions

Pull Request - State: closed - Opened by tosterberg 3 months ago

#2315 - Pass trust_remote_code arg to djl-convert

Pull Request - State: closed - Opened by xyang16 3 months ago

#2314 - [wlm] Minor refactor to remove unused parameter

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2312 - [0.28.0-dlc] fix the integration test to build on staging

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2310 - [secure-mode] Update options allowlist for 0.29.0

Pull Request - State: closed - Opened by ethnzhng 3 months ago

#2308 - [wlm] fail fast if one of the workers dies (#2305)

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2307 - [wlm] fail fast if one of the workers dies (#2305)

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2305 - [wlm] fail fast if one of the workers dies

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago

#2304 - [docs] Fixes broken links

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2300 - [CherryPick][TRTLLM] fix the wrong libnvinfer issues (#2298)

Pull Request - State: closed - Opened by lanking520 3 months ago

#2298 - [TRTLLM] fix the wrong libnvinfer issues

Pull Request - State: closed - Opened by lanking520 3 months ago

#2297 - [serving] Adds error message for download config.json

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2296 - [Neo] [Neuron] Various CX improvements for Neo Neuron entrypoint

Pull Request - State: closed - Opened by a-ys 3 months ago - 4 comments

#2295 - [onnx] Fixes detect need convert model logic for onnx (#2287)

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2292 - [Cherrypick][python] fix the NoneType error when decoded_token is empty (#2289)

Pull Request - State: closed - Opened by lanking520 3 months ago - 1 comment

#2291 - [cherry-pick] [docs][lmi] update user guides for lmi v11 (#2290)

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2290 - [docs][lmi] update user guides for lmi v11

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2289 - [python] fix the NoneType error when decoded_token is empty

Pull Request - State: closed - Opened by sindhuvahinis 3 months ago - 1 comment

#2288 - [Cherry-pick][TRTLLM] take out cudnn (#2286)

Pull Request - State: closed - Opened by lanking520 3 months ago

#2287 - [onnx] Fixes detect need convert model logic for onnx

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2286 - [TRTLLM] take out cudnn

Pull Request - State: closed - Opened by lanking520 3 months ago - 3 comments

#2285 - [docker] Update PyTorch to 2.4.0

Pull Request - State: closed - Opened by frankfliu 3 months ago

#2281 - add ignore_eos support in chat completions schema

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2280 - Add support for Llama-3-8B fp8 quantization with TensorRT LLM

Pull Request - State: closed - Opened by Mancera1 3 months ago

#2279 - [ci] add multimodal tests for sagemaker

Pull Request - State: closed - Opened by siddvenk 3 months ago

#2278 - [ci] fix hf hub flakiness remove unused prepare

Pull Request - State: closed - Opened by tosterberg 3 months ago - 1 comment