Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / deepjavalibrary/djl-serving issues and pull requests

#1022 - Worker type

Pull Request - State: closed - Opened by zachgk over 1 year ago

#1021 - Simplify handling of min/max workers

Pull Request - State: closed - Opened by zachgk over 1 year ago

#1020 - add some fix to the error messages

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#1019 - Allows set TENSOR_PARALLEL_DEGREE=max

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1018 - [docker] disable TORCH_CUDNN_V8_API_DISABLED for PyTorch 2.0.1

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1017 - Adds streaming docs

Pull Request - State: closed - Opened by zachgk over 1 year ago

#1016 - [HF Streaming] use decode instead batch decode for streaming

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#1015 - Fix the rolling batch integration test

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#1014 - fix vllm inference error

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#1013 - [serving] Return proper HTTP status code for each batch

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1012 - Update dependencies version

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1011 - fix the error typing

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#1010 - [serving] Adds unregister model log

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1009 - [serving] Allows print access log to console

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1008 - [python] validate each request in the batch

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1007 - [ci] fix djl_bench snapshot dep and pydantic version mismatch

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#1006 - [python] Fixes batch header key issue

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1005 - add error handling for rolling batch

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#1004 - [serving] Install commong-loggings dependency for XGBoost engine

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1003 - [docs] Updates rolling batch document

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1002 - update ft python wheel with llama support

Pull Request - State: closed - Opened by rohithkrn over 1 year ago

#1001 - [python] Includes individual headers for server side batching

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#1000 - [fix] add no-code rename step to stop runners

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#999 - Fix batch offset computation in FT handler

Pull Request - State: closed - Opened by rohithkrn over 1 year ago - 1 comment

#998 - [ci] move no-code models to s3 to avoid hub download failure

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#997 - [python] Adds pid to python process log

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#996 - [serving] Improves PyProcess lifecycle logging

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#995 - [fix] Fix Kwargs in AutoConfig

Pull Request - State: closed - Opened by KexinFeng over 1 year ago - 2 comments

#994 - Add trust_remote_code to ft handler

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#993 - Install FasterTransformer libs with llama support

Pull Request - State: closed - Opened by rohithkrn over 1 year ago

#992 - Fix the rolling batch integration test

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#991 - Set jsonlines formatter for lmi-dist rolling batch test

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#990 - Update lmi-dist

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#989 - [docker] Upgrade to DJL 0.24.0

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#988 - [python] Fixes json output formatter

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#987 - [python] Refactor lmi_dist rolling batch

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#986 - [python] Make paged attention configurable

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#985 - [serving] Fixes console log configuration

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#984 - [python] Finds optimal batch partition

Pull Request - State: closed - Opened by bryanktliu over 1 year ago

#983 - [python] Clean up dangling process in java

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#982 - Install flash attention using wheel

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#981 - [docker] bump transformers-neuronx for small llama-2 support

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#980 - [docker] bump transformers-neuronx for small llama-2 support

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#979 - Bump up DJL version to 0.24.0

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#978 - [serving] Print out CUDA and Neuron device information

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#977 - [docker] Update install inf2 script dependencies (#976)

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#976 - [docker] Update install inf2 script dependencies

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#975 - [python] Update lmi-dist

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#974 - [serving] Adds more built-in logging options

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#973 - [ci] Use smaller instance for inf2 bloom test

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#972 - [ci] Use smaller instance for inf2 bloom test

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#970 - [fix] Fix the cpu unittests issue due to device_map = 'auto'

Pull Request - State: closed - Opened by KexinFeng over 1 year ago

#969 - [fix] Fix llama model support by specifying pad token id = 0(on gpu)

Pull Request - State: closed - Opened by KexinFeng over 1 year ago - 1 comment

#968 - [fix] TNX add use_sample for llama, freeze docker DJL version

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#967 - [serving] Update tnx handler for 2.12 supported models

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#966 - [docs] Update rolling batch document

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#965 - Add built-in json formatter

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#964 - Enable MPI model by environment variable

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#963 - [serving] Update djlbench snapcraft version to 0.23.0

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#962 - Unable to to download from s3

Issue - State: closed - Opened by monuminu over 1 year ago - 1 comment
Labels: bug

#961 - [wlm] Allows set defatul options with environment variable

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#960 - Enable multi-gpu inference (device_map='auto') on seq_batch_scheduler

Pull Request - State: closed - Opened by KexinFeng over 1 year ago

#959 - patch api-0.23.0 for streaming timeout issue

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#958 - Add support for testing candidate release images in sagemaker tests

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#956 - [docker] [0.23.0-dlc] Update release version and wheels

Pull Request - State: closed - Opened by sindhuvahinis over 1 year ago

#955 - Update docs to djl 0.23.0

Pull Request - State: closed - Opened by sindhuvahinis over 1 year ago

#954 - Update docs to djl 0.23.0

Pull Request - State: closed - Opened by sindhuvahinis over 1 year ago

#953 - Allow overriding truncate parameter in request

Pull Request - State: closed - Opened by maaquib over 1 year ago

#952 - Fix some issues with remote code for lora

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#951 - [docs] Adds document about venv per model

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#950 - Listener timed out in: 30.0 s

Issue - State: open - Opened by yuxin7 over 1 year ago - 3 comments
Labels: bug

#949 - add model revision environment variable

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#948 - add revision in test

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#947 - add revision as part of the model inputs

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#946 - Fix boolean kwargs and typo in load_in_4_bit assignment

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#945 - [docs] Adds s5cmd feature in document

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#944 - bump up bitsandbytes on its fixes

Pull Request - State: closed - Opened by lanking520 over 1 year ago

#943 - [python] Fix the default value for rolling batch request parameters

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#942 - Add lora tests for fastertransformer

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#941 - [serving] Disconnect client when streaming timed out

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#940 - [ci] remove oom tests for hf accelerate performance

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#939 - [python] Send error message in json format

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#938 - [python] Add null check for prefill batch

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#937 - [python] Fixes logging bug

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#936 - [docker] bump bitsandbytes versions

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#935 - [python] Fix repeated output for rolling batch

Pull Request - State: closed - Opened by xyang16 over 1 year ago

#934 - [python] set default maxWorkers to 1 if not configured for TP

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#933 - [docs] Updates model configuration document

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#932 - Add lora support to ft default handler

Pull Request - State: closed - Opened by siddvenk over 1 year ago

#931 - Adding gpt-neox-20b-quantized to workflow

Pull Request - State: closed - Opened by maaquib over 1 year ago - 1 comment

#930 - [python] Only override minWorkers when tp > 1

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#929 - KV cache support in default handler

Pull Request - State: closed - Opened by sindhuvahinis over 1 year ago

#928 - [Fix] bug_fix_for_empty_tensor_input

Pull Request - State: closed - Opened by KexinFeng over 1 year ago

#927 - [fix] add current device for tp > 1 scenario on huggingface handler

Pull Request - State: closed - Opened by tosterberg over 1 year ago

#926 - OOM management doc

Pull Request - State: closed - Opened by rohithkrn over 1 year ago

#925 - [serving] Adds batch size metric

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#924 - [python] Fixes huggingface logging bug

Pull Request - State: closed - Opened by frankfliu over 1 year ago

#923 - [Fix] A bug fix in runtime kv_cache

Pull Request - State: closed - Opened by KexinFeng over 1 year ago