Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / deepjavalibrary/djl-serving issues and pull requests

#1123 - Upgrade flash attention v2 version to 2.3.0

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1122 - Integrate with seq_scheduler wheel

Pull Request - State: closed - Opened by KexinFeng about 1 year ago

#1121 - Add dependency on stop-runners for lora correctness test

Pull Request - State: closed - Opened by rohithkrn about 1 year ago

#1120 - [fix] Device_and_search_config_issue

Pull Request - State: closed - Opened by KexinFeng about 1 year ago

#1119 - [Docker] upgrade dependencies version

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1118 - Add support for stopwords in huggingface handler

Pull Request - State: closed - Opened by ydm-amazon about 1 year ago - 5 comments

#1117 - [INF2][Handler] fix none type check

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1116 - Handling requests concurrently: AWS Sagemaker endpoint

Issue - State: closed - Opened by sneha5gsm about 1 year ago - 3 comments

#1115 - [Handler] add neuron int8 quantization

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1114 - Unmerged lora correctness test

Pull Request - State: closed - Opened by rohithkrn about 1 year ago - 1 comment

#1112 - [vLLM] capture max_rolling_batch settting issues

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1111 - add flash2 support for huggingface accelerate

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1110 - Add unmerged lora integration test

Pull Request - State: closed - Opened by rohithkrn about 1 year ago - 1 comment

#1109 - Creates auto increment ID for models

Pull Request - State: closed - Opened by zachgk about 1 year ago

#1108 - Upgrade lmi_dist

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1107 - [INF2][Handler] added optimization level per Neuron instruction

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1106 - Does deepspeed engine support Llama2 question-answering task?

Issue - State: open - Opened by YunTaoYoung about 1 year ago - 1 comment
Labels: bug

#1105 - fix deepspeed bugs and have better logging

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1104 - Fix typo

Pull Request - State: closed - Opened by rohithkrn about 1 year ago

#1103 - Use custom peft wheel

Pull Request - State: closed - Opened by rohithkrn about 1 year ago

#1102 - [CI] fix the inf2 container build failure

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1101 - [feature] Test llama-7b-gptq on scheduler_rolling_batch

Pull Request - State: closed - Opened by KexinFeng about 1 year ago - 3 comments

#1100 - Cleans some unused pieces of PyProcess

Pull Request - State: closed - Opened by zachgk about 1 year ago

#1099 - Fix setting adapters arg

Pull Request - State: closed - Opened by rohithkrn about 1 year ago

#1098 - fix some bugs in handler

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1097 - add no code testing for rollingbatch

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1096 - Allow DJLServing to take override engine name

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1095 - Removes quick abort of python reader threads

Pull Request - State: closed - Opened by zachgk about 1 year ago - 4 comments

#1094 - Creates adapters by directory

Pull Request - State: closed - Opened by zachgk about 1 year ago - 2 comments

#1093 - [vLLM] add pyarrow dependency

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1092 - tensor parallelism across multiple GPU's

Issue - State: closed - Opened by samanthvishwas about 1 year ago - 2 comments

#1091 - [INF2] grant write permission

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1090 - [INF2] fix some bugs and remove old tests

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1089 - [Neuron] upgrade to Neuron 2.14.0 SDK

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1088 - Add unmerged lora support in HF handler

Pull Request - State: closed - Opened by rohithkrn about 1 year ago - 4 comments

#1087 - [INF2] don't install linux headers

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1085 - [demo] PR

Pull Request - State: closed - Opened by tosterberg about 1 year ago

#1084 - How to allow parallel requests

Issue - State: closed - Opened by thoth42 about 1 year ago - 2 comments

#1083 - [serving] Fixes log rotation issue

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1082 - Adds adapter support

Pull Request - State: closed - Opened by zachgk about 1 year ago

#1081 - [handler] fix a few issues

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1080 - Fixes API responses

Pull Request - State: closed - Opened by zachgk about 1 year ago

#1079 - Fix neuron cores auto-scaling

Pull Request - State: open - Opened by fafriat about 1 year ago - 18 comments

#1078 - add Neuron RollingBatch implementation

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1077 - [Handler] formalize all engines with same settings

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1076 - [Handler] add dynamic batching to transformers neuronx

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1075 - What backend does djl-serving default to for rolling_batch?

Issue - State: open - Opened by yudhiesh about 1 year ago - 3 comments
Labels: bug

#1074 - [fix] Fix device map

Pull Request - State: closed - Opened by KexinFeng about 1 year ago

#1073 - [docker] Change default OMP_NUM_THREADS back to 1 for GPU

Pull Request - State: closed - Opened by frankfliu about 1 year ago - 4 comments

#1072 - [CI] add vllm tests

Pull Request - State: closed - Opened by lanking520 about 1 year ago - 1 comment

#1071 - [CherryPick] use tag for test and not hardcode

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1070 - [RollingBatch][CI] use tag for test and not hardcode

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1069 - Revert "Upgrade lmi_dist"

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1068 - Revert "Upgrade lmi_dist"

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1067 - [serving] Fixes wrong device mapping for non-tp mode

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1066 - update lmi_dist package to 0.9.4

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1065 - [Handler] fix device mapping issues

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1064 - [Handler] make huggingface streamer default

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1063 - [ci] Fixes gradle deprecation warnings

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1062 - [ci] Fixes PMD warning

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1061 - [serving] Make sure extracting jni from jar file in deps folder

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1060 - Upgrade lmi_dist

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1059 - Upgrade lmi_dist

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1058 - [INF2] disable checker for saved model

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1057 - [INF2] Disable checker for load saved model

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1056 - add revision to handler

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1055 - [fix] Device and format and implementation optimization

Pull Request - State: closed - Opened by KexinFeng about 1 year ago

#1054 - [0.23.0 cherry pick] #1034 #1035 #1036 #1041 vLLM changes

Pull Request - State: closed - Opened by lanking520 about 1 year ago - 1 comment

#1052 - [docker] Upgrades to inf2 2.13.2 version

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1050 - [RollingBatch] create request simulator to batch

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1049 - [DeepSpeed] upgrade dependencies

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1048 - Revert "Change rolling batch generations order (#1046)"

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1047 - add data collection and some inf2 bug fixes

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1046 - Change rolling batch generations order

Pull Request - State: closed - Opened by xyang16 about 1 year ago

#1045 - [python] Avoid holding the lock while running inference

Pull Request - State: closed - Opened by frankfliu about 1 year ago - 1 comment

#1044 - update inf2 dependencies to 2.13.1

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1043 - [fix] Two major bugs in rolling_batch_scheduler

Pull Request - State: closed - Opened by KexinFeng about 1 year ago - 1 comment

#1042 - How to use inference of multiple models

Issue - State: open - Opened by faquir-sun about 1 year ago - 5 comments

#1041 - [rollingbatch] add standalone script to run

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1040 - [docker] add version labels for sagemaker

Pull Request - State: closed - Opened by tosterberg about 1 year ago - 2 comments

#1039 - [docs] Document Python engine alias

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1038 - Fixes OOM checker bug

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1037 - [python] Fixes tokenizer bug when using hugging pipeline

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1036 - [VLLM] add option to set batched tokens

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1035 - [VLLM] use more complex logic to ensure all result are captured

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1034 - fix logging bug in vllm

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1033 - Fixed vllm rolling batcher bugs

Pull Request - State: closed - Opened by lanking520 about 1 year ago

#1032 - [docs] Updates TENSOR_PARALLEL_DEGREE description

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1031 - [benchmark] Adds safetensors support

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1030 - [docs] Adds log4j configuration document

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1029 - [ci] Upgrades gradle to 8.3

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1028 - Add FT llama integration test

Pull Request - State: closed - Opened by rohithkrn about 1 year ago

#1027 - [docker] Updates cache directory

Pull Request - State: closed - Opened by frankfliu about 1 year ago

#1026 - Adding docs for llm tuning params

Pull Request - State: closed - Opened by maaquib about 1 year ago

#1025 - How to use vLLM with djl-serving?

Issue - State: closed - Opened by yudhiesh about 1 year ago - 6 comments

#1024 - Adding LLM tuning guide

Pull Request - State: closed - Opened by maaquib about 1 year ago - 2 comments

#1023 - add mpt and starcoder tests

Pull Request - State: closed - Opened by lanking520 about 1 year ago