Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / microsoft/DeepSpeed-MII issues and pull requests

#343 - Loading Bloom-3b in persisten mode fails

Issue - State: closed - Opened by ghpu about 1 year ago - 1 comment

#342 - support chatglm3

Issue - State: open - Opened by zhaozhaoxia about 1 year ago - 1 comment
Labels: model request

#341 - RuntimeError: There is no current event loop in thread 'Thread-1'.

Issue - State: closed - Opened by 4thGardenOfQMH about 1 year ago - 2 comments

#340 - AttributeError: 'NoneType' object has no attribute 'value'

Issue - State: closed - Opened by flexwang about 1 year ago - 2 comments

#339 - Deadlock detected

Issue - State: open - Opened by flexwang about 1 year ago - 13 comments

#338 - Invalid parameter bricks service

Issue - State: open - Opened by mevince about 1 year ago - 1 comment
Labels: bug, enhancement

#337 - fix PR creation in release workflow

Pull Request - State: closed - Opened by mrwyattii about 1 year ago

#336 - Support for Codellama Model in deepspeed-fastgen

Issue - State: open - Opened by LinKiling about 1 year ago - 2 comments

#335 - Base model support

Issue - State: open - Opened by Qingyuncookie about 1 year ago - 1 comment
Labels: model request

#334 - Handling of edge case for text to image model

Pull Request - State: closed - Opened by gauravrajguru about 1 year ago

#333 - What is the recommended way to wrap deepspeed mii.client in a service?

Issue - State: open - Opened by sfc-gh-zhwang about 1 year ago - 1 comment

#332 - mii.serve vs mii.client

Issue - State: closed - Opened by flexwang about 1 year ago - 3 comments

#331 - Add device map config

Pull Request - State: closed - Opened by mrwyattii about 1 year ago

#330 - FastAPI non-persistent wrapper

Issue - State: open - Opened by mevince about 1 year ago - 1 comment

#329 - Is pipeline parallelism supported?

Issue - State: open - Opened by sleepwalker2017 about 1 year ago - 11 comments

#328 - Multi-process RESTful API

Pull Request - State: closed - Opened by mrwyattii about 1 year ago

#327 - fail to run llama-2-7B and llama-2-13B

Issue - State: open - Opened by xzzWZY about 1 year ago - 2 comments

#326 - Add support for zero shot image classification task

Pull Request - State: closed - Opened by PhaniShekhar about 1 year ago

#325 - Low throughput (0.61 reqs/sec) when served with RESTful API

Issue - State: closed - Opened by noiji about 1 year ago - 4 comments

#324 - Performance of RESTful API

Issue - State: open - Opened by kihanc about 1 year ago - 9 comments

#323 - 0.6 req /s is kinda low ,for real?

Issue - State: open - Opened by chuangzhidan about 1 year ago - 2 comments

#321 - Add RTD

Pull Request - State: closed - Opened by mrwyattii about 1 year ago

#319 - Is there gonna be metrics endpoint exposed?

Issue - State: open - Opened by flexwang about 1 year ago - 1 comment

#318 - What is the recommended way of bringing up mii as a service

Issue - State: open - Opened by flexwang about 1 year ago - 1 comment

#317 - Adding OpenAI Compatible RESTful API

Pull Request - State: closed - Opened by PawanOsman over 1 year ago - 19 comments

#316 - openai compatible api

Issue - State: closed - Opened by dongxiaolong over 1 year ago
Labels: enhancement, good first issue

#315 - Use smaller model for unit tests

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#314 - Why is the throughput of mii lower than that of vllm in actual measurements?

Issue - State: closed - Opened by pangr over 1 year ago - 6 comments

#313 - Questions about token throughput about dynamic splitfuse

Issue - State: closed - Opened by ChuanhongLi over 1 year ago - 1 comment

#311 - readable token streaming support

Pull Request - State: closed - Opened by jeffra over 1 year ago - 3 comments

#310 - Is multiple gpu supported with non-persistent pipeline

Issue - State: closed - Opened by yaliqin over 1 year ago - 1 comment

#309 - Illegal memory access error when infering input of length 100K

Issue - State: open - Opened by frankxyy over 1 year ago - 4 comments

#308 - can not test with restful_api

Issue - State: open - Opened by irasin over 1 year ago - 14 comments

#307 - Where to get log of server?

Issue - State: open - Opened by frankxyy over 1 year ago - 1 comment
Labels: enhancement

#306 - Support for token streaming

Issue - State: open - Opened by Archmilio over 1 year ago

#305 - Unable to load relatively large opt models (opt-6.7b opt-30b)

Issue - State: open - Opened by MeloYang05 over 1 year ago - 5 comments

#304 - Server launching error for model Yi-6B-200K-Llamafied

Issue - State: closed - Opened by frankxyy over 1 year ago

#303 - unable to build model pipeline

Issue - State: open - Opened by sumitsahaykoantek over 1 year ago - 4 comments

#301 - The result is irrelevant

Issue - State: closed - Opened by pangr over 1 year ago

#299 - [Serving Stability] one request crashed, other requests can not be posted

Issue - State: closed - Opened by frankxyy over 1 year ago - 3 comments

#298 - How to get num_of_new_tokens while calling client.generate()

Issue - State: closed - Opened by frankxyy over 1 year ago - 3 comments

#297 - tp > 1 inference is very slow

Issue - State: open - Opened by easonfzw over 1 year ago - 2 comments

#296 - Add safetensors support

Pull Request - State: closed - Opened by jihnenglin over 1 year ago - 1 comment

#295 - Unify input/output types

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#294 - Update RESTful API

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#293 - how to set stop tokens?

Issue - State: open - Opened by PawanOsman over 1 year ago - 3 comments

#292 - cannot send top_p temperature parameters through client.generate api calling

Issue - State: closed - Opened by frankxyy over 1 year ago - 3 comments

#291 - Serving error when input of large length is sent

Issue - State: open - Opened by frankxyy over 1 year ago - 4 comments

#290 - Why did an unknown error occur in gprc calls when enable_restful_api is true

Issue - State: closed - Opened by wtontheway over 1 year ago - 4 comments

#289 - TypeError: cannot unpack non-iterable Response object

Issue - State: closed - Opened by flexwang over 1 year ago - 7 comments

#288 - Non deterministic generation result from the same prompt

Issue - State: closed - Opened by flexwang over 1 year ago - 3 comments

#287 - Why generation_config.json is a requirement for start server?

Issue - State: closed - Opened by flexwang over 1 year ago - 6 comments

#286 - Is beam search supported?

Issue - State: open - Opened by flexwang over 1 year ago - 2 comments

#285 - Question around DSStateManagerConfig.max_ragged_batch_size

Issue - State: closed - Opened by flexwang over 1 year ago - 3 comments

#284 - Compatibility with DS Inference KV-cache flexibility PR

Pull Request - State: closed - Opened by cmikeh2 over 1 year ago

#283 - How to select specific gpu index when using tensor parallel?

Issue - State: closed - Opened by frankxyy over 1 year ago - 2 comments

#282 - Server turns into broken state if queried with very long prompt

Issue - State: open - Opened by ttim over 1 year ago - 1 comment

#281 - Streaming api seems broken

Issue - State: open - Opened by ttim over 1 year ago - 2 comments

#280 - Add more generate() kwargs

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#279 - prevent load_with_sys_mem when using stable diffusion

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#278 - Recompute when the deadlock is detected

Pull Request - State: closed - Opened by tohtana over 1 year ago

#277 - Installed CUDA version 11.7 does not match torch version

Issue - State: closed - Opened by frankxyy over 1 year ago - 3 comments

#276 - Update precommit formatting and yapf to match DeepSpeed

Pull Request - State: closed - Opened by loadams over 1 year ago

#275 - Enable multi-prompt input for persistent deployments

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#274 - Mitigate the risk of deadlock

Pull Request - State: closed - Opened by tohtana over 1 year ago

#273 - Unable to load ragged_device_ops op due to no compute capabilities remaining after filtering

Issue - State: open - Opened by rogerbock over 1 year ago - 10 comments
Labels: enhancement

#272 - `FileNotFoundError: No such file or directory: pytorch_model.bin` while loading a HF repository

Issue - State: open - Opened by jihnenglin over 1 year ago - 1 comment
Labels: good first issue

#271 - [FastGen] Hot-swappable LoRA adapters?

Issue - State: open - Opened by corbt over 1 year ago - 1 comment

#270 - Support AsyncPipeline for RESTful API

Issue - State: closed - Opened by toilaluan over 1 year ago - 5 comments

#269 - Reorganize code structure, fix client import bug

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#268 - Expose top-p, top-k, and temperature to generate APIs

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#267 - set device on inference pipeline only if setter available

Pull Request - State: closed - Opened by gauravrajguru over 1 year ago - 1 comment

#266 - Serve Fails while Pipeline is working

Issue - State: closed - Opened by aliozts over 1 year ago - 4 comments

#265 - terminate_server only release memory on one gpu when using tensor_parallel

Issue - State: closed - Opened by baojunliu over 1 year ago - 2 comments

#264 - diffuser model load using model and path params

Pull Request - State: closed - Opened by gauravrajguru over 1 year ago

#263 - DeepSpeed MII Serve error on V100

Issue - State: closed - Opened by amazingkmy over 1 year ago - 4 comments

#262 - Add MII v0.1 unit tests

Pull Request - State: closed - Opened by mrwyattii over 1 year ago

#261 - Add ability to configure temperature, top P, top K, number of beams

Issue - State: closed - Opened by ttim over 1 year ago - 2 comments

#260 - Provide async api in MII client

Issue - State: open - Opened by ttim over 1 year ago - 2 comments

#259 - Fix typo in README.md

Pull Request - State: closed - Opened by eltociear over 1 year ago

#258 - Server crashes whilst trying to spin up Mistral

Issue - State: open - Opened by harryjulian over 1 year ago - 8 comments

#257 - Time to First Token almost same as vllm for large prompts

Issue - State: closed - Opened by idealover over 1 year ago - 2 comments

#255 - Quantization Support for Fastgen?

Issue - State: open - Opened by aliozts over 1 year ago - 4 comments

#254 - [FEATURE] Speculative Decoding

Issue - State: open - Opened by casper-hansen over 1 year ago

#253 - Issues with llama 2 model example

Issue - State: open - Opened by ttim over 1 year ago - 1 comment

#252 - MII v0.1.0 release

Pull Request - State: closed - Opened by tohtana over 1 year ago

#251 - Fail to compile when kicking off the example

Issue - State: open - Opened by mozizhao over 1 year ago

#250 - DeepSpeed bug multi-gpu in single node

Issue - State: open - Opened by muhammad-asn over 1 year ago - 1 comment

#248 - Improved the code quality to ease future maintenance

Pull Request - State: closed - Opened by blackmambaza over 1 year ago - 1 comment

#247 - Loadams/update yapf

Pull Request - State: closed - Opened by loadams over 1 year ago

#246 - Update version.txt after 0.0.8 release

Pull Request - State: closed - Opened by loadams over 1 year ago

#245 - Update autoPR creation in release script

Pull Request - State: closed - Opened by loadams over 1 year ago

#244 - Fixes for AML metatensor loading

Pull Request - State: closed - Opened by mrwyattii over 1 year ago