microsoft/DeepSpeed-MII issues and pull requests

#421 - Remove references to --extra-index-url in MII repo

Pull Request - State: closed - Opened by loadams 7 months ago

#420 - How to set trust_remote_code=True in pipeline

Issue - State: open - Opened by gujingit 7 months ago - 2 comments

#419 - Fp6 eta

Issue - State: open - Opened by nivibilla 7 months ago - 2 comments

#418 - Use of dtype in the mii fastgen

Issue - State: open - Opened by gangooteli 7 months ago - 1 comment

#417 - How does GPT2/Bert models utilize continuous batching feature in MII?

Issue - State: open - Opened by Jye-525 7 months ago - 1 comment

#416 - Is the DeepSpeed-MII will support habana (HPU) hardware?

Issue - State: open - Opened by muhammad-asn 7 months ago - 2 comments

#415 - Add `accelerate` to requirements to improve MII-legacy model load times

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#414 - Add test for loading from local dir

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#413 - Update version.txt after 0.2.2 release

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#412 - add stable diffusion CI workflow

Pull Request - State: open - Opened by mrwyattii 8 months ago - 1 comment

#411 - Disable model check in UT

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#410 - Add support for inpainting task in DS-MII

Pull Request - State: closed - Opened by gauravrajguru 8 months ago

#409 - fix: Fixed the issue where the mii.pipeline.pipe(stop) was ineffective

Pull Request - State: closed - Opened by kitstar 8 months ago - 2 comments

#408 - Fix for missing EOS token

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#407 - text2img task to support negative prompts

Pull Request - State: closed - Opened by gauravrajguru 8 months ago

#406 - How to generate multiple responses in one time?

Issue - State: open - Opened by yangzhch6 8 months ago - 1 comment

#405 - TypeError: expected Tensor as element 0 in argument 0, but got bool

Issue - State: closed - Opened by SiriusWy 8 months ago - 1 comment

#404 - Update version.txt after 0.2.1 release

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#403 - Improve recovery from KV cache starvation

Pull Request - State: closed - Opened by tohtana 8 months ago - 1 comment

#402 - The inference result is inconsistent with hf

Issue - State: open - Opened by bao-xiaoyi 8 months ago - 1 comment

#401 - Fix generate output order

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#400 - RuntimeError: server crashed for some reason, unable to proceed

Issue - State: closed - Opened by bao-xiaoyi 8 months ago - 2 comments

#399 - result is empty

Issue - State: closed - Opened by bao-xiaoyi 8 months ago

#398 - ModuleNotFoundError: No module named 'mii'

Issue - State: closed - Opened by bao-xiaoyi 8 months ago - 2 comments

#397 - Readable token streaming support

Pull Request - State: closed - Opened by greshilov 8 months ago - 2 comments

#396 - Support for repetition penalty during inference with sampling

Issue - State: open - Opened by nischith-sarvam 8 months ago - 1 comment

#395 - Benchmark:Performance is lower than vllm

Issue - State: open - Opened by zhaotyer 8 months ago - 1 comment

#394 - Fix recovery from deadlock

Pull Request - State: closed - Opened by tohtana 8 months ago

#393 - Unable to run inference on free tier Colab.

Issue - State: closed - Opened by sudhir2016 8 months ago - 4 comments

#392 - Update CI workflows

Pull Request - State: closed - Opened by loadams 8 months ago

#391 - Update landing page

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#390 - did "mii.pipeline" support float16?

Issue - State: closed - Opened by wangrendong-yition 8 months ago - 3 comments

#389 - support for mixtral family ?

Issue - State: open - Opened by S-Yacer 8 months ago - 8 comments

#388 - How to eliminate deadlock problem?

Issue - State: open - Opened by BaiStone2017 8 months ago - 1 comment

#387 - import mii not working

Issue - State: open - Opened by pradeepdev-1995 8 months ago - 5 comments

#386 - Error on unknown generate fields

Pull Request - State: closed - Opened by mrwyattii 8 months ago

#385 - `ValueError: channels must be divisible by 8` when new special tokens are added

Issue - State: open - Opened by s-jse 8 months ago - 3 comments

#384 - Make the order of outputs the same as the order of inputs when using `mii.pipeline`

Pull Request - State: closed - Opened by s-jse 8 months ago - 1 comment

#383 - RuntimeError: The server socket has failed to listen on any local network address. The server socket has failed to bind to [::]:29700 (errno: 98 - Address already in use).

Issue - State: open - Opened by Chenhzjs 9 months ago - 2 comments

#382 - Update version.txt after 0.2.0 release

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#381 - Error: "Only able to place X replicas, but Y replicas were requested"

Issue - State: open - Opened by spring1915 9 months ago - 2 comments

#380 - Update required DS version

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#379 - deep speed parallel erro

Issue - State: open - Opened by king-shao 9 months ago - 1 comment

#378 - Test

Pull Request - State: closed - Opened by deas23 9 months ago

#377 - Improve efficiency of scheduling and token sampiling

Pull Request - State: closed - Opened by tohtana 9 months ago

#376 - Improve efficiency of ragged batching scheduler

Pull Request - State: closed - Opened by tohtana 9 months ago - 1 comment

#375 - Remove inefficient loop in TopP logits processor

Pull Request - State: closed - Opened by tohtana 9 months ago - 1 comment

#374 - Mistral 8*7B Out of memory

Issue - State: open - Opened by byerose 9 months ago - 1 comment

#373 - Add model support unit test

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#372 - Make generate params pydantic model

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#371 - pydantic.errors.PydanticUserError

Issue - State: closed - Opened by ArlanCooper 9 months ago - 2 comments

#370 - Restrict when legacy unit tests are run

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#369 - fix address already in use error on UT

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#368 - How to get the logit tensor of generated text?

Issue - State: open - Opened by randomx207 9 months ago - 4 comments

#367 - fix bug when mii_config is None

Pull Request - State: closed - Opened by mrwyattii 9 months ago

#366 - I wonder if we can use batch inference and offload in mii pipeline ?

Issue - State: open - Opened by Kevin-shihello-world 9 months ago - 2 comments

#365 - for loop calling Non Persistent Pipeline will cause Deadlock

Issue - State: open - Opened by CxsGhost 9 months ago - 1 comment

#364 - Add restful_api_host into server args.

Pull Request - State: closed - Opened by sarattha 9 months ago

#363 - one of mii.client() Options, ignore_eos doesn't work

Issue - State: closed - Opened by BaiStone2017 9 months ago

#362 - restful_api_host did not use in anywhere

Issue - State: open - Opened by Bhurinut 9 months ago - 1 comment

#361 - When running mii.serv, it keeps print waiting for server to start.

Issue - State: closed - Opened by cninnovationai 9 months ago - 6 comments

#360 - Update supported models list

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#359 - Add pipeline unit tests

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#358 - Problem while running facebook/opt-125m with MII

Issue - State: closed - Opened by wangtianxia-sjtu 10 months ago - 2 comments

#357 - Can MII support quanted Llama2 of AWQ?

Issue - State: closed - Opened by janelu9 10 months ago

#356 - Reproduced readme results

Issue - State: open - Opened by Traveller2001 10 months ago - 8 comments

#355 - Update version.txt after 0.1.3 release

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#354 - Deployment in kubernetes

Issue - State: open - Opened by nani1149 10 months ago

#353 - mixtral support

Issue - State: closed - Opened by martinshkreli 10 months ago - 2 comments
Labels: enhancement

#352 - Loosen unit test performance assert

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#351 - Can you support DeepSeek's inference acceleration? Thank you very much.

Issue - State: open - Opened by joyhhheee 10 months ago - 3 comments

#350 - Fix for error messages in persistent deployment

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#349 - Error messages spilled from persistent deployment for every request

Issue - State: closed - Opened by weiqisun 10 months ago - 6 comments

#348 - Add RESTful API option for host

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#347 - How to stream tokens?

Issue - State: open - Opened by mevince 10 months ago - 1 comment
Labels: enhancement

#346 - The choice of the split size for splitAndFuse

Issue - State: open - Opened by frankxyy 10 months ago - 4 comments

#345 - Can deepspeed-MII run on AMD GPU?

Issue - State: open - Opened by sunpian1 10 months ago - 2 comments
Labels: enhancement

#344 - restful api host need configuration

Issue - State: closed - Opened by cableyang 10 months ago - 1 comment
Labels: enhancement

#343 - Loading Bloom-3b in persisten mode fails

Issue - State: closed - Opened by ghpu 10 months ago - 1 comment

#342 - support chatglm3

Issue - State: open - Opened by zhaozhaoxia 10 months ago - 1 comment
Labels: model request

#341 - RuntimeError: There is no current event loop in thread 'Thread-1'.

Issue - State: closed - Opened by 4thGardenOfQMH 10 months ago - 2 comments

#340 - AttributeError: 'NoneType' object has no attribute 'value'

Issue - State: closed - Opened by flexwang 10 months ago - 2 comments

#339 - Deadlock detected

Issue - State: open - Opened by flexwang 10 months ago - 13 comments

#338 - Invalid parameter bricks service

Issue - State: open - Opened by mevince 10 months ago - 1 comment
Labels: bug, enhancement

#337 - fix PR creation in release workflow

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#336 - Support for Codellama Model in deepspeed-fastgen

Issue - State: open - Opened by LinKiling 10 months ago - 2 comments

#335 - Base model support

Issue - State: open - Opened by Qingyuncookie 10 months ago - 1 comment
Labels: model request

#334 - Handling of edge case for text to image model

Pull Request - State: closed - Opened by gauravrajguru 10 months ago

#333 - What is the recommended way to wrap deepspeed mii.client in a service?

Issue - State: open - Opened by sfc-gh-zhwang 10 months ago - 1 comment

#332 - mii.serve vs mii.client

Issue - State: closed - Opened by flexwang 10 months ago - 3 comments

#331 - Add device map config

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#330 - FastAPI non-persistent wrapper

Issue - State: open - Opened by mevince 10 months ago - 1 comment

#329 - Is pipeline parallelism supported?

Issue - State: open - Opened by sleepwalker2017 10 months ago - 11 comments

#328 - Multi-process RESTful API

Pull Request - State: closed - Opened by mrwyattii 10 months ago

#327 - fail to run llama-2-7B and llama-2-13B

Issue - State: open - Opened by xzzWZY 10 months ago - 2 comments

#326 - Add support for zero shot image classification task

Pull Request - State: closed - Opened by PhaniShekhar 10 months ago

#325 - Low throughput (0.61 reqs/sec) when served with RESTful API

Issue - State: closed - Opened by noiji 10 months ago - 4 comments

#324 - Performance of RESTful API

Issue - State: open - Opened by kihanc 10 months ago - 9 comments

#323 - 0.6 req /s is kinda low ,for real?

Issue - State: open - Opened by chuangzhidan 11 months ago - 2 comments

#322 - Is there a way for mii to not occupy all the available gpu memory

Issue - State: open - Opened by flexwang 11 months ago

GitHub / microsoft/DeepSpeed-MII issues and pull requests