Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / microsoft/DeepSpeed-MII issues and pull requests
#499 - Test
Pull Request -
State: closed - Opened by trapp3rhat 4 days ago
#498 - [QUERY] Expert Parallelism Supported?
Issue -
State: open - Opened by Shamauk 5 days ago
#497 - Attempting to flush sequence N which does not exist
Issue -
State: open - Opened by aagontuk 8 days ago
#496 - Compute perplexity
Issue -
State: open - Opened by Sh1gechan 11 days ago
#495 - Configure server log level
Issue -
State: open - Opened by sedletsky-f5 13 days ago
#494 - few questions regarding the implementation of streaming and batching
Issue -
State: open - Opened by KimMinSang96 19 days ago
#493 - Add explanations of MII code into comments
Pull Request -
State: open - Opened by mrwyattii 20 days ago
- 1 comment
#492 - Remove Conversation from MII as it was deprecated and removed from transformers.
Pull Request -
State: closed - Opened by loadams 25 days ago
- 1 comment
#491 - Always Flush UIDs after Exceptions
Pull Request -
State: open - Opened by weiqisun 27 days ago
#490 - Always Flush UIDs after `GeneratorReply`
Pull Request -
State: closed - Opened by weiqisun 27 days ago
- 1 comment
#489 - [BUG] MII Backend Hangs After 9999 Exceptions in `MIIAsyncPipeline.put_request`
Issue -
State: open - Opened by weiqisun 27 days ago
- 1 comment
#488 - support stream
Issue -
State: open - Opened by ZZhangxian 30 days ago
#487 - support Qwen1.5
Issue -
State: open - Opened by ZZhangxian 30 days ago
#486 - support Qwen
Issue -
State: open - Opened by ZZhangxian 30 days ago
#485 - Some fixes to make openai entrypoint work out of the box
Pull Request -
State: open - Opened by svaruag about 1 month ago
#484 - Reuse KV cache of prefixes
Pull Request -
State: open - Opened by tohtana about 1 month ago
#483 - Support LLava next stronger
Issue -
State: open - Opened by thesby about 1 month ago
#482 - How can I use the same prompt to produce the same text output as vllm
Issue -
State: open - Opened by Greatpanc about 1 month ago
#481 - Tf32 support
Issue -
State: open - Opened by Chasapas about 2 months ago
#480 - Enable streaming option in the OpenAI API server
Pull Request -
State: open - Opened by adk9 about 2 months ago
#479 - DeepSpeed-MII 能加载量化的int4或者int8的模型吗?
Issue -
State: open - Opened by wangyongpenga about 2 months ago
#478 - Fix deprecation warning on escaped characters
Pull Request -
State: closed - Opened by loadams about 2 months ago
#477 - Does deepspeed-mii support prefix_allowed_tokens_fn?
Issue -
State: open - Opened by zcakzhuu about 2 months ago
#476 - Update mistral tests to fully open source version.
Pull Request -
State: closed - Opened by loadams about 2 months ago
#475 - [REQUEST] LLAMA-3 support
Issue -
State: open - Opened by MRYingLEE about 2 months ago
#474 - [REQUEST] Mixtral-8x22B support
Issue -
State: open - Opened by y-live-koba about 2 months ago
#473 - Allow model to generate added tokens - fix generation issue in Llama3 models
Pull Request -
State: open - Opened by weiqisun about 2 months ago
- 7 comments
#472 - Cannot run Yi-34B-Chat => ValueError: Unsupported q_ratio: 7
Issue -
State: open - Opened by joeking11829 about 2 months ago
- 2 comments
#471 - BUG in run_batch_processing
Issue -
State: open - Opened by zhihui96 about 2 months ago
#470 - fix max_ragged_sequence_count check in _schedule_prompts
Pull Request -
State: closed - Opened by dc3671 about 2 months ago
- 1 comment
#469 - ValueError: Unsupported model type phi3
Issue -
State: open - Opened by abpani 2 months ago
#468 - error when using Qwen1.5-32B
Issue -
State: open - Opened by puppet101 2 months ago
#467 - Performance with vllm
Issue -
State: open - Opened by littletomatodonkey 2 months ago
#466 - [Problem]errno: 98 - Address already in use
Issue -
State: closed - Opened by littletomatodonkey 2 months ago
#465 - Only running one replica even though setting many replicas
Issue -
State: open - Opened by thesby 2 months ago
#464 - RuntimeError: The server socket has failed to listen on any local network address
Issue -
State: open - Opened by thesby 2 months ago
- 1 comment
#463 - [FEATURE] Access to logits and final hidden layer
Issue -
State: open - Opened by lshamis 3 months ago
- 1 comment
#462 - How is the prompt segmentation specifically implemented for Dynamic SplitFuse? Is there any code implement or code snippet ?
Issue -
State: open - Opened by wenyangchou 3 months ago
#461 - Update create a PR workflow to latest version withh node js 20 fixes
Pull Request -
State: closed - Opened by loadams 3 months ago
#460 - How do I launch the api on a graphics card other than cuda: 0
Issue -
State: open - Opened by Stark-zheng 3 months ago
- 1 comment
#459 - Is openai compatible server still working?
Issue -
State: open - Opened by RobinQu 3 months ago
- 1 comment
#458 - how can I use deepspeed to split the model to submit GPU?
Issue -
State: open - Opened by WanBenLe 3 months ago
#457 - [FEATURE REQUEST] Add Support for Qwen1.5-MoE Architecture in DeepSpeed-MII
Issue -
State: open - Opened by freQuensy23-coder 3 months ago
- 1 comment
#452 - inference_core_ops.so: undefined symbol: _Z19cuda_wf6af16_linearRN2at6TensorES1_S1_S1_S1_S1_iiii
Issue -
State: open - Opened by Andronixs 3 months ago
- 6 comments
#450 - How can i use this library with langchain or llama_index?
Issue -
State: open - Opened by risedangel 3 months ago
- 2 comments
#449 - Block when Call client inference in multiprocessing.Process
Issue -
State: open - Opened by zhaotyer 3 months ago
- 3 comments
#445 - Add Kubernetes health check route to REST server
Pull Request -
State: open - Opened by richiejp 3 months ago
#444 - server crashed for some reason, unable to proceed
Issue -
State: open - Opened by Archmilio 4 months ago
- 1 comment
#443 - [BUG] Issue serving Mixtral 8x7B on H100
Issue -
State: open - Opened by Rogerwyf 4 months ago
- 9 comments
#442 - qwen1.5 model Support?
Issue -
State: open - Opened by musexiaoluo 4 months ago
- 3 comments
#441 - On M3 Pro Macbook having issues with installation
Issue -
State: closed - Opened by HariKunapareddy 4 months ago
- 2 comments
#440 - [NEED HELP] Quantization inference
Issue -
State: open - Opened by freQuensy23-coder 4 months ago
- 2 comments
#439 - Quantization inference
Issue -
State: open - Opened by freQuensy23-coder 4 months ago
- 1 comment
#438 - What is the exact meaning of forward tokens?
Issue -
State: open - Opened by frankxyy 4 months ago
#437 - Workarounds for pre-Ampere devices
Issue -
State: open - Opened by jinhachung 4 months ago
- 1 comment
#436 - Kernel execution error with long context length
Issue -
State: open - Opened by qiangxu1996 4 months ago
#435 - Can DeepSpeed-MII inference on multi gpus with only 1 replica?
Issue -
State: open - Opened by gujingit 4 months ago
- 2 comments
#434 - Update version.txt after 0.2.3 release
Pull Request -
State: closed - Opened by mrwyattii 4 months ago
#433 - Add quantization config option
Pull Request -
State: closed - Opened by mrwyattii 4 months ago
#432 - ValueError: Unsupported model type roberta
Issue -
State: open - Opened by pradeepdev-1995 4 months ago
- 2 comments
#431 - MII Example shows that mii is "Slower" than Baseline!
Issue -
State: open - Opened by Weigaa 4 months ago
#430 - How to use DeepSpeed-MII to deploy a LLM model from DeepSpeed/Megatron-DeepSpeed trained checkpoints?
Issue -
State: open - Opened by Jye-525 4 months ago
- 2 comments
#429 - Update model support
Pull Request -
State: open - Opened by mrwyattii 4 months ago
#427 - Requests.exceptions.ConnectionError:
Issue -
State: open - Opened by Weigaa 4 months ago
- 2 comments
#426 - Speeding up loading in inference checkpoints
Issue -
State: open - Opened by amritap-ef 4 months ago
- 2 comments
#425 - Add support for Gemma models
Issue -
State: open - Opened by lullabies777 4 months ago
- 1 comment
#424 - When I start server, after loading model, I got an error of 'grpc.aio._call.AioRpcError'
Issue -
State: closed - Opened by zzz0906 4 months ago
- 5 comments
#423 - Pydantic v2 migration
Pull Request -
State: open - Opened by mrwyattii 4 months ago
#422 - why all-reduce takes lots of time for mixtral which is quite larger than that of vllm and tensorrt-llm
Issue -
State: open - Opened by Eutenacity 4 months ago
#421 - Remove references to --extra-index-url in MII repo
Pull Request -
State: closed - Opened by loadams 4 months ago
#420 - How to set trust_remote_code=True in pipeline
Issue -
State: open - Opened by gujingit 4 months ago
- 2 comments
#419 - Fp6 eta
Issue -
State: open - Opened by nivibilla 4 months ago
- 2 comments
#418 - Use of dtype in the mii fastgen
Issue -
State: open - Opened by gangooteli 4 months ago
- 1 comment
#417 - How does GPT2/Bert models utilize continuous batching feature in MII?
Issue -
State: open - Opened by Jye-525 4 months ago
- 1 comment
#416 - Is the DeepSpeed-MII will support habana (HPU) hardware?
Issue -
State: open - Opened by muhammad-asn 4 months ago
- 2 comments
#415 - Add `accelerate` to requirements to improve MII-legacy model load times
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#414 - Add test for loading from local dir
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#413 - Update version.txt after 0.2.2 release
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#412 - add stable diffusion CI workflow
Pull Request -
State: open - Opened by mrwyattii 5 months ago
#411 - Disable model check in UT
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#410 - Add support for inpainting task in DS-MII
Pull Request -
State: closed - Opened by gauravrajguru 5 months ago
#409 - fix: Fixed the issue where the mii.pipeline.pipe(stop) was ineffective
Pull Request -
State: closed - Opened by kitstar 5 months ago
- 2 comments
#408 - Fix for missing EOS token
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#407 - text2img task to support negative prompts
Pull Request -
State: closed - Opened by gauravrajguru 5 months ago
#406 - How to generate multiple responses in one time?
Issue -
State: open - Opened by yangzhch6 5 months ago
- 1 comment
#405 - TypeError: expected Tensor as element 0 in argument 0, but got bool
Issue -
State: closed - Opened by SiriusWy 5 months ago
- 1 comment
#404 - Update version.txt after 0.2.1 release
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#403 - Improve recovery from KV cache starvation
Pull Request -
State: closed - Opened by tohtana 5 months ago
- 1 comment
#402 - The inference result is inconsistent with hf
Issue -
State: open - Opened by bao-xiaoyi 5 months ago
- 1 comment
#401 - Fix generate output order
Pull Request -
State: closed - Opened by mrwyattii 5 months ago
#400 - RuntimeError: server crashed for some reason, unable to proceed
Issue -
State: closed - Opened by bao-xiaoyi 5 months ago
- 2 comments
#399 - result is empty
Issue -
State: closed - Opened by bao-xiaoyi 5 months ago
#398 - ModuleNotFoundError: No module named 'mii'
Issue -
State: closed - Opened by bao-xiaoyi 5 months ago
- 2 comments
#397 - Readable token streaming support
Pull Request -
State: closed - Opened by greshilov 5 months ago
- 2 comments
#396 - Support for repetition penalty during inference with sampling
Issue -
State: open - Opened by nischith-sarvam 5 months ago
- 1 comment
#395 - Benchmark:Performance is lower than vllm
Issue -
State: open - Opened by zhaotyer 5 months ago
- 1 comment
#394 - Fix recovery from deadlock
Pull Request -
State: closed - Opened by tohtana 5 months ago
#393 - Unable to run inference on free tier Colab.
Issue -
State: closed - Opened by sudhir2016 5 months ago
- 4 comments
#392 - Update CI workflows
Pull Request -
State: closed - Opened by loadams 5 months ago
#391 - Update landing page
Pull Request -
State: closed - Opened by mrwyattii 5 months ago