Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / microsoft/DeepSpeed-MII issues and pull requests

#499 - Test

Pull Request - State: closed - Opened by trapp3rhat 4 days ago

#498 - [QUERY] Expert Parallelism Supported?

Issue - State: open - Opened by Shamauk 5 days ago

#496 - Compute perplexity

Issue - State: open - Opened by Sh1gechan 11 days ago

#495 - Configure server log level

Issue - State: open - Opened by sedletsky-f5 13 days ago

#493 - Add explanations of MII code into comments

Pull Request - State: open - Opened by mrwyattii 20 days ago - 1 comment

#492 - Remove Conversation from MII as it was deprecated and removed from transformers.

Pull Request - State: closed - Opened by loadams 25 days ago - 1 comment

#491 - Always Flush UIDs after Exceptions

Pull Request - State: open - Opened by weiqisun 27 days ago

#490 - Always Flush UIDs after `GeneratorReply`

Pull Request - State: closed - Opened by weiqisun 27 days ago - 1 comment

#488 - support stream

Issue - State: open - Opened by ZZhangxian 30 days ago

#487 - support Qwen1.5

Issue - State: open - Opened by ZZhangxian 30 days ago

#486 - support Qwen

Issue - State: open - Opened by ZZhangxian 30 days ago

#485 - Some fixes to make openai entrypoint work out of the box

Pull Request - State: open - Opened by svaruag about 1 month ago

#484 - Reuse KV cache of prefixes

Pull Request - State: open - Opened by tohtana about 1 month ago

#483 - Support LLava next stronger

Issue - State: open - Opened by thesby about 1 month ago

#481 - Tf32 support

Issue - State: open - Opened by Chasapas about 2 months ago

#480 - Enable streaming option in the OpenAI API server

Pull Request - State: open - Opened by adk9 about 2 months ago

#478 - Fix deprecation warning on escaped characters

Pull Request - State: closed - Opened by loadams about 2 months ago

#477 - Does deepspeed-mii support prefix_allowed_tokens_fn?

Issue - State: open - Opened by zcakzhuu about 2 months ago

#476 - Update mistral tests to fully open source version.

Pull Request - State: closed - Opened by loadams about 2 months ago

#475 - [REQUEST] LLAMA-3 support

Issue - State: open - Opened by MRYingLEE about 2 months ago

#474 - [REQUEST] Mixtral-8x22B support

Issue - State: open - Opened by y-live-koba about 2 months ago

#473 - Allow model to generate added tokens - fix generation issue in Llama3 models

Pull Request - State: open - Opened by weiqisun about 2 months ago - 7 comments

#472 - Cannot run Yi-34B-Chat => ValueError: Unsupported q_ratio: 7

Issue - State: open - Opened by joeking11829 about 2 months ago - 2 comments

#471 - BUG in run_batch_processing

Issue - State: open - Opened by zhihui96 about 2 months ago

#470 - fix max_ragged_sequence_count check in _schedule_prompts

Pull Request - State: closed - Opened by dc3671 about 2 months ago - 1 comment

#469 - ValueError: Unsupported model type phi3

Issue - State: open - Opened by abpani 2 months ago

#468 - error when using Qwen1.5-32B

Issue - State: open - Opened by puppet101 2 months ago

#467 - Performance with vllm

Issue - State: open - Opened by littletomatodonkey 2 months ago

#466 - [Problem]errno: 98 - Address already in use

Issue - State: closed - Opened by littletomatodonkey 2 months ago

#463 - [FEATURE] Access to logits and final hidden layer

Issue - State: open - Opened by lshamis 3 months ago - 1 comment

#460 - How do I launch the api on a graphics card other than cuda: 0

Issue - State: open - Opened by Stark-zheng 3 months ago - 1 comment

#459 - Is openai compatible server still working?

Issue - State: open - Opened by RobinQu 3 months ago - 1 comment

#450 - How can i use this library with langchain or llama_index?

Issue - State: open - Opened by risedangel 3 months ago - 2 comments

#449 - Block when Call client inference in multiprocessing.Process

Issue - State: open - Opened by zhaotyer 3 months ago - 3 comments

#445 - Add Kubernetes health check route to REST server

Pull Request - State: open - Opened by richiejp 3 months ago

#444 - server crashed for some reason, unable to proceed

Issue - State: open - Opened by Archmilio 4 months ago - 1 comment

#443 - [BUG] Issue serving Mixtral 8x7B on H100

Issue - State: open - Opened by Rogerwyf 4 months ago - 9 comments

#442 - qwen1.5 model Support?

Issue - State: open - Opened by musexiaoluo 4 months ago - 3 comments

#441 - On M3 Pro Macbook having issues with installation

Issue - State: closed - Opened by HariKunapareddy 4 months ago - 2 comments

#440 - [NEED HELP] Quantization inference

Issue - State: open - Opened by freQuensy23-coder 4 months ago - 2 comments

#439 - Quantization inference

Issue - State: open - Opened by freQuensy23-coder 4 months ago - 1 comment

#438 - What is the exact meaning of forward tokens?

Issue - State: open - Opened by frankxyy 4 months ago

#437 - Workarounds for pre-Ampere devices

Issue - State: open - Opened by jinhachung 4 months ago - 1 comment

#436 - Kernel execution error with long context length

Issue - State: open - Opened by qiangxu1996 4 months ago

#435 - Can DeepSpeed-MII inference on multi gpus with only 1 replica?

Issue - State: open - Opened by gujingit 4 months ago - 2 comments

#434 - Update version.txt after 0.2.3 release

Pull Request - State: closed - Opened by mrwyattii 4 months ago

#433 - Add quantization config option

Pull Request - State: closed - Opened by mrwyattii 4 months ago

#432 - ValueError: Unsupported model type roberta

Issue - State: open - Opened by pradeepdev-1995 4 months ago - 2 comments

#431 - MII Example shows that mii is "Slower" than Baseline!

Issue - State: open - Opened by Weigaa 4 months ago

#429 - Update model support

Pull Request - State: open - Opened by mrwyattii 4 months ago

#427 - Requests.exceptions.ConnectionError:

Issue - State: open - Opened by Weigaa 4 months ago - 2 comments

#426 - Speeding up loading in inference checkpoints

Issue - State: open - Opened by amritap-ef 4 months ago - 2 comments

#425 - Add support for Gemma models

Issue - State: open - Opened by lullabies777 4 months ago - 1 comment

#423 - Pydantic v2 migration

Pull Request - State: open - Opened by mrwyattii 4 months ago

#421 - Remove references to --extra-index-url in MII repo

Pull Request - State: closed - Opened by loadams 4 months ago

#420 - How to set trust_remote_code=True in pipeline

Issue - State: open - Opened by gujingit 4 months ago - 2 comments

#419 - Fp6 eta

Issue - State: open - Opened by nivibilla 4 months ago - 2 comments

#418 - Use of dtype in the mii fastgen

Issue - State: open - Opened by gangooteli 4 months ago - 1 comment

#417 - How does GPT2/Bert models utilize continuous batching feature in MII?

Issue - State: open - Opened by Jye-525 4 months ago - 1 comment

#416 - Is the DeepSpeed-MII will support habana (HPU) hardware?

Issue - State: open - Opened by muhammad-asn 4 months ago - 2 comments

#414 - Add test for loading from local dir

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#413 - Update version.txt after 0.2.2 release

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#412 - add stable diffusion CI workflow

Pull Request - State: open - Opened by mrwyattii 5 months ago

#411 - Disable model check in UT

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#410 - Add support for inpainting task in DS-MII

Pull Request - State: closed - Opened by gauravrajguru 5 months ago

#409 - fix: Fixed the issue where the mii.pipeline.pipe(stop) was ineffective

Pull Request - State: closed - Opened by kitstar 5 months ago - 2 comments

#408 - Fix for missing EOS token

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#407 - text2img task to support negative prompts

Pull Request - State: closed - Opened by gauravrajguru 5 months ago

#406 - How to generate multiple responses in one time?

Issue - State: open - Opened by yangzhch6 5 months ago - 1 comment

#405 - TypeError: expected Tensor as element 0 in argument 0, but got bool

Issue - State: closed - Opened by SiriusWy 5 months ago - 1 comment

#404 - Update version.txt after 0.2.1 release

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#403 - Improve recovery from KV cache starvation

Pull Request - State: closed - Opened by tohtana 5 months ago - 1 comment

#402 - The inference result is inconsistent with hf

Issue - State: open - Opened by bao-xiaoyi 5 months ago - 1 comment

#401 - Fix generate output order

Pull Request - State: closed - Opened by mrwyattii 5 months ago

#400 - RuntimeError: server crashed for some reason, unable to proceed

Issue - State: closed - Opened by bao-xiaoyi 5 months ago - 2 comments

#399 - result is empty

Issue - State: closed - Opened by bao-xiaoyi 5 months ago

#398 - ModuleNotFoundError: No module named 'mii'

Issue - State: closed - Opened by bao-xiaoyi 5 months ago - 2 comments

#397 - Readable token streaming support

Pull Request - State: closed - Opened by greshilov 5 months ago - 2 comments

#396 - Support for repetition penalty during inference with sampling

Issue - State: open - Opened by nischith-sarvam 5 months ago - 1 comment

#395 - Benchmark:Performance is lower than vllm

Issue - State: open - Opened by zhaotyer 5 months ago - 1 comment

#394 - Fix recovery from deadlock

Pull Request - State: closed - Opened by tohtana 5 months ago

#393 - Unable to run inference on free tier Colab.

Issue - State: closed - Opened by sudhir2016 5 months ago - 4 comments

#392 - Update CI workflows

Pull Request - State: closed - Opened by loadams 5 months ago

#391 - Update landing page

Pull Request - State: closed - Opened by mrwyattii 5 months ago