Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / BlackSamorez/tensor_parallel issues and pull requests

#136 - Customized generate func support?

Issue - State: open - Opened by MonolithFoundation 5 months ago

#134 - Add mixtral support

Pull Request - State: open - Opened by ReinForce-II 9 months ago

#128 - No output when using tensor_parallel

Issue - State: open - Opened by yyya9 11 months ago - 1 comment

#126 - Out of GPU memory for two A10 GPUs

Issue - State: closed - Opened by JunyiYe 12 months ago - 1 comment

#125 - AttributeError: object has no attribute 'devices'

Issue - State: open - Opened by QiueY514 12 months ago

#123 - fix recursion error when setting tp_wrapped_module #122

Pull Request - State: open - Opened by Ar-Kareem 12 months ago - 4 comments

#122 - Max Recursion Error when using with lora

Issue - State: open - Opened by Ar-Kareem almost 1 year ago - 2 comments

#121 - RuntimeError: NCCL Error 3: internal error

Issue - State: open - Opened by smallmocha about 1 year ago - 1 comment

#120 - Segmentation fault (core dumped)

Issue - State: open - Opened by jameswu2014 about 1 year ago

#119 - Support of 8-bit and 4-bit quantization

Issue - State: closed - Opened by ludwigflo about 1 year ago - 1 comment

#117 - 2x slowdown using TP

Issue - State: open - Opened by jph00 about 1 year ago

#115 - distributed TP model forward output's requires_grad is False

Issue - State: open - Opened by lxuechen about 1 year ago - 5 comments

#114 - tensor_parallel method distributed=True

Issue - State: open - Opened by Johnno1011 about 1 year ago - 2 comments

#113 - Forwarding _prepare_model_inputs

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago

#112 - model.generate() with inputs_embeds

Issue - State: closed - Opened by ZhaoxuanWu about 1 year ago - 3 comments

#111 - Fix false positive in tests for finding predefined TP config

Pull Request - State: closed - Opened by tonywang16 about 1 year ago - 2 comments

#110 - find_predefined_tensor_parallel_config try-except fix

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago

#109 - Testing interfaces (soon to be refactored)

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago

#108 - Fix get_llama_config adding model attribute error.

Pull Request - State: closed - Opened by tonywang16 about 1 year ago - 3 comments

#107 - Error loading LLAMA model config

Issue - State: closed - Opened by tonywang16 about 1 year ago

#106 - [WIP] ZeRO-3 refactoring (sharding)

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago - 1 comment
Labels: enhancement

#104 - When I try to do the tensor_parallel on NLLB from meta, there is an error:

Issue - State: closed - Opened by 342215448 about 1 year ago - 1 comment
Labels: duplicate

#103 - Gpt2 fix

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago - 1 comment
Labels: bug

#102 - Version bump

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago

#101 - LLaMA-2

Pull Request - State: closed - Opened by BlackSamorez about 1 year ago - 1 comment

#100 - explicitly choose whether or not to use torch.distribute

Pull Request - State: closed - Opened by tomoki0924 about 1 year ago - 2 comments
Labels: enhancement

#99 - GPT-2 broken starting in v1.2.5

Issue - State: closed - Opened by eric-mitchell about 1 year ago - 1 comment

#98 - Issues if GPU > 2

Issue - State: closed - Opened by Tom-Ryder about 1 year ago - 6 comments

#96 - Example Question (got error) : Try new 40B LLMs demo in Kaggle

Issue - State: closed - Opened by YooSungHyun about 1 year ago - 2 comments

#95 - why raised cuda error?

Issue - State: closed - Opened by YooSungHyun about 1 year ago - 18 comments

#94 - Possibility to run on different GPUs

Issue - State: closed - Opened by Ch4mpa9ne over 1 year ago - 2 comments

#93 - readme fixes

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#92 - Falcon lm_head split hotfix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#91 - Falcon predefined config

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#90 - Fixed dispatch of tp.Sharded models

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#89 - TypeError when multi-thread inference using tensor_parallel

Issue - State: closed - Opened by liulhdarks over 1 year ago - 1 comment

#88 - Question on custom models

Issue - State: open - Opened by vince62s over 1 year ago - 23 comments

#87 - Removing PEFT from dependencies. Replacing with runtime checks

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#84 - Does tensor_parallel support multi-node tensor parallel training?

Issue - State: open - Opened by liguodongiot over 1 year ago - 5 comments

#83 - Can I parallelize just one large layer?

Issue - State: open - Opened by chinmayjog13 over 1 year ago - 1 comment

#82 - Actually using SplitInsideChunks for gpt2

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#81 - Request to fix the content about parallelformers in README.

Issue - State: closed - Opened by hyunwoongko over 1 year ago - 1 comment

#80 - Support for PEFT LoRA and 4-bit quantization

Issue - State: closed - Opened by morecry over 1 year ago - 6 comments

#79 - Not work with 4bit quant

Issue - State: closed - Opened by laoda513 over 1 year ago - 6 comments
Labels: duplicate

#78 - tp.convert_state_dict readme example fix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#77 - Error in README.Md, hence not able to load model with limited memory.

Issue - State: closed - Opened by vishakudupa over 1 year ago - 5 comments

#76 - Torch version requirement

Issue - State: closed - Opened by treya-lin over 1 year ago - 4 comments

#75 - Great work！ and can this work with deepspeedzero?

Issue - State: open - Opened by laoda513 over 1 year ago

#74 - Huggingface Accelerate

Issue - State: closed - Opened by conceptofmind over 1 year ago - 1 comment

#73 - State dict fixes for tied weights

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#72 - What is the difference between this project and autotp of deepspeed?

Issue - State: closed - Opened by frankxyy over 1 year ago - 1 comment

#71 - cuda memory not evenly distributed between devices

Issue - State: closed - Opened by frankxyy over 1 year ago - 6 comments

#70 - Torch distributed hotfix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#69 - set distributed=True, return AttributeError: 'NoneType' object

Issue - State: closed - Opened by rocketsearch over 1 year ago - 2 comments

#68 - Peft LoRA support

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#67 - How to load lora weights？

Issue - State: closed - Opened by Vincent131499 over 1 year ago - 13 comments

#66 - Slow inference performance for large Llama models compared to naive MP

Issue - State: open - Opened by sgsdxzy over 1 year ago - 26 comments

#65 - Mention linear speedup in Readme

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#64 - Set seed for tests reproducibility

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#63 - Small readme patch

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#62 - Added int8 LLMs demo link

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#61 - CodeGen config

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#60 - Converting state dicts without model creation

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 3 comments

#59 - New version for dispatch hotfix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#58 - Shard parameters initial dispatch fix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#57 - Unpersistent buffers meta loading fix

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#56 - _reorder_cache fix for generation utils

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#55 - GPT NeoX config

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago
Labels: enhancement

#54 - Meta devices support

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment
Labels: enhancement

#53 - LLaMa models

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago
Labels: enhancement

#52 - Removing accelerate hooks before splitting the model

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#51 - Support LLaMA Models, including HuggingFace-adapted variants

Issue - State: closed - Opened by zoidbb over 1 year ago - 7 comments
Labels: bug, enhancement

#50 - Version update

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#49 - Saving utilities

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#48 - How to use trained models?

Issue - State: closed - Opened by Den4ikAI over 1 year ago - 3 comments

#47 - Adding support for more model architectures

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 2 comments

#46 - False negative test results for test_convs. Flaky test

Issue - State: closed - Opened by BlackSamorez over 1 year ago - 1 comment

#45 - Add more predefined configs

Issue - State: open - Opened by BlackSamorez over 1 year ago
Labels: enhancement, good first issue

#44 - Replace architecture with model_type

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#43 - Config refactoring

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago - 4 comments

#42 - New ideas

Issue - State: open - Opened by aizamaksutova over 1 year ago

#41 - GPU Contention

Issue - State: open - Opened by aizamaksutova over 1 year ago

#40 - Fixed PyPi link in readme

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#39 - Version update

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago

#38 - _TensorParallelWrapper attribute forwarding

Pull Request - State: closed - Opened by BlackSamorez over 1 year ago