b4rtaz/distributed-llama issues and pull requests

#133 - Android devices Support

Issue - State: open - Opened by qtyandhasee 28 days ago - 5 comments

#132 - Mixtral-8x7B don't work

Issue - State: open - Opened by MichaelFomenko 28 days ago - 1 comment

#131 - Model is not supported: llama3_2_3b_instruct_q40

Issue - State: open - Opened by Znbne 30 days ago - 6 comments

#128 - Feature Request - Add Sliding Window Memory Scheduling

Issue - State: open - Opened by githuba9f5404 about 1 month ago

#127 - Segmentation fault

Issue - State: open - Opened by YueZhan721 about 1 month ago - 6 comments

#126 - Tokenizer reported as incompatible tokenizer

Issue - State: open - Opened by elektroinformaciobiz about 1 month ago

#121 - Output all are "!"

Issue - State: open - Opened by HysenX-LI 2 months ago - 5 comments

#120 - Phi-3.5 support and "KeyError: 'low_freq_factor'"

Issue - State: open - Opened by unclemusclez 3 months ago

#119 - dllama-api & chat ui

Issue - State: open - Opened by twuerfl 3 months ago

#118 - feat: reduction of writeMany/readMany calls.

Pull Request - State: closed - Opened by b4rtaz 3 months ago

#117 - update readme.md.

Pull Request - State: closed - Opened by b4rtaz 3 months ago

#116 - Add the function of non-uniform proportional distribution based on distributed-llama

Pull Request - State: closed - Opened by fromthefox 3 months ago - 2 comments

#115 - Support for Gemma 2?

Issue - State: open - Opened by sdmorrey 3 months ago - 2 comments

#112 - feat: add llama3_1_405b_instruct_q40 to launch.py.

Pull Request - State: closed - Opened by b4rtaz 3 months ago

#111 - feat: improved performance of quantization to q40.

Pull Request - State: closed - Opened by b4rtaz 4 months ago

#109 - feat: --max-seq-len argument.

Pull Request - State: closed - Opened by b4rtaz 4 months ago

#108 - llama 3.1 -> terminate called after throwing an instance of 'WriteSocketException'

Issue - State: closed - Opened by oeschlberger 4 months ago - 16 comments

#107 - [Feature Suggest] Support for AVX instruction set

Issue - State: open - Opened by mrlicmi 4 months ago - 1 comment

#106 - feat: support llama 3.1.

Pull Request - State: closed - Opened by b4rtaz 4 months ago

#105 - Segmentation Fault

Issue - State: open - Opened by dot-ammar 4 months ago - 11 comments

#103 - Cannot download llama2 tokenizer from huggingface

Issue - State: closed - Opened by IterableTrucks 4 months ago - 2 comments

#102 - Some error I can't solve

Issue - State: closed - Opened by fromthefox 4 months ago - 4 comments

#101 - [Feature request] Kubernetes setup

Issue - State: open - Opened by kami4ka 4 months ago

#100 - feat: chat-template argument.

Pull Request - State: closed - Opened by b4rtaz 4 months ago

#99 - Unexpected Ċ, Ġ and D characters

Issue - State: closed - Opened by EntusiastaIApy 4 months ago - 2 comments

#98 - dllama: src/commands.cpp:102: MultiHeadAttSlice::MultiHeadAttSlice(unsigned int, unsigned int, unsigned int, slice_index_t): Assertion `nHeads % nSlices == 0' failed.

Issue - State: open - Opened by EntusiastaIApy 4 months ago - 3 comments

#97 - what(): The tokenizer does not include chat template

Issue - State: closed - Opened by EntusiastaIApy 4 months ago - 2 comments

#96 - [New Feature] Add new route for dllama api for embeding models

Issue - State: open - Opened by testing0mon21 4 months ago - 5 comments

#95 - refactor.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#94 - Support for GGUF files?

Issue - State: closed - Opened by ravor-org 5 months ago - 1 comment

#93 - Hugging Face models without tokenizer.model file

Issue - State: closed - Opened by EntusiastaIApy 5 months ago - 2 comments

#92 - Exception: max_seq_len is required, please update params.json with convert-llama.py on Meta-Llama-3-8B-Instruct

Issue - State: closed - Opened by unclemusclez 5 months ago - 1 comment

#91 - feat: vulkan.

Pull Request - State: closed - Opened by b4rtaz 5 months ago - 2 comments

#90 - feat: accelerator structure.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#89 - What about mobile phones?

Issue - State: open - Opened by dcale 5 months ago - 6 comments

#88 - fix: windows wsa startup.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#87 - what(): Cannot create socket

Issue - State: open - Opened by Slaghton 5 months ago - 2 comments

#86 - dllama-api invokes "what(): Invalid tokenizer file "

Issue - State: closed - Opened by unclemusclez 5 months ago - 2 comments

#85 - feat: update readme, add model.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#84 - feat: optional weights float type argument.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#83 - feat: tokenizer v1.

Pull Request - State: closed - Opened by b4rtaz 5 months ago

#82 - dllama-api hosted on 127.0.0.1

Issue - State: open - Opened by unclemusclez 6 months ago - 2 comments

#81 - float-type f32 will not start

Issue - State: open - Opened by unclemusclez 6 months ago - 2 comments

#80 - master and worker started but with problems

Issue - State: open - Opened by fabgat 6 months ago - 8 comments

#79 - support multi nvidia jetson agx orin?

Issue - State: open - Opened by WangFengtu1996 6 months ago - 3 comments

#78 - convert into .bin

Issue - State: closed - Opened by fabgat 6 months ago - 2 comments

#77 - Request: Community Discord?

Issue - State: closed - Opened by unclemusclez 6 months ago - 1 comment

#76 - feat: add to tokenizer chat configuration.

Pull Request - State: closed - Opened by b4rtaz 6 months ago - 5 comments

#75 - feat: naive cache.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#74 - fix: windows fseek.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#73 - Add additional chat templates to dllama-api

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 8 comments

#72 - chore: refactor http request a bit.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#71 - [Feature Suggest] Config File alternative to Command Line Arguments

Issue - State: closed - Opened by DifferentialityDevelopment 6 months ago - 2 comments

#70 - Support nSlices > nKvHeads

Issue - State: open - Opened by b4rtaz 6 months ago

#69 - [Feature Suggest] From All-Reduce to Ring-All-Reduce

Issue - State: closed - Opened by zhengpeirong 6 months ago - 3 comments

#68 - Support for another models (ollama models)

Issue - State: open - Opened by testing0mon21 6 months ago - 3 comments

#67 - [Setup] Multiple Apple Silicon Macs: Questions

Issue - State: open - Opened by s04 6 months ago - 1 comment

#66 - chore: dllama-api tiny clean up.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#65 - fix: chunked stream, close stream without econnreset.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#64 - feat: speed up synchronization of mlp.

Pull Request - State: closed - Opened by b4rtaz 6 months ago - 1 comment

#63 - feat: windows support

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 20 comments

#62 - feat: convert-hf.py

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#61 - fix: use non-blocking sockets.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#59 - (Crashing on Low Memory SBC) main invoked oom-killer: gfp_mask=0x1100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0

Issue - State: closed - Opened by unclemusclez 6 months ago - 51 comments

#58 - network utilization

Issue - State: open - Opened by zhengpeirong 6 months ago - 3 comments

#57 - feat: use avx2 to speedup dotProduct

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#56 - feat: use avx2 to speedup matmulF32

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#55 - How To Add Suppoerted Model

Issue - State: open - Opened by hyperbolic-c 6 months ago - 2 comments

#54 - Use AVX2 to speedup matmulQ40

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 3 comments

#53 - Use AVX2 to speedup matmulQ40

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 2 comments

#52 - Add safe tensor support to convert-llama.py

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 10 comments

#51 - fix: convert-llama.py supports different max_seq_len.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#50 - Vulkan Acceleration

Issue - State: open - Opened by DifferentialityDevelopment 6 months ago - 35 comments

#49 - chore: update macbeth.sh

Pull Request - State: closed - Opened by eltociear 6 months ago - 2 comments

#48 - terminate called after throwing an instance of 'ReadSocketException'

Issue - State: open - Opened by unclemusclez 6 months ago - 35 comments

#47 - API Server

Pull Request - State: closed - Opened by DifferentialityDevelopment 6 months ago - 3 comments

#46 - feat: splitting multihead attention into all nodes.

Pull Request - State: closed - Opened by b4rtaz 6 months ago - 5 comments

#45 - JSONDecodeError("Expecting value", s, err.value) from None

Issue - State: open - Opened by unclemusclez 6 months ago - 10 comments

#44 - feat: avg tokens / second.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#43 - fix: support max kv cache length.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#42 - feat: support for any number of threads.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#40 - Unknown header key's while converting llama 3 70b to distributed format

Issue - State: open - Opened by DifferentialityDevelopment 6 months ago - 1 comment

#39 - Fleshing out API mode

Issue - State: closed - Opened by DifferentialityDevelopment 6 months ago - 13 comments

#38 - rope slice.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#37 - sync pos.

Pull Request - State: closed - Opened by b4rtaz 6 months ago

#36 - revert qkv.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#35 - Will this awesome proj consider supporting GPU acceleration？

Issue - State: open - Opened by galenyu 7 months ago - 4 comments

#32 - sync qkv.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#31 - funcs-test.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#30 - To support Hugging Face model

Issue - State: closed - Opened by hyperbolic-c 7 months ago - 10 comments

#29 - [Feature Suggest] Tensor Parallellism for Accelerating LLM

Issue - State: open - Opened by zhengpeirong 7 months ago - 22 comments

#28 - llamafile sgemm.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#26 - Assertion `d % nSlices == 0' failed.

Issue - State: closed - Opened by joelewing 7 months ago - 2 comments

#25 - Compiling error related to include of <ctime>

Issue - State: closed - Opened by joelewing 7 months ago - 1 comment

#24 - arch builder.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#22 - mixtral 8x22B support.

Pull Request - State: closed - Opened by b4rtaz 7 months ago

#21 - Need help in set up all the devices

Issue - State: open - Opened by MarcuXu 7 months ago

GitHub / b4rtaz/distributed-llama issues and pull requests