sgl-project/sglang issues and pull requests

#1945 - [Bug] tp-size=2，model launch error

Issue - State: open - Opened by linqingxu 3 days ago - 1 comment

#1944 - [Bug] http_request Function Causing 403 Error

Issue - State: open - Opened by tanushmahalka 3 days ago

#1943 - [Bug] Issue with reward model API

Issue - State: open - Opened by dmakhervaks 3 days ago

#1942 - [Docs] fix 404 - Contributor Guide

Pull Request - State: closed - Opened by HaiShaw 3 days ago

#1941 - [Performance, Triton Kernel Args] extend_attention, optimize kern args to _fwd_kernel

Pull Request - State: open - Opened by HaiShaw 3 days ago

#1940 - fix black in pre-commit

Pull Request - State: open - Opened by zhaochenyang20 3 days ago

#1939 - [ENV, ROCm] update environment settings

Pull Request - State: open - Opened by HaiShaw 3 days ago

#1938 - ci: enable `black-jupyter` in pre-commit CI

Pull Request - State: closed - Opened by XuehaiPan 4 days ago - 1 comment

#1937 - [Feature] How to serve GGUF model?

Issue - State: open - Opened by hahmad2008 4 days ago

#1936 - [Feature] Add LoRA Support for Chat Completion in SGLang

Issue - State: open - Opened by mssongit 4 days ago

#1934 - [WIP] Byhsu/approx tree

Pull Request - State: open - Opened by ByronHsu 4 days ago

#1933 - Monitoring documentation

Pull Request - State: open - Opened by binarycrayon 4 days ago - 2 comments

#1932 - [Feature] Save cache from requests and load

Issue - State: open - Opened by SinanAkkoyun 4 days ago

#1931 - [Bug] Seeing random output with nvidia/Llama-3.1-Nemotron-70B-Reward

Issue - State: open - Opened by pgimenes 4 days ago

#1930 - [Bug] Incompatible with outlines>=0.1.0

Issue - State: open - Opened by dzimmerman-nci 4 days ago

#1929 - Instructions on Profiling SGLang Infer System with AMD GPUs

Pull Request - State: open - Opened by leishaoSC 4 days ago - 2 comments

#1928 - fix url in ipv6-only when warm-up

Pull Request - State: closed - Opened by cauyxy 5 days ago

#1927 - Get connection error when use sglang python module

Issue - State: open - Opened by kanebay 5 days ago

#1926 - minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces

Pull Request - State: closed - Opened by XuehaiPan 5 days ago - 5 comments

#1925 - [Bug] Torch 2.5 issue with Tensor Parallel Size > 1

Issue - State: open - Opened by CortexEdgeUser 5 days ago

#1924 - [Doc] improve relative links and structure

Pull Request - State: closed - Opened by merrymercy 5 days ago

#1923 - [Bug] Launching a server with `--enable-torch-compile` produce torch dynamo error

Issue - State: open - Opened by msublee 5 days ago

#1922 - [rust] refactor server and router

Pull Request - State: closed - Opened by ByronHsu 5 days ago

#1921 - [Bug] AssertionError: compatibility of lora and cuda graph and radix attention is in progress

Issue - State: open - Opened by LIUKAI0815 5 days ago

#1920 - Change judge to classify & Modify make file

Pull Request - State: closed - Opened by zhaochenyang20 5 days ago

#1919 - Add get latest commit

Pull Request - State: open - Opened by Ying1123 5 days ago

#1918 - [Feature] add model LlavaOnevisionForConditionalGeneration

Issue - State: open - Opened by zhangucan 5 days ago

#1917 - [Release, ROCm] release ROCm docker build for AMD MI GPUs

Pull Request - State: open - Opened by HaiShaw 5 days ago - 14 comments

#1916 - Update CODEOWNERS

Pull Request - State: closed - Opened by ByronHsu 5 days ago

#1915 - [Docs, ROCm] update install to cover ROCm with MI GPUs

Pull Request - State: closed - Opened by HaiShaw 6 days ago

#1913 - Scheduler methods

Pull Request - State: open - Opened by josephydu 6 days ago - 1 comment

#1912 - [Feature]Support Qwen2_5...etc tools calling by OpenAI API

Issue - State: open - Opened by CedricHwong 6 days ago

#1911 - [Feature] sglang.rocm support flashinfer?

Issue - State: closed - Opened by linqingxu 6 days ago - 2 comments

#1910 - Add Reward API Docs etc

Pull Request - State: closed - Opened by zhaochenyang20 6 days ago

#1909 - Fix regex docs

Pull Request - State: closed - Opened by merrymercy 6 days ago

#1908 - Release v0.3.5

Pull Request - State: closed - Opened by merrymercy 6 days ago

#1907 - Let reward model take text inputs instead of message lists

Pull Request - State: closed - Opened by merrymercy 6 days ago

#1906 - feat: support truss endpoint for benchmark serving

Pull Request - State: closed - Opened by zhyncs 6 days ago

#1905 - Unify the model type checking

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1904 - Simplify tokenizer manager

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1903 - Allow passing dtype and max_new_tokens to HF reference script

Pull Request - State: closed - Opened by janimo 7 days ago

#1902 - Escape backwards slash

Pull Request - State: closed - Opened by inakineitor 7 days ago

#1901 - Fix issue with `stop_token_ids` not being iterable.

Pull Request - State: open - Opened by inakineitor 7 days ago

#1900 - Expose max_total_num_tokens for Token Limit Calculation in Request Handling

Issue - State: open - Opened by hahmad2008 7 days ago - 1 comment

#1899 - Simplify tokenizer manager

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1898 - Difference between TokenizerManager and Runtime class

Issue - State: open - Opened by NrKhader 7 days ago - 1 comment

#1897 - QWen VL Follow-up Fixes

Issue - State: open - Opened by merrymercy 7 days ago

#1896 - Do not use longest prefix matching when #queue-req is large

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1895 - turn off log for the offline engine

Pull Request - State: closed - Opened by zhaochenyang20 7 days ago

#1894 - Add engine api

Pull Request - State: closed - Opened by zhaochenyang20 7 days ago

#1893 - [router] Impl radix tree and set up CI

Pull Request - State: closed - Opened by ByronHsu 7 days ago - 1 comment

#1892 - Fix ci and link error

Pull Request - State: closed - Opened by zhaochenyang20 7 days ago

#1891 - Add Rust Router Python Binding

Pull Request - State: closed - Opened by austin362667 7 days ago

#1890 - Fix docs

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1889 - Fix docs

Pull Request - State: closed - Opened by merrymercy 7 days ago

#1888 - Fix docs ci

Pull Request - State: closed - Opened by zhaochenyang20 8 days ago

#1887 - Fix docs

Pull Request - State: closed - Opened by zhaochenyang20 8 days ago