xorbitsai/inference issues and pull requests

#2584 - 超长上下文造成服务挂死

Issue - State: open - Opened by luckfu about 13 hours ago
Labels: gpu

#2583 - xoscar.errors.ServerClosed: [address=0.0.0.0:13276, pid=39] Remote server unixsocket:///20447232 closed

Issue - State: open - Opened by erliang-sf about 13 hours ago - 1 comment
Labels: gpu

#2582 - FEAT: support glm-edge-chat model

Pull Request - State: open - Opened by amumu96 about 14 hours ago
Labels: feature

#2581 - Feat: Support glm-edge-chat model

Pull Request - State: closed - Opened by amumu96 about 14 hours ago
Labels: feature

#2580 - Feat: Support glm-edge-1.5b-chat Model

Pull Request - State: closed - Opened by amumu96 about 15 hours ago
Labels: feature

#2579 - 运行嵌入模型报错Remote server 192.0.0.181:44667 closed

Issue - State: open - Opened by minglong-huang about 17 hours ago
Labels: gpu

#2578 - 模型无法选择gpu，只有cpu选项

Issue - State: open - Opened by wz96cj 1 day ago - 1 comment
Labels: gpu

#2577 - pip install "xinference[all]"报错

Issue - State: closed - Opened by liguoyu666 1 day ago - 2 comments
Labels: gpu

#2576 - FEAT: whisper support for Mac MLX

Pull Request - State: closed - Opened by qinxuye 2 days ago
Labels: feature

#2575 - Dify调用xinfer中的rerank模型正常，ragflow调用xinfer中的rerank模型就OOM了

Issue - State: open - Opened by cnrbi1 2 days ago
Labels: gpu

#2574 - xinference-client rerank has some bug

Issue - State: open - Opened by swy0915 2 days ago - 4 comments

#2573 - xinference v1.0.0及以下版本需要升级transformer 4.45或以上，才可以正常启动qwen2-VL-instruct 模型

Issue - State: open - Opened by cnrbi1 2 days ago - 1 comment
Labels: gpu

#2572 - /v1/completions prompt输入无效

Issue - State: closed - Opened by alvinlee518 4 days ago - 4 comments
Labels: gpu

#2571 - BUG: request_limits does not work with streaming interfaces

Pull Request - State: closed - Opened by ChengjieLi28 4 days ago
Labels: bug

#2570 - Xinference use chattts BUG

Issue - State: open - Opened by NieSf 5 days ago - 4 comments

#2569 - Xinference integration with Microsoft Word

Pull Request - State: open - Opened by GPTLocalhost 5 days ago - 2 comments

#2568 - 加载本地ChatGLM2-6b报错，找不到generation_config.json

Issue - State: closed - Opened by congge27 5 days ago - 1 comment
Labels: gpu

#2567 - Add download progress management or optimizing logs

Issue - State: open - Opened by redreamality 5 days ago - 1 comment
Labels: feature

#2566 - rerank模型不返回document信息

Issue - State: open - Opened by mengxianglong123 5 days ago - 1 comment

#2565 - BUG: GTE-qwen2 Embedding Dimension error

Pull Request - State: closed - Opened by cyhasuka 5 days ago
Labels: bug

#2564 - xinference v1.0.0无法正常启动qwen2-instruct

Issue - State: open - Opened by m199369309 5 days ago
Labels: gpu

#2563 - 模型启动问题

Issue - State: open - Opened by sdlssq 5 days ago - 1 comment

#2562 - FEAT: Fish speech stream

Pull Request - State: closed - Opened by codingl2k1 5 days ago
Labels: feature

#2561 - update Xinference depends error

Issue - State: open - Opened by pingyuan2016 6 days ago
Labels: gpu

#2560 - requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Issue - State: open - Opened by ChengRuiLiang 6 days ago - 1 comment

#2559 - Got an Error while using Langchain Chatchat with internlm2.5.

Issue - State: open - Opened by frankSARU 8 days ago - 2 comments
Labels: gpu

#2558 - glm-4v模型只传文字报KeyError: 'images'，同时传文字和图片不报错

Issue - State: open - Opened by JumpNew 8 days ago - 2 comments

#2557 - 启动FLUX.1-schnell 报错，

Issue - State: closed - Opened by James-Dao 10 days ago - 2 comments

#2556 - qwen2-instruct chat failed if set stop parameter

Issue - State: open - Opened by liunux4odoo 11 days ago - 1 comment

#2555 - ENH: Update fish audio

Pull Request - State: closed - Opened by codingl2k1 11 days ago - 1 comment
Labels: enhancement

#2554 - 最新版本的xinference无法正常启动qwen2-vl-instruct模型

Issue - State: open - Opened by majestichou 11 days ago - 9 comments
Labels: gpu

#2553 - 为什么调用rerank模型后速度变慢了？

Issue - State: open - Opened by fg2501 12 days ago - 2 comments
Labels: gpu, stale

#2552 - vllm+cpu 后端（无 gpu 硬件）时，tensor_parallel_size 应该默认设置成 1 而不是 cuda_count(等于 0)

Issue - State: open - Opened by Diffizle 12 days ago - 2 comments
Labels: gpu, stale

#2551 - 【企业版】910B部署自定义bge_embedding和bge_rerank模型报错

Issue - State: closed - Opened by Jayc-Z 12 days ago - 1 comment

#2550 - 在windows系统的conda中启动系统，提示错误信息。

Issue - State: open - Opened by cns-cash 12 days ago - 1 comment
Labels: gpu, stale

#2549 - 加载 qwen2-vl-instruct 报错：cannot import name 'Qwen2VLForConditionalGeneration' from 'transformers'

Issue - State: open - Opened by EthanKk 12 days ago - 1 comment

#2548 - Can we support paraformer?

Issue - State: open - Opened by wy96f 12 days ago - 1 comment
Labels: feature

#2547 - BUG: fix variant error for image model

Pull Request - State: closed - Opened by qinxuye 12 days ago
Labels: bug

#2546 - Qwen2的hidden size是3584维,所以gte-Qwen2-7B输出的维度也是3584维

Issue - State: closed - Opened by sanshanya 13 days ago - 1 comment

#2545 - black-forest-labs/FLUX.1-dev 模型启动失败：You are trying to load the model files of the `variant=fp16`, but no such modeling files are available

Issue - State: closed - Opened by majestichou 13 days ago - 1 comment
Labels: gpu

#2544 - H20显卡推理 glm9b-chat失败，版本0.16.3

Issue - State: open - Opened by yangyu6 14 days ago - 3 comments
Labels: gpu

#2543 - FEAT: Add qwen2.5-coder 0.5B 1.5B 3B 14B 32B

Pull Request - State: closed - Opened by frostyplanet 14 days ago - 2 comments
Labels: feature

#2542 - ENH: Support fish speech reference audio

Pull Request - State: closed - Opened by codingl2k1 14 days ago - 6 comments
Labels: enhancement

#2541 - 寻求一个可以在一个GPU上部署多个小模型的方案

Issue - State: open - Opened by RichardFans 14 days ago - 2 comments
Labels: feature, gpu

#2540 - FEAT: support sparse vector for bge-m3

Pull Request - State: closed - Opened by pengjunfeng11 15 days ago - 3 comments
Labels: feature

#2539 - 命令行启动总是提示 You must specify extra kwargs with `--` prefix.

Issue - State: closed - Opened by luckfu 15 days ago - 5 comments
Labels: gpu

#2538 - internvl2 awq 的launch 错误

Issue - State: closed - Opened by frostyplanet 15 days ago - 2 comments
Labels: gpu, stale

#2537 - 0.16.3 版本docker镜像无法选择 sglang 作为推理引擎

Issue - State: open - Opened by machgity 15 days ago - 3 comments
Labels: gpu

#2536 - 镜像拉取xinference后，glm-4v-transformer-9b出错

Issue - State: closed - Opened by Erincrying 15 days ago - 3 comments
Labels: gpu

#2535 - libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory

Issue - State: closed - Opened by Yanhuanjin 15 days ago - 1 comment

#2534 - FEAT: support kvcache in multi-round chat for MLX

Pull Request - State: closed - Opened by qinxuye 15 days ago
Labels: feature

#2533 - DOC: Add paper citation

Pull Request - State: closed - Opened by luweizheng 15 days ago
Labels: documentation

#2532 - Fish-Speech启用reference-audio

Issue - State: closed - Opened by bjwswang 18 days ago - 2 comments
Labels: feature, stale

#2531 - MiniCPM-V-2.6 cannot be run on engine vllm

Issue - State: closed - Opened by leeyis 19 days ago - 2 comments
Labels: gpu, stale

#2530 - BUG: transformers logs missing

Pull Request - State: closed - Opened by ChengjieLi28 19 days ago
Labels: bug

#2529 - vllm后端启动不支持异步

Issue - State: open - Opened by prettyprettyboy 19 days ago - 7 comments
Labels: stale

#2528 - FEAT: Basic cancel support for image model

Pull Request - State: closed - Opened by codingl2k1 19 days ago
Labels: feature

#2527 - xf不支持生成稀疏向量

Issue - State: closed - Opened by pengjunfeng11 19 days ago - 2 comments
Labels: feature, stale

#2526 - FishSpeech报错The expanded size of the tensor (968) must match the existing size (1023) at non-singleton dimension 1

Issue - State: closed - Opened by zg9uagfv 20 days ago - 2 comments
Labels: gpu

#2525 - [vllm] got an unexpected keyword argument '--gpu-memory-utilization'

Issue - State: closed - Opened by jizusun 20 days ago - 1 comment
Labels: gpu

#2524 - 编译cogvlm2报错ninja: build stopped: subcommand failed.

Issue - State: closed - Opened by jarbox 20 days ago - 2 comments
Labels: gpu, stale

#2523 - 部署glm-4v-9b模型后，模型可以加载但是无法进行对话，报[address=0.0.0.0:33271, pid=892267] ChatGLM4Tokenizer._pad() got an unexpected keyword argument 'padding_side'

Issue - State: open - Opened by GengyuXu 20 days ago - 5 comments
Labels: stale

#2522 - 下载glm4-chat时，选择8-bit，下载报错Server error: 500 - [address=127.0.0.1:4389, pid=15108] 'transfomg.word_embeddings.weight'

Issue - State: closed - Opened by MrTLin 20 days ago - 2 comments
Labels: gpu, stale

#2521 - xinference部署的qwen模型怎样在后台看到request请求的日志及问的内容

Issue - State: closed - Opened by wangyongpenga 21 days ago - 3 comments
Labels: feature, stale

#2520 - BUG: Compat with ChatTTS 0.2.1

Pull Request - State: closed - Opened by codingl2k1 21 days ago - 2 comments
Labels: bug

#2519 - 希望转录transcription可以支持流式输出文本

Issue - State: closed - Opened by Jimmy-L99 21 days ago - 3 comments
Labels: feature, stale

#2518 - An error was reported importing the image2tex model

Issue - State: closed - Opened by ZxnSnowy 21 days ago - 3 comments
Labels: gpu

#2517 - 无法安装FishSpeech

Issue - State: closed - Opened by zg9uagfv 21 days ago - 2 comments
Labels: gpu

#2516 - REF: Remove replica total count in internal `replica_model_uid`

Pull Request - State: closed - Opened by ChengjieLi28 21 days ago
Labels: refactor

#2515 - xinference版本升级到0.16.1之后出现并发性能减弱的情况

Issue - State: closed - Opened by magthub 21 days ago - 2 comments
Labels: gpu, stale

#2514 - docker部署vllm引擎，4张80G卡运行qwen2.5-instruct 72b卡住，nvidia-smi 100%

Issue - State: closed - Opened by kevinchi8781 21 days ago - 2 comments
Labels: gpu

#2513 - xinference[vllm]无法单机多卡启动一个模型

Issue - State: open - Opened by Weishaoya 21 days ago - 5 comments

#2512 - chatglm3-6b、chatglm3-6b-32k 通过docker启动时报如下错误

Issue - State: open - Opened by xinhen 21 days ago - 3 comments
Labels: stale

#2511 - qwen2.5 用vllm引擎报错failed to infer device type

Issue - State: closed - Opened by kevinchi8781 22 days ago - 5 comments
Labels: stale

#2510 - 新版本使用 fastGPT 文本内容提取工具提取日期报错 Cannot delete property '0' of [object String]

Issue - State: closed - Opened by 305607610 22 days ago - 2 comments
Labels: gpu, stale

#2509 - ENH: add normalize to rerank model

Pull Request - State: closed - Opened by hustyichi 22 days ago - 9 comments
Labels: enhancement

#2508 - Failed to launch model, detail: [address=0.0.0.0:58184, pid=1222936] No available slot found for the model

Issue - State: closed - Opened by chaoStart 24 days ago - 5 comments
Labels: gpu, stale

#2507 - cl: 命令行 error D8021 :无效的数值参数“/Wno-register”

Issue - State: closed - Opened by tsxuzhiqiang 25 days ago - 2 comments
Labels: stale

#2506 - An error occurred during streaming

Issue - State: closed - Opened by Andy1018 25 days ago - 8 comments
Labels: gpu, stale

#2505 - Qwen2.5-instruct-AWQ Quantization Int4 cannot launch from latest docker containers with

Issue - State: closed - Opened by zhyuchao123 25 days ago - 2 comments
Labels: gpu, stale

#2504 - FEAT: add download from openmind_hub

Pull Request - State: closed - Opened by cookieyyds 25 days ago
Labels: feature

#2503 - BLD: Remove Python 3.8 & Support Python 3.12

Pull Request - State: closed - Opened by ChengjieLi28 26 days ago
Labels: build

#2502 - Error in calling model.chat() for InternVL2 model: async_chat() got an unexpected keyword argument 'request_id'

Issue - State: closed - Opened by True-deng 26 days ago - 2 comments
Labels: gpu, stale

#2501 - Empty reply from server

Issue - State: closed - Opened by ZxnSnowy 26 days ago
Labels: gpu

#2500 - Running Xinference with Docker has ImportError

Issue - State: closed - Opened by DavidSche 26 days ago - 1 comment
Labels: gpu

#2499 - Jetson 可不可以部署xinference？我在信息上发现xinference没有获取到GPU的信息，在对话的时候明显发现很慢，应该是只调用了cpu的算力

Issue - State: closed - Opened by GengyuXu 26 days ago - 3 comments
Labels: gpu, stale

#2498 - 使用glm-4v模型的时候可以加载，但是无法对话，报错 fail to generate chat competition [address=0.0.0.0:46723, pid=1157264]

Issue - State: closed - Opened by GengyuXu 26 days ago - 2 comments
Labels: stale

#2497 - 使用modelscope启动xinference服务时，无法通过host:port访问ui界面

Issue - State: closed - Opened by RollinsSeth 27 days ago - 5 comments
Labels: gpu, stale

#2496 - request limit限制可以设置成一个队列吗

Issue - State: closed - Opened by jasinliu 27 days ago - 4 comments
Labels: feature, stale

#2495 - BUG: fix bge-reranker-v2-minicpm-layerwise rerank issue

Pull Request - State: closed - Opened by hustyichi 28 days ago - 1 comment
Labels: bug

#2494 - FEAT: add download from openmind_hub

Pull Request - State: closed - Opened by cookieyyds 28 days ago
Labels: feature

#2493 - xinference 的 Qwen2-VL-Instruct 测试页效果比 qwen 官方还好，又找不到原因

Issue - State: closed - Opened by Valdanitooooo 28 days ago - 6 comments
Labels: stale

#2492 - DOC: Add doc for ocr

Pull Request - State: closed - Opened by codingl2k1 28 days ago
Labels: documentation

#2491 - ClientDisconnect with Specific Code Content - Qwen 2.5 7B Coder Instruct

Issue - State: closed - Opened by danialcheung 29 days ago - 2 comments
Labels: gpu, stale

#2490 - [910B4]Embedding模型返回问题

Issue - State: closed - Opened by JasonFlyBeauty 29 days ago - 2 comments
Labels: gpu

#2489 - dify接入xinference CosyVoice-300M-SFT播放无声音，SenseVoiceSmall报错

Issue - State: closed - Opened by Wudaoguang 30 days ago - 5 comments
Labels: gpu, stale

#2488 - Qwen2-VL-instruct总是无法导入启动

Issue - State: closed - Opened by kaji331 about 1 month ago - 4 comments
Labels: gpu

#2487 - 使用 vLLM 启动 qwen2.5-32b-instruct 推理结果都是感叹号

Issue - State: closed - Opened by andylzming about 1 month ago - 11 comments
Labels: gpu, stale

#2486 - Model Engine 使用 vLLm 和 Transformers 启动 qwen2.5-32b-instruct 均出错

Issue - State: closed - Opened by andylzming about 1 month ago - 4 comments
Labels: gpu, stale

#2485 - DOC: modify NPU doc

Pull Request - State: closed - Opened by qinxuye about 1 month ago
Labels: documentation

GitHub / xorbitsai/inference issues and pull requests