Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / THUDM/AgentBench issues and pull requests

#168 - [Assistance] 如何实现demo视频中的效果

Issue - State: open - Opened by XGJ111 about 1 month ago
Labels: bug, help wanted

#167 - webshop场景，为什么有些搜索没有结果，导致任务失败

Issue - State: open - Opened by kai0705 2 months ago
Labels: enhancement

#165 - Update README.md

Pull Request - State: closed - Opened by Xiao9905 3 months ago

#164 - OS-task catch errors in container init

Pull Request - State: open - Opened by rjmoss 3 months ago

#163 - Fixed hanging bash commands from agent in os-task

Pull Request - State: open - Opened by rjmoss 3 months ago

#162 - Fixed terminal output parsing

Pull Request - State: open - Opened by rjmoss 3 months ago

#161 - where can we find the api of sparql？

Issue - State: closed - Opened by cssx1234 3 months ago
Labels: enhancement

#160 - kg的服务我部署好了，但是还是不能够正常测评kg任务，具体错误如下

Issue - State: closed - Opened by minleminzui 3 months ago - 1 comment
Labels: bug, help wanted

#159 - [Bug/Assistance] kg-std issues

Issue - State: closed - Opened by night-chen 3 months ago - 1 comment
Labels: bug, help wanted

#158 - Update README.md for local deployment of KG service

Pull Request - State: closed - Opened by finger1517 3 months ago

#157 - Update README.md for local deployment of KG service

Pull Request - State: closed - Opened by finger1517 3 months ago

#156 - Add Quitting for OS task

Pull Request - State: closed - Opened by dillonmsandhu 3 months ago

#155 - Any plans to add new models?

Issue - State: open - Opened by ryoungj 3 months ago - 1 comment

#154 - [Bug/Assistance]

Issue - State: open - Opened by matinaghaei 4 months ago
Labels: bug, help wanted

#152 - Could you please upload the dockerfile?

Issue - State: open - Opened by HCHCXY 4 months ago - 3 comments
Labels: bug, help wanted

#151 - [Bug/Assistance] A lot of os-std tasks are impossible

Issue - State: open - Opened by rjmoss 4 months ago
Labels: bug, help wanted

#150 - [Bug/Assistance] how to use local model to replace gpt3.5?

Issue - State: open - Opened by lambda7xx 4 months ago - 2 comments
Labels: bug, help wanted

#149 - fix: fix AgentBench/data/os_interaction/data/4/ N11.json

Pull Request - State: open - Opened by minleminzui 4 months ago

#146 - 通过fastchat部署本地模型遇到的问题

Issue - State: open - Opened by YinSonglin1997 4 months ago - 12 comments
Labels: bug, help wanted

#145 - DBbench-std task with error "Can't connect to MySQL server"

Issue - State: open - Opened by realbillbao 5 months ago - 2 comments
Labels: bug, help wanted

#143 - Fix typo in os agent instruction

Pull Request - State: closed - Opened by rjmoss 5 months ago

#142 - 请问trajectories有公开吗

Issue - State: open - Opened by yananchen1989 5 months ago

#141 - [Feature] Add a LICENSE to the project

Issue - State: closed - Opened by cjoverbay 5 months ago - 2 comments
Labels: enhancement

#140 - Stupidd cupid patch 1

Pull Request - State: closed - Opened by StupiddCupid 6 months ago

#139 - Zifei

Pull Request - State: closed - Opened by StupiddCupid 6 months ago - 3 comments

#134 - Fix typo in README.md

Pull Request - State: closed - Opened by petrgazarov 6 months ago

#132 - 请问支持使用openai的tool_call接口进行测试吗？

Issue - State: open - Opened by Maybewuss 7 months ago - 1 comment
Labels: enhancement

#130 - Excellent Job! Well, no offense, it seems LLM-Bench rather than AgentBench in essence.

Issue - State: open - Opened by Konisberg 8 months ago - 1 comment
Labels: enhancement

#129 - [Bug/Assistance] mind2web的unknown是怎么回事？

Issue - State: open - Opened by Tangent-90C 8 months ago - 1 comment
Labels: bug, help wanted

#128 - OS std 测试集结果

Issue - State: open - Opened by xqun3 8 months ago - 1 comment
Labels: bug, help wanted

#127 - [Bug/Assistance] - Reproducing Results on Alfworld (HH) (vs. ReAct paper)

Issue - State: open - Opened by ai-nikolai 8 months ago - 4 comments
Labels: bug, help wanted

#126 - 增加对Cluade3的评测

Issue - State: open - Opened by xqun3 8 months ago - 2 comments
Labels: enhancement

#125 - format all files using black

Pull Request - State: closed - Opened by EYH0602 8 months ago

#124 - Connection error

Issue - State: closed - Opened by StupiddCupid 8 months ago - 3 comments
Labels: bug, help wanted

#123 - Fix Execution Permission Issue and Adjust LTP Task Rounds

Pull Request - State: closed - Opened by Taishi-N324 8 months ago - 2 comments

#122 - Benchmark for mistral models

Issue - State: open - Opened by mingxuan-he 9 months ago - 1 comment
Labels: enhancement

#121 - Card_Game这个任务跑不起来

Issue - State: open - Opened by yupeijei1997 9 months ago - 4 comments
Labels: bug, help wanted

#120 - 修复因容器与宿主机控制器连接问题导致的“Task does not exist”

Pull Request - State: closed - Opened by Tangent-90C 9 months ago - 3 comments

#118 - [Feature] Use for benchmarking agents like AutoGPT?

Issue - State: closed - Opened by shruti222patel 9 months ago - 1 comment
Labels: enhancement

#117 - Update README.md

Pull Request - State: closed - Opened by Longin-Yu 9 months ago

#116 - [Bug/Assistance] kg-std任务运行的runs.jsonl文件中问题在数据集中找不到

Issue - State: closed - Opened by 13416157913 9 months ago - 4 comments
Labels: bug, help wanted

#115 - [Bug/Assistance] 测试kg-std任务时，输出文件中全部状态都是task limit reached

Issue - State: open - Opened by 13416157913 9 months ago - 1 comment
Labels: bug, help wanted

#113 - [Bug/Assistance] 测试os-std任务，提示Message: 0 samples remaining.

Issue - State: closed - Opened by 13416157913 10 months ago - 6 comments
Labels: bug, help wanted

#112 - [Bug/Assistance] OS任务报错AttributeError: 'NpipeSocket' object has no attribute '_sock'

Issue - State: closed - Opened by 13416157913 10 months ago - 2 comments
Labels: bug, help wanted

#110 - ltp无法启动

Issue - State: open - Opened by Fu-Dayuan 10 months ago - 1 comment
Labels: bug, help wanted

#109 - [Bug/Assistance]

Issue - State: open - Opened by ibingzhaoi 10 months ago - 5 comments
Labels: bug, help wanted

#108 - dbbench-std: Task Output Seems Correct But MD5 Mismatches

Issue - State: open - Opened by wchen-github 10 months ago - 1 comment
Labels: bug, help wanted

#107 - agentbench 能跑训练集么？

Issue - State: open - Opened by Fu-Dayuan 10 months ago - 1 comment
Labels: bug, help wanted

#106 - [Bug/Assistance] DBBench Unknown database

Issue - State: open - Opened by LittleWhite0208 10 months ago - 1 comment
Labels: bug, help wanted

#105 - [Bug/Assistance] os-std某一条数据报错Worker not responding

Issue - State: open - Opened by Xccanxin 10 months ago - 1 comment
Labels: bug, help wanted

#103 - [Assistance] Need some example running logs

Issue - State: open - Opened by ROCKYWWWW 10 months ago - 2 comments
Labels: bug, help wanted

#102 - [Bug/Assistance] 怎么配置configs/agents/openai-chat.yaml

Issue - State: closed - Opened by yananchen1989 10 months ago - 1 comment
Labels: bug, help wanted

#100 - 请问一下为什么output文件夹里没有overall.json?

Issue - State: closed - Opened by tml2002 10 months ago
Labels: enhancement

#99 - [Bug/Assistance]

Issue - State: closed - Opened by tml2002 10 months ago
Labels: bug, help wanted

#98 - [Bug/Assistance]

Issue - State: closed - Opened by tml2002 10 months ago
Labels: bug, help wanted

#97 - cg和kg都遇到了Worker not responding

Issue - State: open - Opened by WarBean 11 months ago - 1 comment
Labels: bug, help wanted

#96 - 游戏任务启动失败[Assistance]

Issue - State: open - Opened by smartliuhw 11 months ago - 3 comments
Labels: bug, help wanted

#95 - Update Config_en.md

Pull Request - State: closed - Opened by ZiyueWang25 11 months ago

#94 - Update README.md

Pull Request - State: closed - Opened by ZiyueWang25 11 months ago

#93 - 可否不用docker配置环境

Issue - State: closed - Opened by smartliuhw 11 months ago - 2 comments
Labels: enhancement

#92 - 我想看一下agent和server的交互函数，可以指导一下嘛

Issue - State: closed - Opened by hushuang909 11 months ago - 2 comments
Labels: bug, help wanted

#91 - About Webshop

Issue - State: closed - Opened by dapengchen1234 11 months ago - 1 comment
Labels: bug, help wanted

#90 - Fix typo: AgentClient.reference --> AgentClient.inference

Pull Request - State: closed - Opened by BarryRun 11 months ago

#89 - [Bug/Assistance] DBbench任务评测结果与leaderboard不一致

Issue - State: open - Opened by SummerXIATIAN 11 months ago - 1 comment
Labels: bug, help wanted

#88 - KBQA 任务数据集信息确认

Issue - State: closed - Opened by WuXuan374 11 months ago

#87 - cg任务没有一条执行成功而且task server没有收到任何信息

Issue - State: open - Opened by Jianzhao-Huang 11 months ago - 1 comment
Labels: bug, help wanted

#86 - [Assistance] Connection Error

Issue - State: closed - Opened by wz1211 11 months ago - 1 comment
Labels: bug, help wanted

#85 - [Bug/Assistance] The option link fails to jump

Issue - State: open - Opened by zhimin-z 11 months ago
Labels: bug, help wanted

#84 - Error with Command “python -m src.start_task -a”

Issue - State: closed - Opened by ericzdzhang 11 months ago - 5 comments

#83 - How to test in self customed data?

Issue - State: closed - Opened by Reason-Wang 11 months ago - 1 comment

#81 - Separate server for task and model

Issue - State: closed - Opened by Reason-Wang 12 months ago - 2 comments

#80 - [Assistance] 如何获得每个task的得分？

Issue - State: closed - Opened by Jiaqi0109 12 months ago - 1 comment
Labels: bug, help wanted

#79 - How to calculate the overall score?

Issue - State: closed - Opened by zhimin-z 12 months ago - 1 comment
Labels: bug, help wanted

#78 - 运行AgentBench报错

Issue - State: closed - Opened by QingChengLineOne 12 months ago - 2 comments
Labels: bug, help wanted

#77 - 我想将api接口改为ChatGLM3，我该怎么做

Issue - State: closed - Opened by QingChengLineOne 12 months ago - 5 comments
Labels: bug, help wanted

#76 - [Assistance] How to change the prompt in the task

Issue - State: closed - Opened by Z-ZHHH 12 months ago - 2 comments
Labels: bug, help wanted

#75 - How can I use other LLM, such as LLAMA2?

Issue - State: closed - Opened by wangyf456 almost 1 year ago - 4 comments
Labels: bug, help wanted

#73 - 关于('{"detail":"Error: Task does not exist"}', 400, 'alfworld-std')问题

Issue - State: closed - Opened by XiaoShihua about 1 year ago - 8 comments

#72 - [Assistance] Number of problems in the OS dataset

Issue - State: open - Opened by deema-A about 1 year ago - 2 comments
Labels: bug, help wanted

#71 - [Feature] Integrate with LiteLLM - Evaluate 100+LLMs, 92% faster

Issue - State: closed - Opened by ishaan-jaff about 1 year ago - 1 comment
Labels: enhancement

#70 - INTERACT_FAILED Error: Session does not exist

Issue - State: closed - Opened by glad4enkonm about 1 year ago - 3 comments
Labels: bug, help wanted

#69 - Evaluation results is always 0, and different from the Leaderboard

Issue - State: open - Opened by lynneChan about 1 year ago - 4 comments

#68 - Can not run webshop task correctly

Issue - State: closed - Opened by lynneChan about 1 year ago - 4 comments
Labels: bug, help wanted

#67 - [Bug/Assistance] document typos

Issue - State: closed - Opened by bwin90 about 1 year ago - 1 comment
Labels: bug, help wanted