Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / tatsu-lab/alpaca_eval issues and pull requests

#250 - [ENH] add contextual

Pull Request - State: closed - Opened by YannDubs 12 months ago

#249 - [ENH] add contextual

Pull Request - State: closed - Opened by YannDubs 12 months ago

#248 - [ENH] length controlled ALpacaEval

Pull Request - State: closed - Opened by YannDubs 12 months ago

#247 - [ENH] add claude 3

Pull Request - State: closed - Opened by YannDubs 12 months ago

#246 - add Contextual-KTO-Mistral-PairRM to AlpacaEval2

Pull Request - State: closed - Opened by xwinxu 12 months ago - 7 comments

#245 - add Mistral-7B-ReMax-v0.1

Pull Request - State: closed - Opened by liziniu 12 months ago - 1 comment

#244 - [NOTEBOOK] adding final length correction notebook.

Pull Request - State: closed - Opened by YannDubs 12 months ago

#243 - Possibility of adding a version signature

Issue - State: closed - Opened by mathewhuen 12 months ago - 4 comments

#242 - [DATA] Add Gemma

Pull Request - State: closed - Opened by YannDubs 12 months ago

#241 - [ENH] update to allow AF to use AE

Pull Request - State: closed - Opened by YannDubs 12 months ago

#240 - [ENH] add inverse mapper to make sure in and out types are the same

Pull Request - State: closed - Opened by YannDubs 12 months ago

#239 - Adding trust remote code=True into model_kwargs

Pull Request - State: closed - Opened by abgoswam almost 1 year ago

#238 - Repeated deprecation errors

Issue - State: closed - Opened by syleedandekar almost 1 year ago - 2 comments

#237 - [NOTEBOOK] add length-corrected GLM

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#236 - TypeError: Subscripted generics cannot be used with class and instance checks

Issue - State: closed - Opened by iGangao about 1 year ago - 1 comment

#235 - update ELO for llama-2-13b-chat-hf

Pull Request - State: closed - Opened by gblazex about 1 year ago

#234 - [DATA] add results from the Arena openai models

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#233 - Update ELO scores to Feb 2

Pull Request - State: closed - Opened by gblazex about 1 year ago - 1 comment

#232 - [DOC] add annotation interpretation

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#231 - [DEV] Analyzing length-controlled metrics.

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#230 - Update README.md - Add missing "Y" to "ou"

Pull Request - State: closed - Opened by yoderj about 1 year ago

#229 - [DATA] Adding annotations for the arena models

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#226 - Add Qwen1.5-72B-Chat to AlpacaEval

Pull Request - State: closed - Opened by Lukeming-tsinghua about 1 year ago - 3 comments

#225 - Potential length-controlled metric for Alpaca Eval 2.0

Issue - State: closed - Opened by viethoangtranduong about 1 year ago - 25 comments

#224 - [ENH] add referenced_models locally

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#223 - clarification on annotation entries from `alpaca_eval`

Issue - State: closed - Opened by xwinxu about 1 year ago - 4 comments

#222 - delete

Pull Request - State: closed - Opened by sambroy about 1 year ago

#221 - Add xwinlm-70b-v0.3 to AlpacaEval

Pull Request - State: closed - Opened by nbl97 about 1 year ago

#220 - Trouble re-creating the guanaco_33b evaluator baseline

Issue - State: closed - Opened by mathewhuen about 1 year ago - 6 comments

#219 - Enable specifying a custom path for model configs ?

Issue - State: closed - Opened by stoicio about 1 year ago - 3 comments

#218 - [RES] add 3 models for arena correlations

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#216 - update InternLM2 chat template

Pull Request - State: closed - Opened by C1rN09 about 1 year ago

#215 - Add Snorkel-Mistral-PairRM-DPO (best-of-16) to Alpaca Eval 2.0

Pull Request - State: closed - Opened by viethoangtranduong about 1 year ago - 2 comments

#214 - [TEST]: fix ordering of df

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#213 - Update README.md (small typo)

Pull Request - State: closed - Opened by xwinxu about 1 year ago - 1 comment

#212 - dolphin 2.1.1 configs.yaml

Pull Request - State: closed - Opened by gblazex about 1 year ago - 1 comment

#211 - Any arxiv paper for reference?

Issue - State: closed - Opened by zhimin-z about 1 year ago - 3 comments

#210 - Add PairRM 0.4B + Yi-34B-Chat to AlpacaEval 2.0

Pull Request - State: closed - Opened by jdf-prog about 1 year ago - 4 comments

#209 - [ENH] add outputs & configs form dolphin 2.2.1

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 2 comments

#208 - prettify "pretty_name" of internlm2

Pull Request - State: closed - Opened by C1rN09 about 1 year ago - 1 comment

#207 - [ENH] add internlm2-chat-20b-ppo

Pull Request - State: closed - Opened by C1rN09 about 1 year ago - 1 comment

#206 - log prob error when using Azure API

Issue - State: open - Opened by chengjl19 about 1 year ago - 2 comments

#205 - [ENH] add mistral-medium

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#204 - Bugs in parsing alpaca eval 2.0

Issue - State: closed - Opened by chengjl19 about 1 year ago - 8 comments

#203 - [ENH] add OpenHermes

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#202 - [BUG] force openai >1.5.0

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#201 - TypeError When Using weighted_alpaca_eval_gpt4_turbo Annotator

Issue - State: closed - Opened by efrick2002 about 1 year ago - 2 comments

#200 - [BUG] fix no OAI org id set

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#199 - [WIP] precompute all leaderboard for AE2

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#198 - a bug about `openai_organization_ids`

Issue - State: closed - Opened by yuchenlin about 1 year ago - 2 comments

#197 - Yann/alpaca eval 2

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#196 - [ENH] alpaca_eval 2.0

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#195 - [ENH] new models: Gemini / claude2.1 / mistral / mixtral / ..

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#195 - [ENH] new models: Gemini / claude2.1 / mistral / mixtral / ..

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#194 - [ENH] new models: Gemini / claude2.1 / mistral / mixtral

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#194 - [ENH] new models: Gemini / claude2.1 / mistral / mixtral

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#193 - [ENH] Azure OAI client & more general way of switching between client configs

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#193 - [ENH] Azure OAI client & more general way of switching between client configs

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#192 - Add deita-7b-v1.0 model

Pull Request - State: closed - Opened by VPeterV about 1 year ago - 1 comment

#191 - chore: fix links

Pull Request - State: closed - Opened by lxuechen about 1 year ago

#190 - chore: add link for phi-2-sft

Pull Request - State: closed - Opened by lxuechen about 1 year ago

#189 - [ENH] Weighted win rates

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#188 - Update openai.py

Pull Request - State: closed - Opened by Muennighoff about 1 year ago

#187 - [ENH] potential improvements for AlpacaEval 2.0

Issue - State: closed - Opened by YannDubs about 1 year ago
Labels: enhancement

#186 - add cut-13b

Pull Request - State: closed - Opened by wwxu21 about 1 year ago - 1 comment

#185 - chore: add phi-2 dpo

Pull Request - State: closed - Opened by lxuechen about 1 year ago - 3 comments

#184 - chore: add phi-2 sft

Pull Request - State: closed - Opened by lxuechen about 1 year ago

#183 - Support phi2, Support SOLAR 10.7B LMCocktail

Pull Request - State: closed - Opened by yhyu13 about 1 year ago - 9 comments

#182 - Verify Yi

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#181 - Add PairRM best-of-16 to AlpacaEval

Pull Request - State: closed - Opened by jdf-prog about 1 year ago - 3 comments

#180 - How to rate limit for gpt4 ?

Issue - State: closed - Opened by jaiabhayk about 1 year ago - 1 comment

#179 - Modify configs of 01-ai/Yi-34B-Chat to make model verified

Pull Request - State: closed - Opened by HyperdriveHustle about 1 year ago - 2 comments

#178 - show img in readme

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#177 - feat: add way to verify results

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#176 - Add MiniChat-1.5-3B to AlpacaEval and Fix MiniChat-3B

Pull Request - State: closed - Opened by GeneZC about 1 year ago

#175 - Add 01-ai/Yi-34B-Chat to AlpacaEval

Pull Request - State: closed - Opened by HyperdriveHustle about 1 year ago - 1 comment

#174 - Fix mssg check

Pull Request - State: closed - Opened by Muennighoff about 1 year ago

#173 - Fix the results of MiniChat-3B

Pull Request - State: closed - Opened by GeneZC about 1 year ago

#172 - Data parallelism in `evaluate_from_model`?

Issue - State: closed - Opened by liutianlin0121 about 1 year ago - 2 comments

#171 - Add Tulu 2 models to AlpacaEval

Pull Request - State: closed - Opened by hamishivi over 1 year ago - 1 comment

#171 - Add Tulu 2 models to AlpacaEval

Pull Request - State: closed - Opened by hamishivi over 1 year ago - 1 comment

#170 - feat: verify all the cohere model & use it as eval

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#169 - fix: filter openai spam filter

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#168 - OpenAI response error

Issue - State: closed - Opened by RZFan525 over 1 year ago - 3 comments

#167 - Add minichat-3b to AlpacaEval

Pull Request - State: closed - Opened by GeneZC over 1 year ago - 1 comment

#166 - text davinci-003 is closed?????

Issue - State: closed - Opened by kkwhale7 over 1 year ago - 5 comments

#165 - [ENH] add GPT4 turbo as evaluator in README

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#164 - Bugs from reference model while evaluating

Issue - State: closed - Opened by kkwhale7 over 1 year ago - 7 comments

#163 - alpaca_eval_gpt4_turbo as evaluator quality

Issue - State: closed - Opened by nlpcat over 1 year ago - 1 comment

#162 - Evaluator fails with weird generations

Issue - State: closed - Opened by natolambert over 1 year ago - 8 comments

#161 - Modify the baseline to a stronger model, such as ChatGPT

Issue - State: closed - Opened by chengjl19 over 1 year ago - 4 comments

#160 - [WIP] GPT4 turbo as evaluator

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#159 - Gpt4 turbo

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#157 - feat: upgrade to openai 1.0.0

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#156 - what is alpaca_eval_gpt4

Issue - State: closed - Opened by zhixiongzh over 1 year ago - 1 comment

#155 - cohere update

Pull Request - State: closed - Opened by sanderland over 1 year ago - 2 comments