Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / tatsu-lab/alpaca_eval issues and pull requests

#154 - Is there a way to re-generate the leaderboard's index.html?

Issue - State: closed - Opened by VincentV89 over 1 year ago - 1 comment

#153 - Add Evo v2 7B

Pull Request - State: closed - Opened by zfang over 1 year ago - 1 comment

#152 - Add Zephyr 7B evals

Pull Request - State: closed - Opened by lewtun over 1 year ago - 1 comment

#151 - Add decoder for calling Anthropic models via Amazon Bedrock

Pull Request - State: closed - Opened by billcai over 1 year ago - 3 comments

#150 - How to change GPT-4 API url?

Issue - State: closed - Opened by hijkzzz over 1 year ago - 2 comments

#149 - test leaderboard

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#148 - Add CausalLM/14B to AlpacaEval

Pull Request - State: closed - Opened by CausalLM over 1 year ago - 1 comment

#147 - Add claude2-alpaca-13b, recycled-wizardlm-7b-v1.0, recycled-wizardlm-…

Pull Request - State: closed - Opened by MingLiiii over 1 year ago - 1 comment

#146 - Add NEFTune models to AlpacaEval

Pull Request - State: closed - Opened by neelsjain over 1 year ago - 1 comment

#145 - H4 changes

Pull Request - State: closed - Opened by lewtun over 1 year ago - 1 comment

#144 - Add evo-7b to AlpacaEval

Pull Request - State: closed - Opened by zfang over 1 year ago - 1 comment

#143 - Add PlatoLM-7B to AlpacaEval

Pull Request - State: closed - Opened by renatz over 1 year ago - 1 comment

#142 - Add annotations & fix leaderboard

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#141 - refresh Cohere

Pull Request - State: closed - Opened by sanderland over 1 year ago - 2 comments

#140 - Add realm-7b to AlpacaEval

Pull Request - State: closed - Opened by renatz over 1 year ago - 2 comments

#139 - add UltraLM-13b-V2.0/UltraLM-13b-V2.0-best-of-16/UltraLM-13b-best-of-16 to AlpacaEval

Pull Request - State: closed - Opened by lifan-yuan over 1 year ago - 4 comments

#137 - [ENH] add gpt 3.5 instruct

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#136 - What to think ?

Issue - State: closed - Opened by vince62s over 1 year ago - 4 comments

#135 - Add results of Xwin-LM

Pull Request - State: closed - Opened by nbl97 over 1 year ago - 1 comment

#134 - error while using guanaco_33b as evaluator

Issue - State: closed - Opened by vince62s over 1 year ago - 1 comment

#133 - llama2 evaluator

Issue - State: closed - Opened by vince62s over 1 year ago

#132 - Seems the leaderboard has stopped updating for a long time?

Issue - State: closed - Opened by zmcmcc over 1 year ago - 1 comment

#131 - [ENH] add max concurrency oai

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#130 - Add a fixed version GPT4 evaluator for the leaderboard

Issue - State: closed - Opened by imoneoi over 1 year ago - 7 comments
Labels: wontfix

#129 - Add openbuddy-llama2-13b-v11.1

Pull Request - State: closed - Opened by 44670 over 1 year ago - 1 comment

#128 - API call fails with long outputs

Issue - State: closed - Opened by imoneoi over 1 year ago - 6 comments

#127 - [ENH] add fixed gpt4 version annotator

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#126 - Consider using a fixed version of GPT-4

Issue - State: closed - Opened by imoneoi over 1 year ago - 2 comments

#125 - [ENH] return `completions_all` and allow sequence of max_tokens

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#124 - Add `vllm` decoder for model inference

Pull Request - State: closed - Opened by 44670 over 1 year ago - 7 comments

#123 - [ENH] use async instead of multi processing

Issue - State: closed - Opened by YannDubs over 1 year ago - 1 comment

#122 - add link to Humpback paper

Pull Request - State: closed - Opened by xianxl over 1 year ago

#121 - update Humpback results

Pull Request - State: closed - Opened by xianxl over 1 year ago

#120 - Add Humpback to AlpacaEval

Pull Request - State: closed - Opened by xianxl over 1 year ago - 6 comments

#119 - Update llama 2 chat results without system prompt

Issue - State: closed - Opened by rtaori over 1 year ago - 1 comment

#118 - [BUG] jina chat error in configs

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#117 - Add JinaChat to the leaderboards

Pull Request - State: closed - Opened by jupyterjazz over 1 year ago - 3 comments

#116 - Ensure primary keys are string & decrease processes for OpenAI

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#115 - GPT4 rate limit

Issue - State: closed - Opened by bing0037 over 1 year ago - 2 comments

#114 - Separate files per provider?

Issue - State: closed - Opened by krrishdholakia over 1 year ago - 2 comments

#113 - [BUG] utils.DUMMY_EXAMPLE

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#112 - [CLEAN] remove warning

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#111 - Add openbuddy-falcon-40b-v9 to AlpacaEval

Pull Request - State: closed - Opened by 44670 over 1 year ago - 2 comments

#110 - Fix typo on pretty_name

Pull Request - State: closed - Opened by 44670 over 1 year ago

#109 - [BUG] closes #77

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#108 - Add openbuddy-llama-30b-v7.1 to AlpacaEval

Pull Request - State: closed - Opened by 44670 over 1 year ago - 9 comments

#107 - [DOC] Update Anthropic docstring

Issue - State: closed - Opened by krrishdholakia over 1 year ago - 3 comments

#106 - [ENH] add example with HF API

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#105 - Add OpenChat 3.1 Results

Pull Request - State: closed - Opened by imoneoi over 1 year ago - 1 comment

#104 - [CLEAN] Single annotator not abstract

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#103 - [ENH] add pipeline meta parser

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#102 - freewilly missing?

Issue - State: closed - Opened by jtoy over 1 year ago - 2 comments
Labels: good first issue

#101 - Use chatGPT as baseline?

Issue - State: closed - Opened by YannDubs over 1 year ago - 3 comments

#100 - [ENH] llama70B and chunking

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#99 - Update WizardLM 13B V1.2 results

Pull Request - State: closed - Opened by victorsungo over 1 year ago - 1 comment

#98 - [STYLE] PR medium eval (ANNOTATOR_COLUMN)

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#97 - [STYLE] fix ill-formatted logging message

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#96 - [style] fix ill-formatted logging message

Issue - State: closed - Opened by lxuechen over 1 year ago

#95 - [ENH] adds processors

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#94 - [ENH] json parser

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#93 - [ENH] eval_parser

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#92 - [ENH] optionally return raw completions

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#91 - [ENH] add llama 70B outputs

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#90 - [ENH] add replicate + llama 70B

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#89 - Check matching `input_ids` for llama2

Issue - State: closed - Opened by YannDubs over 1 year ago - 1 comment

#88 - [ENH] automatically add minimal/verified

Pull Request - State: closed - Opened by YannDubs over 1 year ago - 1 comment

#87 - [ENH] add chat llama2

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#86 - Add ChatGLM2-6B Results

Pull Request - State: closed - Opened by inferLLM over 1 year ago - 1 comment

#85 - Add Baichuan-13B-Chat Results

Pull Request - State: closed - Opened by inferLLM over 1 year ago

#84 - Add Baichuan-13B-Chat Results

Pull Request - State: closed - Opened by inferLLM over 1 year ago

#83 - [ENH] adding samples to leadeboard

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#82 - [ENH] add format_sample_sheets.py to CI

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#81 - [ENH] make completion_parser easier to inherit

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#80 - chatgpt_fn returned json parsing error

Issue - State: closed - Opened by hl395 over 1 year ago - 2 comments

#79 - [ENH] Add length

Pull Request - State: closed - Opened by YannDubs over 1 year ago - 1 comment

#78 - [ENH] add claude v2

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#77 - evaluate from model doesn't load leaderboard correctly

Issue - State: closed - Opened by YannDubs over 1 year ago

#76 - [ENH] add base annotator

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#75 - gpt4 annotations for vicuna v1.3

Pull Request - State: closed - Opened by rtaori over 1 year ago

#74 - add vicuna v1.3 results

Pull Request - State: closed - Opened by rtaori over 1 year ago

#73 - [ENH] make. it easier to cache to a DB

Pull Request - State: closed - Opened by YannDubs over 1 year ago - 1 comment

#72 - [CI] auto release

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#71 - [CI] Ci pypi

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#70 - [ENH] add CI for package release

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#69 - [TESTS] Integration tests

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#68 - [TEST] Use doctest

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#67 - [BUG] push after running github action

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#66 - Update WizardLM 13B V1.1 results

Pull Request - State: closed - Opened by victorsungo over 1 year ago - 2 comments

#65 - Pandas merge error

Issue - State: closed - Opened by congyingxia over 1 year ago - 2 comments

#64 - [WIP] simplify inheritance for caching in DB

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#63 - Add OpenChat V2 Results

Pull Request - State: closed - Opened by imoneoi over 1 year ago - 3 comments

#62 - Add Baize-v2 7B

Pull Request - State: closed - Opened by JetRunner over 1 year ago - 3 comments

#61 - [CLEAN] setting up precommit

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#60 - [ENH] remove inputs from example

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#59 - format leaderboard for baize

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#58 - [CI] leaderboard formatting

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#57 - [CI] CI for integration tests

Issue - State: closed - Opened by YannDubs over 1 year ago

#56 - [TEST] setting up test CI

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#55 - [CLEAN] black

Pull Request - State: closed - Opened by YannDubs over 1 year ago