Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / tatsu-lab/alpaca_eval issues and pull requests
#156 - what is alpaca_eval_gpt4
Issue -
State: closed - Opened by zhixiongzh 8 months ago
- 1 comment
#155 - cohere update
Pull Request -
State: closed - Opened by sanderland 8 months ago
- 2 comments
#154 - Is there a way to re-generate the leaderboard's index.html?
Issue -
State: closed - Opened by VincentV89 8 months ago
- 1 comment
#153 - Add Evo v2 7B
Pull Request -
State: closed - Opened by zfang 8 months ago
- 1 comment
#152 - Add Zephyr 7B evals
Pull Request -
State: closed - Opened by lewtun 8 months ago
- 1 comment
#151 - Add decoder for calling Anthropic models via Amazon Bedrock
Pull Request -
State: closed - Opened by billcai 8 months ago
- 3 comments
#150 - How to change GPT-4 API url?
Issue -
State: closed - Opened by hijkzzz 8 months ago
- 2 comments
#149 - test leaderboard
Pull Request -
State: closed - Opened by YannDubs 8 months ago
#148 - Add CausalLM/14B to AlpacaEval
Pull Request -
State: closed - Opened by CausalLM 8 months ago
- 1 comment
#147 - Add claude2-alpaca-13b, recycled-wizardlm-7b-v1.0, recycled-wizardlm-…
Pull Request -
State: closed - Opened by MingLiiii 8 months ago
- 1 comment
#146 - Add NEFTune models to AlpacaEval
Pull Request -
State: closed - Opened by neelsjain 9 months ago
- 1 comment
#145 - H4 changes
Pull Request -
State: closed - Opened by lewtun 9 months ago
- 1 comment
#144 - Add evo-7b to AlpacaEval
Pull Request -
State: closed - Opened by zfang 9 months ago
- 1 comment
#143 - Add PlatoLM-7B to AlpacaEval
Pull Request -
State: closed - Opened by renatz 9 months ago
- 1 comment
#142 - Add annotations & fix leaderboard
Pull Request -
State: closed - Opened by YannDubs 9 months ago
#141 - refresh Cohere
Pull Request -
State: closed - Opened by sanderland 9 months ago
- 2 comments
#140 - Add realm-7b to AlpacaEval
Pull Request -
State: closed - Opened by renatz 9 months ago
- 2 comments
#139 - add UltraLM-13b-V2.0/UltraLM-13b-V2.0-best-of-16/UltraLM-13b-best-of-16 to AlpacaEval
Pull Request -
State: closed - Opened by lifan-yuan 9 months ago
- 4 comments
#138 - Add UltraLM-13B-V2.0/UltraLM-13B-V2.0-best-of-16/UltraLM-13B-best-of-16 to AlpacaEval
Pull Request -
State: closed - Opened by lifan-yuan 9 months ago
#137 - [ENH] add gpt 3.5 instruct
Pull Request -
State: closed - Opened by YannDubs 10 months ago
#136 - What to think ?
Issue -
State: closed - Opened by vince62s 10 months ago
- 4 comments
#135 - Add results of Xwin-LM
Pull Request -
State: closed - Opened by nbl97 10 months ago
- 1 comment
#134 - error while using guanaco_33b as evaluator
Issue -
State: closed - Opened by vince62s 10 months ago
- 1 comment
#133 - llama2 evaluator
Issue -
State: closed - Opened by vince62s 10 months ago
#132 - Seems the leaderboard has stopped updating for a long time?
Issue -
State: closed - Opened by zmcmcc 10 months ago
- 1 comment
#131 - [ENH] add max concurrency oai
Pull Request -
State: closed - Opened by YannDubs 10 months ago
#130 - Add a fixed version GPT4 evaluator for the leaderboard
Issue -
State: closed - Opened by imoneoi 10 months ago
- 7 comments
Labels: wontfix
#129 - Add openbuddy-llama2-13b-v11.1
Pull Request -
State: closed - Opened by 44670 10 months ago
- 1 comment
#128 - API call fails with long outputs
Issue -
State: closed - Opened by imoneoi 10 months ago
- 6 comments
#127 - [ENH] add fixed gpt4 version annotator
Pull Request -
State: closed - Opened by YannDubs 10 months ago
#126 - Consider using a fixed version of GPT-4
Issue -
State: closed - Opened by imoneoi 10 months ago
- 2 comments
#125 - [ENH] return `completions_all` and allow sequence of max_tokens
Pull Request -
State: closed - Opened by YannDubs 10 months ago
#124 - Add `vllm` decoder for model inference
Pull Request -
State: closed - Opened by 44670 10 months ago
- 7 comments
#123 - [ENH] use async instead of multi processing
Issue -
State: closed - Opened by YannDubs 11 months ago
- 1 comment
#122 - add link to Humpback paper
Pull Request -
State: closed - Opened by xianxl 11 months ago
#121 - update Humpback results
Pull Request -
State: closed - Opened by xianxl 11 months ago
#120 - Add Humpback to AlpacaEval
Pull Request -
State: closed - Opened by xianxl 11 months ago
- 6 comments
#119 - Update llama 2 chat results without system prompt
Issue -
State: closed - Opened by rtaori 11 months ago
- 1 comment
#118 - [BUG] jina chat error in configs
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#117 - Add JinaChat to the leaderboards
Pull Request -
State: closed - Opened by jupyterjazz 11 months ago
- 3 comments
#116 - Ensure primary keys are string & decrease processes for OpenAI
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#115 - GPT4 rate limit
Issue -
State: closed - Opened by bing0037 11 months ago
- 2 comments
#114 - Separate files per provider?
Issue -
State: closed - Opened by krrishdholakia 11 months ago
- 2 comments
#113 - [BUG] utils.DUMMY_EXAMPLE
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#112 - [CLEAN] remove warning
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#111 - Add openbuddy-falcon-40b-v9 to AlpacaEval
Pull Request -
State: closed - Opened by 44670 11 months ago
- 2 comments
#110 - Fix typo on pretty_name
Pull Request -
State: closed - Opened by 44670 11 months ago
#109 - [BUG] closes #77
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#108 - Add openbuddy-llama-30b-v7.1 to AlpacaEval
Pull Request -
State: closed - Opened by 44670 11 months ago
- 9 comments
#107 - [DOC] Update Anthropic docstring
Issue -
State: closed - Opened by krrishdholakia 11 months ago
- 3 comments
#106 - [ENH] add example with HF API
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#105 - Add OpenChat 3.1 Results
Pull Request -
State: closed - Opened by imoneoi 11 months ago
- 1 comment
#104 - [CLEAN] Single annotator not abstract
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#103 - [ENH] add pipeline meta parser
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#102 - freewilly missing?
Issue -
State: closed - Opened by jtoy 11 months ago
- 2 comments
Labels: good first issue
#101 - Use chatGPT as baseline?
Issue -
State: closed - Opened by YannDubs 11 months ago
- 3 comments
#100 - [ENH] llama70B and chunking
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#99 - Update WizardLM 13B V1.2 results
Pull Request -
State: closed - Opened by victorsungo 11 months ago
- 1 comment
#98 - [STYLE] PR medium eval (ANNOTATOR_COLUMN)
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#97 - [STYLE] fix ill-formatted logging message
Pull Request -
State: closed - Opened by YannDubs 11 months ago
#96 - [style] fix ill-formatted logging message
Issue -
State: closed - Opened by lxuechen 11 months ago
#95 - [ENH] adds processors
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#94 - [ENH] json parser
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#93 - [ENH] eval_parser
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#92 - [ENH] optionally return raw completions
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#91 - [ENH] add llama 70B outputs
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#90 - [ENH] add replicate + llama 70B
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#89 - Check matching `input_ids` for llama2
Issue -
State: closed - Opened by YannDubs 12 months ago
- 1 comment
#88 - [ENH] automatically add minimal/verified
Pull Request -
State: closed - Opened by YannDubs 12 months ago
- 1 comment
#87 - [ENH] add chat llama2
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#86 - Add ChatGLM2-6B Results
Pull Request -
State: closed - Opened by inferLLM 12 months ago
- 1 comment
#85 - Add Baichuan-13B-Chat Results
Pull Request -
State: closed - Opened by inferLLM 12 months ago
#84 - Add Baichuan-13B-Chat Results
Pull Request -
State: closed - Opened by inferLLM 12 months ago
#83 - [ENH] adding samples to leadeboard
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#82 - [ENH] add format_sample_sheets.py to CI
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#81 - [ENH] make completion_parser easier to inherit
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#80 - chatgpt_fn returned json parsing error
Issue -
State: closed - Opened by hl395 12 months ago
- 2 comments
#79 - [ENH] Add length
Pull Request -
State: closed - Opened by YannDubs 12 months ago
- 1 comment
#78 - [ENH] add claude v2
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#77 - evaluate from model doesn't load leaderboard correctly
Issue -
State: closed - Opened by YannDubs 12 months ago
#76 - [ENH] add base annotator
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#75 - gpt4 annotations for vicuna v1.3
Pull Request -
State: closed - Opened by rtaori 12 months ago
#74 - add vicuna v1.3 results
Pull Request -
State: closed - Opened by rtaori 12 months ago
#73 - [ENH] make. it easier to cache to a DB
Pull Request -
State: closed - Opened by YannDubs 12 months ago
- 1 comment
#72 - [CI] auto release
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#71 - [CI] Ci pypi
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#70 - [ENH] add CI for package release
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#69 - [TESTS] Integration tests
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#68 - [TEST] Use doctest
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#67 - [BUG] push after running github action
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#66 - Update WizardLM 13B V1.1 results
Pull Request -
State: closed - Opened by victorsungo 12 months ago
- 2 comments
#65 - Pandas merge error
Issue -
State: closed - Opened by congyingxia 12 months ago
- 2 comments
#64 - [WIP] simplify inheritance for caching in DB
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#63 - Add OpenChat V2 Results
Pull Request -
State: closed - Opened by imoneoi 12 months ago
- 3 comments
#62 - Add Baize-v2 7B
Pull Request -
State: closed - Opened by JetRunner 12 months ago
- 3 comments
#61 - [CLEAN] setting up precommit
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#60 - [ENH] remove inputs from example
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#59 - format leaderboard for baize
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#58 - [CI] leaderboard formatting
Pull Request -
State: closed - Opened by YannDubs 12 months ago
#57 - [CI] CI for integration tests
Issue -
State: closed - Opened by YannDubs 12 months ago