tatsu-lab/alpaca_eval issues and pull requests

#116 - Ensure primary keys are string & decrease processes for OpenAI

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#115 - GPT4 rate limit

Issue - State: closed - Opened by bing0037 about 1 year ago - 2 comments

#114 - Separate files per provider?

Issue - State: closed - Opened by krrishdholakia about 1 year ago - 2 comments

#113 - [BUG] utils.DUMMY_EXAMPLE

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#112 - [CLEAN] remove warning

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#111 - Add openbuddy-falcon-40b-v9 to AlpacaEval

Pull Request - State: closed - Opened by 44670 about 1 year ago - 2 comments

#110 - Fix typo on pretty_name

Pull Request - State: closed - Opened by 44670 about 1 year ago

#109 - [BUG] closes #77

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#108 - Add openbuddy-llama-30b-v7.1 to AlpacaEval

Pull Request - State: closed - Opened by 44670 about 1 year ago - 9 comments

#107 - [DOC] Update Anthropic docstring

Issue - State: closed - Opened by krrishdholakia about 1 year ago - 3 comments

#106 - [ENH] add example with HF API

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#105 - Add OpenChat 3.1 Results

Pull Request - State: closed - Opened by imoneoi about 1 year ago - 1 comment

#104 - [CLEAN] Single annotator not abstract

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#103 - [ENH] add pipeline meta parser

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#102 - freewilly missing?

Issue - State: closed - Opened by jtoy about 1 year ago - 2 comments
Labels: good first issue

#101 - Use chatGPT as baseline?

Issue - State: closed - Opened by YannDubs about 1 year ago - 3 comments

#100 - [ENH] llama70B and chunking

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#99 - Update WizardLM 13B V1.2 results

Pull Request - State: closed - Opened by victorsungo about 1 year ago - 1 comment

#98 - [STYLE] PR medium eval (ANNOTATOR_COLUMN)

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#97 - [STYLE] fix ill-formatted logging message

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#96 - [style] fix ill-formatted logging message

Issue - State: closed - Opened by lxuechen about 1 year ago

#95 - [ENH] adds processors

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#94 - [ENH] json parser

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#93 - [ENH] eval_parser

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#92 - [ENH] optionally return raw completions

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#91 - [ENH] add llama 70B outputs

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#90 - [ENH] add replicate + llama 70B

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#89 - Check matching `input_ids` for llama2

Issue - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#88 - [ENH] automatically add minimal/verified

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#87 - [ENH] add chat llama2

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#86 - Add ChatGLM2-6B Results

Pull Request - State: closed - Opened by inferLLM about 1 year ago - 1 comment

#85 - Add Baichuan-13B-Chat Results

Pull Request - State: closed - Opened by inferLLM about 1 year ago

#84 - Add Baichuan-13B-Chat Results

Pull Request - State: closed - Opened by inferLLM about 1 year ago

#83 - [ENH] adding samples to leadeboard

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#82 - [ENH] add format_sample_sheets.py to CI

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#81 - [ENH] make completion_parser easier to inherit

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#80 - chatgpt_fn returned json parsing error

Issue - State: closed - Opened by hl395 about 1 year ago - 2 comments

#79 - [ENH] Add length

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#78 - [ENH] add claude v2

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#77 - evaluate from model doesn't load leaderboard correctly

Issue - State: closed - Opened by YannDubs about 1 year ago

#76 - [ENH] add base annotator

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#75 - gpt4 annotations for vicuna v1.3

Pull Request - State: closed - Opened by rtaori about 1 year ago

#74 - add vicuna v1.3 results

Pull Request - State: closed - Opened by rtaori about 1 year ago

#73 - [ENH] make. it easier to cache to a DB

Pull Request - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#72 - [CI] auto release

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#71 - [CI] Ci pypi

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#70 - [ENH] add CI for package release

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#69 - [TESTS] Integration tests

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#68 - [TEST] Use doctest

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#67 - [BUG] push after running github action

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#66 - Update WizardLM 13B V1.1 results

Pull Request - State: closed - Opened by victorsungo about 1 year ago - 2 comments

#65 - Pandas merge error

Issue - State: closed - Opened by congyingxia about 1 year ago - 2 comments

#64 - [WIP] simplify inheritance for caching in DB

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#63 - Add OpenChat V2 Results

Pull Request - State: closed - Opened by imoneoi about 1 year ago - 3 comments

#62 - Add Baize-v2 7B

Pull Request - State: closed - Opened by JetRunner about 1 year ago - 3 comments

#61 - [CLEAN] setting up precommit

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#60 - [ENH] remove inputs from example

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#59 - format leaderboard for baize

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#58 - [CI] leaderboard formatting

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#57 - [CI] CI for integration tests

Issue - State: closed - Opened by YannDubs about 1 year ago

#56 - [TEST] setting up test CI

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#55 - [CLEAN] black

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#54 - [ENH] upgrade anthropic 0.3

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#53 - [ENH] upgrade anthropic 0.3

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#52 - [ENH] anthropic 0.3

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#51 - [CLEAN] apply black

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#50 - [TEST] test

Pull Request - State: closed - Opened by YannDubs about 1 year ago

#49 - Add Baize v2 13B

Pull Request - State: closed - Opened by JetRunner about 1 year ago - 1 comment

#48 - Update `example/outputs.json` to use merged instruction

Issue - State: closed - Opened by rtaori about 1 year ago

#47 - get falcon models running decoding

Pull Request - State: closed - Opened by rtaori about 1 year ago

#46 - Falcon support for generation

Issue - State: closed - Opened by rtaori about 1 year ago - 1 comment

#45 - [CI] add leaderboard formatting to CI

Issue - State: closed - Opened by YannDubs about 1 year ago

#44 - [CI] Make automatic deployment on pypi

Issue - State: closed - Opened by YannDubs about 1 year ago

#43 - [TEST] add tests + CI

Issue - State: closed - Opened by YannDubs about 1 year ago - 1 comment

#42 - Add Custom OpenAI API Endpoint Support and OpenChat Results

Pull Request - State: closed - Opened by imoneoi about 1 year ago - 1 comment

#41 - Add support for a anthropic 0.3

Issue - State: closed - Opened by YannDubs over 1 year ago

#40 - add ultralm results

Pull Request - State: closed - Opened by cyl628 over 1 year ago - 3 comments

#39 - llama 13b as evaluate_from_model does not use GPU

Issue - State: closed - Opened by RealTong over 1 year ago - 8 comments

#38 - Add support for python 3.9

Issue - State: closed - Opened by rtaori over 1 year ago - 2 comments

#37 - [ENH] add chatgpt leaderboard

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#36 - [ENH] openAI function API

Pull Request - State: closed - Opened by YannDubs over 1 year ago

#34 - Post-analysis of the annotations

Issue - State: closed - Opened by haorannlp over 1 year ago - 2 comments

#33 - Question about 805 eval examples

Issue - State: closed - Opened by nbl97 over 1 year ago - 1 comment

#32 - [Question] Alternative for GPT4 eval?

Issue - State: closed - Opened by simplelifetime over 1 year ago - 2 comments

#29 - Why evaluate_from_model run so slow on my side

Issue - State: closed - Opened by hanningzhang over 1 year ago - 7 comments

#11 - alpaca farm migration

Issue - State: closed - Opened by YannDubs over 1 year ago

GitHub / tatsu-lab/alpaca_eval issues and pull requests