Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / eleutherai/elk issues and pull requests

#92 - Migrate CLI to `simple_parsing`, allowing for YAML based configs

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#91 - Add Python 3.11 to CI tests

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#90 - Add Pyright type checking to GitHub CI

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#89 - Feature eval

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#86 - Train reporters for different layers in parallel

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#85 - Move tests outside the module

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#84 - Add few-shot prompt support; small PromptCollator improvements

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#83 - Remove redundant 'elk train' command

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#82 - (WIP) Use torch.multiprocessing to parallelize probe training across layers

Pull Request - State: closed - Opened by anshradh almost 2 years ago - 2 comments

#81 - Weights & Biases integration

Issue - State: closed - Opened by norabelrose almost 2 years ago - 2 comments
Labels: enhancement

#80 - Docstrings

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#79 - docstrings initial

Pull Request - State: closed - Opened by mariopenglee almost 2 years ago - 1 comment

#78 - Add "elk list" command

Pull Request - State: closed - Opened by norabelrose almost 2 years ago - 1 comment

#77 - Create an "elk list" command to show info about all cached runs

Issue - State: closed - Opened by norabelrose almost 2 years ago
Labels: enhancement

#76 - Evaluate models' zero-shot accuracy

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement

#75 - Support few-shot prompts

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement

#74 - Feature sweep

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#73 - fixed max_example_bug

Pull Request - State: closed - Opened by EricMungai97 almost 2 years ago

#72 - Add automatic type checking to the linting pipeline

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement, good first issue

#71 - Add docstrings to all user-facing APIs

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: documentation, good first issue

#70 - Support Neel Nanda's counterfact dataset

Issue - State: open - Opened by norabelrose almost 2 years ago - 3 comments
Labels: enhancement

#69 - Story cloze doesn't work anymore

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 4 comments
Labels: bug, good first issue

#68 - Error when max examples larger than examples in ds

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 1 comment
Labels: bug, good first issue

#67 - Plotting for individual predictions across depth

Issue - State: open - Opened by norabelrose almost 2 years ago
Labels: enhancement

#66 - Rename CCS -> Reporter

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#65 - Rename "CCS" class to "Reporter"

Issue - State: closed - Opened by norabelrose almost 2 years ago
Labels: refactor

#64 - Parallelize probe training across layers

Issue - State: closed - Opened by norabelrose almost 2 years ago - 3 comments
Labels: enhancement, good first issue

#62 - Object-oriented loss API

Issue - State: closed - Opened by norabelrose almost 2 years ago - 2 comments
Labels: refactor

#61 - Migrate to HF datasets for hidden state storage

Issue - State: closed - Opened by norabelrose almost 2 years ago - 2 comments
Labels: bug, enhancement, refactor

#60 - Layer ensembles

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement

#59 - Use one file per layer

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago - 1 comment

#58 - Initial implementation of semi-supervised training

Pull Request - State: closed - Opened by norabelrose almost 2 years ago - 2 comments

#57 - Add recursive CCS

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago

#56 - Feature balance

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#55 - Remove TrainParams

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#54 - Support semi-supervised probe training

Issue - State: closed - Opened by norabelrose almost 2 years ago

#53 - Configure GitHub Actions CI For CPU Tests

Pull Request - State: closed - Opened by Kyle1668 almost 2 years ago - 1 comment

#52 - Make extract_hiddens usable programmatically

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#51 - Update README.md

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#50 - Write new eval script for dataset-to-dataset transfer

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement, good first issue

#49 - Remove vestigial eval code

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#48 - Create Jupyter notebook-friendly API

Issue - State: closed - Opened by norabelrose almost 2 years ago

#47 - Add CLI option for skipping prompts that are too long (instead of truncating)

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement, good first issue

#46 - Add option to balance datasets as in original code of the DLK paper

Issue - State: closed - Opened by lauritowal almost 2 years ago - 1 comment

#45 - Move changes over from mother-of-all-refactors

Pull Request - State: closed - Opened by norabelrose almost 2 years ago

#44 - Mother of all refactors

Pull Request - State: closed - Opened by norabelrose almost 2 years ago - 1 comment

#43 - Not able to reproduce Figure1

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 11 comments
Labels: bug

#42 - Add custom run name for train & eval

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago

#41 - [pre-commit.ci] pre-commit autoupdate

Pull Request - State: closed - Opened by pre-commit-ci[bot] almost 2 years ago

#40 - Creating results.csv from Appendix E in DLK paper

Issue - State: closed - Opened by lauritowal almost 2 years ago

#39 - Create GitHub Actions CI Pipeline for CPU Tests

Issue - State: closed - Opened by Kyle1668 almost 2 years ago

#38 - Enforce Dashes in CLI Argument Names

Pull Request - State: closed - Opened by Kyle1668 almost 2 years ago

#37 - Update README.md

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#35 - Use better filenames for the pkls so they don't get overwritten

Issue - State: closed - Opened by QuintinPope almost 2 years ago - 1 comment

#34 - Support train & eval on multiple datasets at once

Issue - State: closed - Opened by norabelrose almost 2 years ago - 1 comment
Labels: enhancement

#31 - Support all HF models

Pull Request - State: closed - Opened by neverix almost 2 years ago - 2 comments

#30 - Use train data for computing normalization

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago - 4 comments

#29 - Add an informative error message for story-cloze

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago

#28 - Add RecursiveCCS

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago

#27 - RecursiveCCS

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 1 comment
Labels: enhancement

#26 - Rename "generation" to "extraction" of hidden states

Issue - State: closed - Opened by norabelrose almost 2 years ago

#25 - Stop using a separate `cache_dir` to save pretrained models

Issue - State: closed - Opened by norabelrose almost 2 years ago

#24 - Cleaning up / Simplifying Generation of hidden states

Issue - State: closed - Opened by lauritowal almost 2 years ago

#23 - Task set random seed

Pull Request - State: closed - Opened by lauritowal almost 2 years ago - 1 comment

#22 - fix format of string for promptsource dependency

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#21 - Support cross-validation for selecting the best weight decay / L2 penalty

Issue - State: closed - Opened by norabelrose almost 2 years ago - 2 comments

#20 - Update README.md

Pull Request - State: closed - Opened by lauritowal almost 2 years ago

#19 - implement test cases for reproducing the results

Pull Request - State: closed - Opened by mariopenglee almost 2 years ago

#18 - Make CCS a module

Issue - State: closed - Opened by norabelrose almost 2 years ago - 3 comments

#17 - Support custom loss functions

Issue - State: closed - Opened by norabelrose almost 2 years ago - 4 comments

#16 - Add continuous integration

Issue - State: closed - Opened by norabelrose almost 2 years ago - 2 comments

#15 - Change CLI args to use dashes, not underscores

Issue - State: closed - Opened by norabelrose almost 2 years ago

#14 - Story_cloze fails

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 6 comments
Labels: bug

#13 - Make CCS use any kind of probes

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago - 3 comments

#12 - Add Lbfgs

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago

#11 - Fix misc bugs and issues

Pull Request - State: closed - Opened by FabienRoger almost 2 years ago - 2 comments

#10 - Support arbitrary Huggingface autoregressive models

Issue - State: closed - Opened by norabelrose almost 2 years ago

#9 - Turn the extraction and evaluate scripts into CLI commands

Issue - State: closed - Opened by norabelrose almost 2 years ago

#8 - Add typing

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 1 comment

#7 - Enable more general probes

Issue - State: closed - Opened by FabienRoger almost 2 years ago

#6 - Enable multi-class classification

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 2 comments
Labels: enhancement

#5 - Normalization should use the training data to compute means and stds

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 2 comments

#4 - Add tests

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 2 comments

#3 - Update the readme with installation & dev instructions

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 2 comments

#2 - Fix docstrings

Issue - State: closed - Opened by FabienRoger almost 2 years ago - 1 comment

#1 - Enable LBFGS

Issue - State: closed - Opened by FabienRoger almost 2 years ago