leondz/garak issues and pull requests

#54 - probe: parl.ai Bot Adversarial Dialogue Dataset

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes

#53 - probe: microsoft examples

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#52 - probe: initial Language Model Risk Cards

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#51 - probe: avid

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#50 - risky words detectors

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment

#49 - probe: RealToxicityPrompts

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: enhancement, probes

#48 - probe: slur continuations

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#47 - API call volume mgmt

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#46 - generator: OpenAI chat (maybe a global "chat modality" option?)

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, generators

#45 - autoredteam

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: probes

#44 - on pypi, installable with pip as CLI tool

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, cli

#43 - probe for willingness to generate malicious code

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, probes, new plugin

#42 - more asserts

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, tests

#41 - cli option for verbose

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#40 - put module load exceptions & traces into log

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#39 - check if goodside threat actually needed

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, probes, quality-accuracy

#38 - snowball probes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#37 - presupposition insertion detection

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, detectors, new plugin

#36 - skip tqdm on one-call probings

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#35 - check the DAN outputs - are the detectors OK?

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, detectors

#34 - refutation/denial detector

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, detectors, new plugin

#33 - plugin cache / hf detector cache

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, detectors

#32 - cli option for quiet

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, cli

#31 - cli option to pass a yaml config

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#30 - lazier huggingface lib, object import

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: bug

#29 - probe/detector factory for simple string detectors

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes, detectors

#28 - leverage the anthropic resources

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes, detectors

#27 - approx scoring scale not good

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: bug

#26 - add a ListDetector that takes answers in order

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, detectors

#25 - probe: feddit xtzeth book

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#24 - probe: promptingguide adversarial

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#23 - probe: representational bias

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#22 - finer granularity probe/detector coupling

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

#21 - fact check probe

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#20 - bad word list probe & detector

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, probes

#19 - DAN probes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#18 - probe: promptinject (perez & ribeiro)

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#17 - generator: llama integration

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: enhancement, generators

#16 - openai integration

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#15 - new harnesses

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

#14 - per-detector and per-probe quantity override

Issue - State: open - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#13 - optional "gate" to confirm execution of many calls for api-based generators & detectors

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, generators

#12 - enable probing parallelisation

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#11 - tests, lint & black check etc in build

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: enhancement

#10 - docs

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: documentation, enhancement

#9 - make it possible to run scan remotely

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#8 - put a demo up in HF space

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#7 - better report generation

Issue - State: closed - Opened by leondz over 1 year ago - 4 comments
Labels: enhancement, reporting

#6 - better (any?) logging

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#5 - implement a buff

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, buffs

#4 - Add tqdm: for test generation, for per-detector run

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

#3 - Support supplying & using recommended detector(s) in probe class

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#2 - Implement command-line params

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#1 - Add decorator-based plugin & class enumeration for probes and detectors

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

GitHub / leondz/garak issues and pull requests