NVIDIA/garak issues and pull requests

#86 - add an `active` flag to allow marking plugins as inactive

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, detectors, generators

#86 - add an `active` flag to allow marking plugins as inactive

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, detectors, generators

#85 - config file

Issue - State: closed - Opened by leondz over 1 year ago - 4 comments
Labels: enhancement

#85 - config file

Issue - State: closed - Opened by leondz over 1 year ago - 4 comments
Labels: enhancement

#84 - remove Detector keyword from detector classes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, detectors

#84 - remove Detector keyword from detector classes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, detectors

#83 - set up a global "prompts-per-probe" cap

Issue - State: open - Opened by leondz over 1 year ago
Labels: cli

#83 - set up a global "prompts-per-probe" cap

Issue - State: open - Opened by leondz over 1 year ago
Labels: cli

#82 - unify & expose the "generation length" parameter

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: generators

#82 - unify & expose the "generation length" parameter

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: generators

#81 - detector: refutation

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, detectors

#81 - detector: refutation

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, detectors

#80 - detector: misogyny

Issue - State: open - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, detectors, new plugin

#80 - detector: misogyny

Issue - State: open - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, detectors, new plugin

#79 - probes should include a verbose "goal" attribute

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes

#79 - probes should include a verbose "goal" attribute

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes

#78 - evaluators should read from attempt objects

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#78 - evaluators should read from attempt objects

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#77 - provide multiple output modes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, wontfix, cli

#77 - provide multiple output modes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, wontfix, cli

#76 - probe: prompt revelation attacks

Issue - State: open - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#76 - probe: prompt revelation attacks

Issue - State: open - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#75 - probe: obfuscation, injection, virtualisation

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#75 - probe: obfuscation, injection, virtualisation

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#74 - probe: injection with tags

Issue - State: open - Opened by leondz over 1 year ago
Labels: good first issue

#74 - probe: injection with tags

Issue - State: open - Opened by leondz over 1 year ago
Labels: good first issue

#73 - probe: injection in whois

Issue - State: closed - Opened by leondz over 1 year ago
Labels: good first issue, probes, new plugin

#73 - probe: injection in whois

Issue - State: closed - Opened by leondz over 1 year ago
Labels: good first issue, probes, new plugin

#72 - probe: swyx techniques

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#72 - probe: swyx techniques

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#71 - probe: lmrc

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#71 - probe: lmrc

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#70 - probe: fake references

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#70 - probe: fake references

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#69 - probe: misleading questions

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: probes, new plugin

#69 - probe: misleading questions

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: probes, new plugin

#68 - throw exception if plugin invoked where key is missing

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#68 - throw exception if plugin invoked where key is missing

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#67 - detector: mitigations

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, good first issue, detectors, new plugin

#67 - detector: mitigations

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, good first issue, detectors, new plugin

#66 - print (probe|detector|generator) description at the side of the class name when listing

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#66 - print (probe|detector|generator) description at the side of the class name when listing

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#65 - sync testing expectations with what --list* delivers

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#65 - sync testing expectations with what --list* delivers

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#64 - api call parallelism

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: duplicate, enhancement

#64 - api call parallelism

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: duplicate, enhancement

#63 - add --detectors CLI option

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, detectors, cli

#63 - add --detectors CLI option

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, detectors, cli

#62 - hit log

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

#62 - hit log

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement

#61 - add typing hints around core functions

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#61 - add typing hints around core functions

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement

#60 - generator: llama.cpp integration

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, generators

#60 - generator: llama.cpp integration

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, generators

#59 - log probe params at probe invocation

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, probes, reporting

#59 - log probe params at probe invocation

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, probes, reporting

#58 - log generator params at invocation

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, generators, reporting

#58 - log generator params at invocation

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, generators, reporting

#57 - add Replicate generator

Pull Request - State: closed - Opened by leondz over 1 year ago

#57 - add Replicate generator

Pull Request - State: closed - Opened by leondz over 1 year ago

#56 - add n-gram approx matching, default 4-gram

Pull Request - State: closed - Opened by leondz over 1 year ago

#56 - add n-gram approx matching, default 4-gram

Pull Request - State: closed - Opened by leondz over 1 year ago

#55 - Dir layout

Pull Request - State: closed - Opened by leondz over 1 year ago

#55 - Dir layout

Pull Request - State: closed - Opened by leondz over 1 year ago

#54 - probe: parl.ai Bot Adversarial Dialogue Dataset

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes

#54 - probe: parl.ai Bot Adversarial Dialogue Dataset

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes

#53 - probe: microsoft examples

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#53 - probe: microsoft examples

Issue - State: open - Opened by leondz over 1 year ago
Labels: probes, new plugin

#52 - probe: initial Language Model Risk Cards

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#52 - probe: initial Language Model Risk Cards

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#51 - probe: avid

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: wontfix, probes, new plugin

#51 - probe: avid

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: wontfix, probes, new plugin

#50 - risky words detectors

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment

#50 - risky words detectors

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment

#49 - probe: RealToxicityPrompts

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: enhancement, probes

#49 - probe: RealToxicityPrompts

Issue - State: closed - Opened by leondz over 1 year ago - 3 comments
Labels: enhancement, probes

#48 - probe: slur continuations

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#48 - probe: slur continuations

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, new plugin

#47 - API call volume mgmt

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#47 - API call volume mgmt

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#46 - generator: OpenAI chat (maybe a global "chat modality" option?)

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, generators

#46 - generator: OpenAI chat (maybe a global "chat modality" option?)

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, generators

#45 - autoredteam

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: probes

#45 - autoredteam

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: probes

#44 - on pypi, installable with pip as CLI tool

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, cli

#44 - on pypi, installable with pip as CLI tool

Issue - State: closed - Opened by leondz over 1 year ago - 2 comments
Labels: enhancement, cli

#43 - probe for willingness to generate malicious code

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, probes, new plugin

#43 - probe for willingness to generate malicious code

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: good first issue, probes, new plugin

#42 - more asserts

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, tests

#42 - more asserts

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, tests

#41 - cli option for verbose

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#41 - cli option for verbose

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, cli

#40 - put module load exceptions & traces into log

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#40 - put module load exceptions & traces into log

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement

#39 - check if goodside threat actually needed

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, probes, quality-accuracy

#39 - check if goodside threat actually needed

Issue - State: open - Opened by leondz over 1 year ago
Labels: enhancement, probes, quality-accuracy

#38 - snowball probes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#38 - snowball probes

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: enhancement, probes

#37 - presupposition insertion detection

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, detectors, new plugin

#37 - presupposition insertion detection

Issue - State: closed - Opened by leondz over 1 year ago - 1 comment
Labels: probes, detectors, new plugin

GitHub / NVIDIA/garak issues and pull requests