NVIDIA/garak issues and pull requests

#975 - detector: non-engagement

Issue - State: open - Opened by leondz 3 months ago
Labels: detectors, new plugin

#968 - docs: clarify "contributing" document

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: documentation

#967 - doc: update to use rst toctree href syntax

Pull Request - State: closed - Opened by leondz 3 months ago

#966 - docs: require core config top- and second-level params to be documented

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: documentation

#955 - experimental feature: policy scan base infrastructure

Pull Request - State: open - Opened by leondz 3 months ago
Labels: architecture

#955 - experimental feature: policy scan base infrastructure

Pull Request - State: open - Opened by leondz 3 months ago
Labels: architecture

#952 - REST URI client error: 422

Issue - State: closed - Opened by samfhritp 3 months ago - 3 comments
Labels: question

#950 - Package hallucination

Pull Request - State: open - Opened by arjun-krishna1 3 months ago - 3 comments

#948 - Probe&Detector/SciSafeEval: The State-of-the-Art Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

Pull Request - State: open - Opened by DavidLee528 3 months ago - 10 comments

#944 - probe: autodan-turbo

Issue - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#943 - Feature/multilingual

Pull Request - State: open - Opened by SnowMasaya 4 months ago - 4 comments

#943 - Feature/multilingual

Pull Request - State: open - Opened by SnowMasaya 4 months ago - 4 comments

#938 - probe: missing newline in `latentinjection.LatentInjectionFactSnippetEiffel`

Pull Request - State: closed - Opened by leondz 4 months ago

#936 - reporting: tidy config details, add payload audit info

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: reporting

#927 - security: warn if `api_key` given in config and config is world-readable

Issue - State: closed - Opened by leondz 4 months ago
Labels: architecture, cli

#924 - probe: Past Tense Vulnerability

Pull Request - State: open - Opened by Shine-afk 4 months ago - 11 comments

#924 - probe: Past Tense Vulnerability

Pull Request - State: closed - Opened by Shine-afk 4 months ago - 11 comments

#889 - add support for loading renamed plugins

Issue - State: closed - Opened by leondz 5 months ago - 3 comments
Labels: architecture

#879 - Bug in Leakreplay Probe

Issue - State: closed - Opened by bleszily 5 months ago - 3 comments
Labels: bug, probes

#878 - Add proxy, token refresh, and multi_endpoint features to generators

Pull Request - State: open - Opened by au70ma70n 5 months ago - 6 comments

#878 - Add proxy, token refresh, and multi_endpoint features to generators

Pull Request - State: open - Opened by au70ma70n 5 months ago - 6 comments

#853 - checking out branch w/ new plugin class updated skips cache rebuild, making class inaccessible

Issue - State: open - Opened by leondz 5 months ago - 2 comments
Labels: bug

#841 - probe: pdf injection

Issue - State: open - Opened by leondz 5 months ago
Labels: probes, new plugin

#839 - calibration symlink broken

Issue - State: open - Opened by leondz 5 months ago - 4 comments
Labels: bug, windows

#823 - HF Detector loading process sometimes times out, then crashes the run

Issue - State: open - Opened by leondz 5 months ago
Labels: bug, detectors, ux

#805 - leaked semaphore objects

Issue - State: open - Opened by leondz 6 months ago - 1 comment
Labels: bug, cli, ux

#804 - probes that fail to elicit valid generator output should result in SKIP

Issue - State: open - Opened by leondz 6 months ago - 1 comment
Labels: bug, ux, quality-accuracy

#791 - LRLBuff manipulates attempt.outputs

Issue - State: open - Opened by leondz 6 months ago
Labels: bug, buffs

#783 - probe: sandwich probe

Pull Request - State: open - Opened by leondz 7 months ago
Labels: probes, new plugin

#758 - probe: Explicit: forced instruction-following.

Issue - State: open - Opened by leondz 7 months ago
Labels: probes, new plugin

#757 - Add Skeleton Key probe

Issue - State: open - Opened by erickgalinkin 7 months ago - 1 comment
Labels: probes, new plugin

#752 - probe: VisualWebArena-Adv

Issue - State: open - Opened by leondz 7 months ago
Labels: probes, new plugin

#750 - atkgen forking HF tokenizer despite being non-parallel

Issue - State: open - Opened by leondz 7 months ago
Labels: bug, probes, quality-speed

#746 - probe: chatgpt pii exfil

Issue - State: open - Opened by leondz 7 months ago
Labels: probes, new plugin

#734 - probe: disguise & reconstruct

Issue - State: open - Opened by leondz 8 months ago
Labels: probes, new plugin

#715 - generators: move read timeout up to global/`base.Generator` config item

Issue - State: open - Opened by leondz 8 months ago - 1 comment
Labels: generators

#704 - probe or buff: non-standard unicode

Issue - State: open - Opened by leondz 8 months ago
Labels: probes, new plugin, buffs

#700 - probe: polyglo toxicity prompts

Issue - State: open - Opened by leondz 8 months ago
Labels: architecture, probes, new plugin

#692 - generators: openrouter

Issue - State: open - Opened by leondz 8 months ago - 1 comment
Labels: generators, new plugin

#683 - probe: persuasive jailbreak

Issue - State: open - Opened by leondz 8 months ago
Labels: probes, new plugin

#666 - generator: sensenova

Issue - State: open - Opened by beike2020 9 months ago - 7 comments
Labels: generators, new plugin

#646 - `_config` object contains `state` not suitable for multiprocessing

Issue - State: open - Opened by jmartin-tech 9 months ago - 3 comments
Labels: bug, architecture

#602 - Enable more complex `prompts`

Issue - State: open - Opened by jmartin-tech 10 months ago - 1 comment
Labels: architecture

#593 - tests: test garak's loading performance

Issue - State: open - Opened by leondz 10 months ago - 1 comment
Labels: architecture, tests, quality-speed

#583 - probe: simple adaptive attacks

Issue - State: open - Opened by leondz 10 months ago
Labels: good first issue, probes, new plugin

#582 - probe: giving examples

Issue - State: open - Opened by leondz 10 months ago
Labels: probes, new plugin

#579 - probe: separation

Issue - State: open - Opened by leondz 10 months ago
Labels: probes, new plugin

#570 - probe: token-length side-channel

Issue - State: open - Opened by leondz 10 months ago
Labels: probes, new plugin

#549 - detector: classifier-based mitigation detection

Issue - State: open - Opened by leondz 11 months ago
Labels: detectors, new plugin

#548 - probe: simplesafetytests

Issue - State: open - Opened by leondz 11 months ago
Labels: probes, new plugin

#547 - probe: textattack

Issue - State: open - Opened by leondz 11 months ago
Labels: probes, new plugin

#535 - buff: ascii art

Issue - State: open - Opened by leondz 11 months ago
Labels: probes, new plugin

#531 - base probe should use primary/extended detector and not recommended detector

Issue - State: open - Opened by leondz 11 months ago
Labels: bug, architecture, probes

#529 - Add PAP Attack

Issue - State: open - Opened by erickgalinkin 11 months ago
Labels: probes, new plugin

#524 - base prompt attempts inherit buff notes

Issue - State: open - Opened by leondz 11 months ago
Labels: bug, buffs

#512 - RANTI needs a payload

Issue - State: open - Opened by leondz 11 months ago
Labels: bug, probes, quality-accuracy

#482 - bug: unhappy torch detectors should fail gracefully

Issue - State: open - Opened by leondz 11 months ago - 1 comment
Labels: bug, detectors

#473 - test order affects test success/speed. it shouldn't

Issue - State: open - Opened by leondz 12 months ago
Labels: bug, tests

#407 - `--parallel_attempts` on osx causes crash around file writing

Issue - State: open - Opened by leondz about 1 year ago - 2 comments
Labels: bug, mac

#361 - fail more helpfully when parallelism not supported

Issue - State: open - Opened by leondz about 1 year ago
Labels: bug, ux

#358 - Relax plugin-specific requirements

Issue - State: open - Opened by leondz about 1 year ago
Labels: bug, architecture

#335 - -v counting broke

Issue - State: open - Opened by leondz about 1 year ago - 1 comment
Labels: bug

#324 - unhide probe name during detection run

Issue - State: open - Opened by leondz about 1 year ago - 1 comment
Labels: bug, ux

#320 - REST Generator and response code 4xx

Issue - State: open - Opened by Eric-Hacker about 1 year ago - 2 comments
Labels: bug, architecture

#319 - Detector bug from REST generator

Issue - State: open - Opened by Eric-Hacker about 1 year ago - 2 comments
Labels: bug

#314 - seed warnings with nemo generator if parallel_attempts > 1

Issue - State: open - Opened by leondz about 1 year ago
Labels: bug, generators

#308 - fix flickering tqdm w/ parallel_attempts >1

Issue - State: open - Opened by leondz about 1 year ago
Labels: bug, ux

#277 - test probe attempts for null target

Issue - State: open - Opened by leondz over 1 year ago
Labels: bug, probes