Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / NVIDIA/garak issues and pull requests

#1085 - pedantic spelling

Pull Request - State: open - Opened by dltemple 4 days ago - 3 comments

#1084 - detector: StringDetector matchtype `fullmatch`

Issue - State: open - Opened by leondz 6 days ago
Labels: detectors, new plugin

#1083 - update refusal prompt

Pull Request - State: open - Opened by katherine-luna 7 days ago - 4 comments

#1081 - guard against `None` in `leakreplay` attempt history management

Pull Request - State: closed - Opened by leondz 8 days ago
Labels: bug, probes

#1080 - Add caching to GH actions environment for HF items

Issue - State: open - Opened by leondz 8 days ago
Labels: housekeeping

#1079 - Add dry run option that includes stats

Issue - State: open - Opened by leondz 8 days ago
Labels: architecture, generators

#1078 - Output estimated run cost at top of run

Issue - State: open - Opened by leondz 8 days ago
Labels: architecture

#1077 - Order prompts within probes for max bag attack success rate

Issue - State: open - Opened by leondz 8 days ago
Labels: architecture, probes

#1076 - feature: align hallucinated package named with outputs

Pull Request - State: open - Opened by leondz 8 days ago
Labels: detectors

#1075 - detector: add `startswith` matching for `StringDetector` base class

Pull Request - State: closed - Opened by leondz 8 days ago
Labels: detectors

#1074 - Feature: configuration based rest ssl suppression

Pull Request - State: closed - Opened by jmartin-tech 10 days ago

#1073 - Feature: configuration based rest proxy support

Pull Request - State: closed - Opened by jmartin-tech 10 days ago - 1 comment

#1072 - `StringDetector.case_sensitive` now operates as expected

Pull Request - State: closed - Opened by leondz 10 days ago - 1 comment
Labels: bug, detectors

#1071 - Estimate # prompts, # input tokens per probe

Issue - State: open - Opened by leondz 12 days ago
Labels: architecture

#1070 - Enable stopping when call (/token) budget hit

Issue - State: open - Opened by leondz 12 days ago
Labels: architecture

#1069 - Automate updating the calibration process

Issue - State: open - Opened by leondz 13 days ago
Labels: housekeeping

#1068 - Harness that orders probes for expected attack success rate (ASR) according to bag

Issue - State: open - Opened by leondz 13 days ago - 2 comments
Labels: architecture

#1067 - Add “real” multi-turn support to chat-enabled generators

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, generators

#1066 - probe: Adapt sandwich attack to auto-find effective languages

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1066 - probe: Adapt sandwich attack to auto-find effective languages

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1065 - Bayesian optimisation base class

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1065 - Bayesian optimisation base class

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1064 - Pattern for probes that can react to generator/detector responses

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1064 - Pattern for probes that can react to generator/detector responses

Issue - State: open - Opened by leondz 13 days ago
Labels: architecture, probes

#1063 - docs: model type and name mixed up

Issue - State: open - Opened by dennisrall 13 days ago
Labels: documentation

#1063 - docs: model type and name mixed up

Issue - State: open - Opened by dennisrall 13 days ago - 1 comment
Labels: documentation

#1062 - support for python 3.13

Issue - State: open - Opened by leondz 15 days ago
Labels: architecture

#1062 - support for python 3.13

Issue - State: open - Opened by leondz 15 days ago - 2 comments
Labels: architecture

#1061 - Include information about why detectors are skipped

Issue - State: open - Opened by erickgalinkin 16 days ago
Labels: detectors, reporting, ux, quality-accuracy

#1060 - Additional XSS Exfil Probes

Pull Request - State: open - Opened by erickgalinkin 17 days ago
Labels: probes

#1059 - Detector Shields for testing LLM Application Firewalls

Pull Request - State: open - Opened by Eric-Hacker 17 days ago - 8 comments

#1059 - Detector Shields for testing LLM Application Firewalls

Pull Request - State: closed - Opened by Eric-Hacker 17 days ago - 8 comments

#1058 - Added watsonx.ai generator

Pull Request - State: open - Opened by iamnotcj 19 days ago - 4 comments

#1057 - adjust label ref syntax

Pull Request - State: closed - Opened by jmartin-tech 19 days ago

#1056 - probe: text augmentations

Issue - State: open - Opened by rafaelsandroni 20 days ago

#1054 - StringDetector case sensitivity seems inverted.

Issue - State: closed - Opened by Eric-Hacker 21 days ago - 4 comments
Labels: bug

#1053 - docs: update index.rst

Pull Request - State: closed - Opened by eltociear 23 days ago - 3 comments

#1051 - Added generator for OpenRouter.ai

Pull Request - State: open - Opened by pedramamini about 1 month ago - 8 comments

#1050 - docs: update readme

Pull Request - State: closed - Opened by leondz about 1 month ago
Labels: documentation

#1049 - Warn if api_key in Config

Pull Request - State: closed - Opened by erickgalinkin about 1 month ago - 1 comment

#1047 - Feature: use hf chat support

Pull Request - State: closed - Opened by jmartin-tech about 1 month ago

#1046 - Use HF safetensors by default

Pull Request - State: closed - Opened by leondz about 1 month ago - 1 comment

#1045 - Fix tap.PAIR run error - run_tap param evaluator_model

Pull Request - State: closed - Opened by rafaelsandroni about 1 month ago - 3 comments

#1044 - tap.PAIR getting "run_tap() got an unexpected keyword argument 'evaluator_model'"

Issue - State: closed - Opened by rafaelsandroni about 1 month ago - 1 comment
Labels: bug

#1043 - Feature/autoeval

Pull Request - State: closed - Opened by erickgalinkin about 1 month ago

#1042 - pass device to transfomers pipeline

Pull Request - State: closed - Opened by jmartin-tech about 1 month ago

#1042 - pass device to transfomers pipeline

Pull Request - State: closed - Opened by jmartin-tech about 1 month ago

#1041 - FEAT Add a custom REST LLM API template to Garak Config yaml file

Issue - State: open - Opened by EricXQiu about 1 month ago - 1 comment
Labels: question

#1041 - FEAT Add a custom REST LLM API template to Garak Config yaml file

Issue - State: closed - Opened by EricXQiu about 1 month ago - 2 comments
Labels: question

#1039 - Restrict transformers version until MPS issue is addressed

Pull Request - State: closed - Opened by jmartin-tech about 2 months ago - 1 comment

#1039 - Restrict transformers version until MPS issue is addressed

Pull Request - State: closed - Opened by jmartin-tech about 2 months ago - 1 comment

#1038 - generator feature: add deprefix prefix

Issue - State: closed - Opened by leondz about 2 months ago - 2 comments
Labels: generators

#1037 - CLA: determine if commit can be added to the local PR branch

Issue - State: closed - Opened by leondz about 2 months ago - 1 comment
Labels: housekeeping

#1036 - package garak for kali

Issue - State: open - Opened by leondz about 2 months ago
Labels: housekeeping

#1036 - package garak for kali

Issue - State: open - Opened by leondz about 2 months ago - 3 comments
Labels: housekeeping

#1035 - error: 'latin-1' codec can't encode character '\u2026'

Issue - State: open - Opened by wiktorw767 about 2 months ago - 5 comments
Labels: bug

#1034 - probes: fileformats probe fails when targeting a local modal path

Issue - State: open - Opened by jmartin-tech about 2 months ago
Labels: bug

#1034 - probes: fileformats probe fails when targeting a local modal path

Issue - State: open - Opened by jmartin-tech about 2 months ago
Labels: bug

#1033 - Fall back when online services can't be reached.

Issue - State: open - Opened by erickgalinkin about 2 months ago - 1 comment
Labels: bug

#1033 - Fall back when online services can't be reached.

Issue - State: open - Opened by erickgalinkin about 2 months ago - 1 comment
Labels: bug

#1032 - align prompt count per probe

Issue - State: open - Opened by leondz about 2 months ago
Labels: architecture, probes

#1032 - align prompt count per probe

Issue - State: open - Opened by leondz about 2 months ago
Labels: architecture, probes

#1031 - per-probe tags now adjustable based on payload selection

Pull Request - State: closed - Opened by leondz about 2 months ago
Labels: bug

#1031 - per-probe tags now adjustable based on payload selection

Pull Request - State: closed - Opened by leondz about 2 months ago
Labels: bug

#1030 - docs: add arxiv lozenge

Pull Request - State: closed - Opened by leondz about 2 months ago
Labels: documentation

#1030 - docs: add arxiv lozenge

Pull Request - State: closed - Opened by leondz about 2 months ago
Labels: documentation

#1029 - Allow applying prompt template to the input for huggingface models.

Issue - State: closed - Opened by zijiachen95 about 2 months ago - 2 comments

#1027 - refactor attempt to utilized property annotations

Pull Request - State: closed - Opened by jmartin-tech about 2 months ago

#1026 - Feature: enable configuration updates

Pull Request - State: closed - Opened by jmartin-tech about 2 months ago - 2 comments

#1025 - probe: ANSI terminal takeover

Pull Request - State: closed - Opened by leondz about 2 months ago - 1 comment
Labels: probes, detectors, new plugin

#1023 - probe: Image Hijack

Issue - State: open - Opened by leondz about 2 months ago
Labels: probes, new plugin

#1022 - fix: correct argument in TAP

Pull Request - State: closed - Opened by harshraj172 2 months ago - 9 comments

#1021 - generator: promote OpenAICompatible as first class generator

Pull Request - State: closed - Opened by jmartin-tech 2 months ago - 1 comment

#1020 - probe: gritty pixy

Issue - State: closed - Opened by leondz 2 months ago - 3 comments
Labels: probes, new plugin

#1019 - compatible with oobabooga text-generation-web-ui

Pull Request - State: closed - Opened by cycloarcane 2 months ago - 5 comments

#1018 - probe: add Arabic DAN

Pull Request - State: open - Opened by Eaalghamdi 2 months ago - 5 comments

#1017 - probe: add Arabic donotanswer

Pull Request - State: open - Opened by Eaalghamdi 2 months ago - 3 comments

#1008 - [Feature Request] Support any OpenAI compatible endpoints by adding two flags

Issue - State: closed - Opened by regunakyle 2 months ago - 6 comments
Labels: architecture, generators

#1003 - feature: add experimental features flag to be only accessible in core config

Pull Request - State: closed - Opened by leondz 2 months ago - 2 comments
Labels: architecture, cli

#997 - probe: ANSI/Stök

Issue - State: closed - Opened by leondz 2 months ago - 2 comments
Labels: probes, new plugin

#996 - probe: more xss exfil probes

Issue - State: open - Opened by leondz 2 months ago
Labels: probes

#995 - feature: Refactor attempt to not override getters & setters

Issue - State: closed - Opened by leondz 2 months ago
Labels: architecture

#994 - atkgen: disinfo probe

Issue - State: open - Opened by leondz 2 months ago
Labels: probes

#993 - atkgen: illegal content probe

Issue - State: open - Opened by leondz 2 months ago
Labels: probes

#992 - detector: Use LLMaaJ to see if models are following payload instructions

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture, detectors

#992 - detector: Use LLMaaJ to see if models are following payload instructions

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture, detectors

#990 - feature: Enable `use_safetensors/True` for HF models and expose as config var

Issue - State: closed - Opened by leondz 2 months ago - 1 comment
Labels: architecture

#989 - feature: Block pickle loading unless garak’s configured to do unsafe things

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture

#988 - feature: add plugin file date/hash to plugin load messages & logs for auditing

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture

#988 - feature: add plugin file date/hash to plugin load messages & logs for auditing

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture

#987 - test: run tests automatically at fixed time intervals, to catch shifts in dependencies

Issue - State: open - Opened by leondz 2 months ago
Labels: tests, housekeeping

#985 - feature: support audit trails, where asset IDs can be plugged in

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture

#983 - feature: Enable skipping of tests for policy points target is permissive over

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture, policy

#982 - feature: report target’s apparent policy & match to a predefined policies

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture, policy

#981 - atkgen: flexible probe

Issue - State: open - Opened by leondz 2 months ago
Labels: probes

#980 - atkgen: Establish pipeline toolkit from target (represented as a detector) to trained model

Issue - State: open - Opened by leondz 2 months ago
Labels: architecture, probes