Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / leondz/garak issues and pull requests

#786 - chatscanner integration live chatbot

Issue - State: open - Opened by priamai 2 months ago

#784 - Does REST generator support multiple end point and responses?

Issue - State: open - Opened by sumukh5 2 months ago - 2 comments

#783 - probe: sandwich probe

Pull Request - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#782 - defensive coding: allow python generators more places

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: enhancement

#781 - don't skip listing probes in 100%-pass-rate modules

Pull Request - State: open - Opened by leondz 3 months ago
Labels: reporting

#780 - stop encoding.payloads accumulating each load

Pull Request - State: open - Opened by leondz 3 months ago
Labels: bug, probes

#779 - move REST generator exception into garak.exceptions

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: enhancement, generators

#778 - always log nvcf payload

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: generators

#777 - atkgen - add configurability, allow config of which adversarial generator to use

Pull Request - State: open - Opened by leondz 3 months ago
Labels: generators

#776 - generator: lorem ipsum

Pull Request - State: open - Opened by leondz 3 months ago
Labels: generators

#775 - enhancement: enable lazy setting of nested dicts

Pull Request - State: open - Opened by leondz 3 months ago - 2 comments
Labels: enhancement

#774 - Revert "enable nested setting of plugin configs"

Pull Request - State: closed - Opened by leondz 3 months ago

#773 - enable nested setting of plugin configs

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: enhancement

#772 - update NVCF to support versioning, custom payload keys

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: generators

#771 - add DEFAULT_PARAMS to base types that are configurables

Pull Request - State: closed - Opened by leondz 3 months ago

#770 - reduce zalgo-ness

Pull Request - State: closed - Opened by leondz 3 months ago

#769 - add colour theme config

Pull Request - State: closed - Opened by leondz 3 months ago

#768 - cached plugin enum

Pull Request - State: open - Opened by jmartin-tech 3 months ago - 1 comment
Labels: quality-speed

#767 - probe: assess 🤗 repos for potentially malicious files

Pull Request - State: open - Opened by leondz 3 months ago - 1 comment
Labels: probes

#764 - probe: topic pushing

Pull Request - State: open - Opened by leondz 3 months ago
Labels: probes

#763 - add grandma substance synth probe, update docs and var names

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: probes

#762 - add mitigation strings, add test for string-based mitigation skipping

Pull Request - State: open - Opened by leondz 3 months ago
Labels: detectors

#761 - retry openaicompatible requests if invalid content received

Pull Request - State: closed - Opened by leondz 3 months ago - 1 comment
Labels: bug, generators

#760 - AssertionError: _call_model must return a list

Issue - State: closed - Opened by brunomcuesta 3 months ago - 5 comments
Labels: bug

#759 - rename class level probe doc_uri

Pull Request - State: closed - Opened by jmartin-tech 3 months ago

#758 - probe: Explicit: forced instruction-following.

Issue - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#757 - Add Skeleton Key probe

Issue - State: open - Opened by erickgalinkin 3 months ago - 1 comment
Labels: probes, new plugin

#756 - align detector output types & results

Issue - State: open - Opened by leondz 3 months ago
Labels: enhancement, detectors

#755 - `litellm` generator input validation breaks on default cases

Issue - State: open - Opened by leondz 3 months ago
Labels: bug, generators

#754 - probe: assess 🤗 repos for potentially malicious files

Pull Request - State: closed - Opened by leondz 3 months ago - 2 comments
Labels: probes, new plugin

#753 - only download HF models when generation is required

Issue - State: open - Opened by leondz 3 months ago
Labels: enhancement, generators, quality-speed

#752 - probe: VisualWebArena-Adv

Issue - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#751 - log the config of each plugin at load time

Issue - State: open - Opened by leondz 3 months ago
Labels: enhancement

#750 - atkgen forking HF tokenizer despite being non-parallel

Issue - State: open - Opened by leondz 3 months ago
Labels: bug, probes

#749 - remove extraneous default params for nims that expect conservative pa…

Pull Request - State: closed - Opened by leondz 3 months ago - 1 comment
Labels: bug, generators

#748 - nim openai endpoints reject requests with unexpected params in payload

Issue - State: closed - Opened by leondz 3 months ago - 1 comment
Labels: bug, generators

#747 - InjectZalgo sometimes has very high prompt counts

Issue - State: open - Opened by leondz 3 months ago
Labels: bug

#746 - probe: chatgpt pii exfil

Issue - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#745 - feature: generator temp / random seed shuffle

Issue - State: open - Opened by leondz 3 months ago
Labels: enhancement, generators

#744 - Handle openai 400 errors more gracefully

Pull Request - State: closed - Opened by leondz 3 months ago

#743 - return value from `batch_decode` is already a list

Pull Request - State: closed - Opened by jmartin-tech 3 months ago

#742 - Refactor `huggingface` config support

Pull Request - State: closed - Opened by jmartin-tech 3 months ago

#741 - only set generations if still default

Pull Request - State: closed - Opened by jmartin-tech 3 months ago

#740 - add probe for "grandma" attack

Pull Request - State: closed - Opened by leondz 3 months ago - 1 comment

#739 - add garak preprint paper

Pull Request - State: closed - Opened by leondz 3 months ago

#738 - require probes to match input modality of the generator

Pull Request - State: closed - Opened by jmartin-tech 3 months ago

#737 - update `glitch` plugin

Issue - State: open - Opened by leondz 3 months ago
Labels: probes

#736 - Target LM Studio

Issue - State: closed - Opened by osok 3 months ago - 1 comment

#735 - add 'complete' message to report at end of run

Pull Request - State: closed - Opened by leondz 3 months ago
Labels: reporting

#734 - probe: disguise & reconstruct

Issue - State: open - Opened by leondz 3 months ago
Labels: probes, new plugin

#733 - Linting - 20240612

Pull Request - State: closed - Opened by leondz 3 months ago

#732 - ensure report is printed in `utf-8` for Windows

Pull Request - State: closed - Opened by jmartin-tech 3 months ago - 1 comment

#731 - CUDA out of memory

Issue - State: closed - Opened by kbmlcoding 4 months ago - 1 comment

#730 - utilize GarakException in cli

Pull Request - State: closed - Opened by jmartin-tech 4 months ago

#729 - restore `req_template_json_object` lost in refactor

Pull Request - State: closed - Opened by jmartin-tech 4 months ago

#728 - BEAST Attack Implementation

Pull Request - State: closed - Opened by erickgalinkin 4 months ago

#727 - add attack vector & impact info to probes

Issue - State: open - Opened by leondz 4 months ago
Labels: probes

#726 - add "done" entry to report.jsonl

Issue - State: closed - Opened by leondz 4 months ago
Labels: reporting

#724 - give more friendly message when `openaicompatible`/`nim` named model not found

Issue - State: open - Opened by leondz 4 months ago
Labels: generators, ux

#723 - don't do a `tqdm` when generating just one item

Issue - State: closed - Opened by leondz 4 months ago - 3 comments
Labels: generators, cli

#722 - InferenceEndpoint Generator fails for AWS because of missing Authorization header

Issue - State: open - Opened by erikinfo 4 months ago - 4 comments
Labels: generators

#721 - Contributing to examples or tutorials

Issue - State: open - Opened by cyberholics 4 months ago - 2 comments

#720 - add hooks in `base.Detector`

Issue - State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors, quality-speed

#719 - stop instantiating detectors in `analyze.report_digest`

Issue - State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors

#718 - Remove support for openai api v0

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: generators

#717 - lazi-fy huggingface, langchain serve, litellm loading

Pull Request - State: open - Opened by leondz 4 months ago - 2 comments
Labels: generators, quality-speed

#716 - octo generator loads slow

Issue - State: open - Opened by leondz 4 months ago
Labels: generators, quality-speed

#715 - generators: move read timeout up to global/`base.Generator` config item

Issue - State: open - Opened by leondz 4 months ago
Labels: generators

#714 - refactor detectors.dan

Issue - State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors

#712 - doc & inheritance fixes to DanInTheWildMini

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: probes

#711 - Feature: configurable plugins

Pull Request - State: closed - Opened by jmartin-tech 4 months ago - 2 comments
Labels: enhancement

#710 - additional `coherechat` generator to reflect `/chat` endpoint

Issue - State: closed - Opened by GangGreenTemperTatum 4 months ago - 1 comment

#709 - remove `openai_v0` plugin

Issue - State: closed - Opened by leondz 4 months ago
Labels: enhancement, generators

#708 - Improve cloze data for `leakreplay` probe

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: probes

#707 - speedup report generation by using class attribs instead of instantiating

Pull Request - State: closed - Opened by leondz 4 months ago - 3 comments
Labels: reporting, quality-speed

#706 - add CONTRIBUTING.md, update contributing guide in docs

Pull Request - State: closed - Opened by leondz 4 months ago - 1 comment
Labels: documentation

#705 - feat: start new cohere chat generator

Pull Request - State: closed - Opened by GangGreenTemperTatum 4 months ago - 4 comments

#704 - probe or buff: non-standard unicode

Issue - State: open - Opened by leondz 4 months ago
Labels: probes, new plugin, buffs

#703 - Update the quality taxonomy

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: probes

#702 - generator & detector structure validation

Pull Request - State: closed - Opened by leondz 4 months ago

#701 - patch ambiguity around generate() filtering or not filtering `None`s

Pull Request - State: closed - Opened by leondz 4 months ago - 1 comment

#700 - probe: polyglo toxicity prompts

Issue - State: open - Opened by leondz 4 months ago
Labels: enhancement, probes, new plugin

#699 - reporting: update `quality` taxonomy & probes

Issue - State: open - Opened by leondz 4 months ago
Labels: reporting

#698 - reporting: separate data and presentation

Issue - State: open - Opened by leondz 4 months ago
Labels: reporting

#697 - typo in racial slur prompt

Pull Request - State: closed - Opened by leondz 4 months ago

#696 - Split NVCF generator into completion chat

Pull Request - State: closed - Opened by leondz 4 months ago - 2 comments
Labels: generators

#695 - test: generator methods should return objects of correct type

Issue - State: open - Opened by leondz 4 months ago - 1 comment
Labels: generators, tests

#694 - Set generator `_call_model()` and `_generate()` type hints; amend functions

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: generators

#693 - rest generator response must be a list

Pull Request - State: closed - Opened by jmartin-tech 4 months ago - 1 comment

#692 - generators: openrouter

Issue - State: open - Opened by leondz 4 months ago
Labels: generators, new plugin

#691 - generator: phi-3 multimodal nim

Issue - State: open - Opened by leondz 4 months ago
Labels: generators, new plugin

#690 - Catch generic exception from transformers in huggingface

Pull Request - State: closed - Opened by jmartin-tech 4 months ago - 1 comment

#689 - stablize openai parallel

Pull Request - State: closed - Opened by jmartin-tech 4 months ago

#688 - Tutorial for contributing a generator

Pull Request - State: closed - Opened by leondz 4 months ago
Labels: documentation

#687 - Bump/black 24.4.2 -- and apply formatting

Pull Request - State: closed - Opened by leondz 4 months ago

#686 - bump & fix black version

Pull Request - State: closed - Opened by leondz 4 months ago