Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / leondz/garak issues and pull requests
#786 - chatscanner integration live chatbot
Issue -
State: open - Opened by priamai 2 months ago
#785 - Does garak use LLM's to generate prompts? or default set of prompts/probes
Issue -
State: open - Opened by sumukh5 2 months ago
#784 - Does REST generator support multiple end point and responses?
Issue -
State: open - Opened by sumukh5 3 months ago
- 2 comments
#783 - probe: sandwich probe
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: probes, new plugin
#782 - defensive coding: allow python generators more places
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: enhancement
#781 - don't skip listing probes in 100%-pass-rate modules
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: reporting
#780 - stop encoding.payloads accumulating each load
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: bug, probes
#779 - move REST generator exception into garak.exceptions
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: enhancement, generators
#778 - always log nvcf payload
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: generators
#777 - atkgen - add configurability, allow config of which adversarial generator to use
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: generators
#776 - generator: lorem ipsum
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: generators
#775 - enhancement: enable lazy setting of nested dicts
Pull Request -
State: open - Opened by leondz 3 months ago
- 2 comments
Labels: enhancement
#774 - Revert "enable nested setting of plugin configs"
Pull Request -
State: closed - Opened by leondz 3 months ago
#773 - enable nested setting of plugin configs
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: enhancement
#772 - update NVCF to support versioning, custom payload keys
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: generators
#771 - add DEFAULT_PARAMS to base types that are configurables
Pull Request -
State: closed - Opened by leondz 3 months ago
#770 - reduce zalgo-ness
Pull Request -
State: closed - Opened by leondz 3 months ago
#769 - add colour theme config
Pull Request -
State: closed - Opened by leondz 3 months ago
#768 - cached plugin enum
Pull Request -
State: open - Opened by jmartin-tech 3 months ago
- 1 comment
Labels: quality-speed
#767 - probe: assess 🤗 repos for potentially malicious files
Pull Request -
State: open - Opened by leondz 3 months ago
- 1 comment
Labels: probes
#766 - Revert "probe: assess 🤗 repos for potentially malicious files"
Pull Request -
State: closed - Opened by leondz 3 months ago
#764 - probe: topic pushing
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: probes
#763 - add grandma substance synth probe, update docs and var names
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: probes
#762 - add mitigation strings, add test for string-based mitigation skipping
Pull Request -
State: open - Opened by leondz 3 months ago
Labels: detectors
#761 - retry openaicompatible requests if invalid content received
Pull Request -
State: closed - Opened by leondz 3 months ago
- 1 comment
Labels: bug, generators
#760 - AssertionError: _call_model must return a list
Issue -
State: closed - Opened by brunomcuesta 3 months ago
- 5 comments
Labels: bug
#759 - rename class level probe doc_uri
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
#758 - probe: Explicit: forced instruction-following.
Issue -
State: open - Opened by leondz 3 months ago
Labels: probes, new plugin
#757 - Add Skeleton Key probe
Issue -
State: open - Opened by erickgalinkin 3 months ago
- 1 comment
Labels: probes, new plugin
#756 - align detector output types & results
Issue -
State: open - Opened by leondz 3 months ago
Labels: enhancement, detectors
#755 - `litellm` generator input validation breaks on default cases
Issue -
State: open - Opened by leondz 3 months ago
Labels: bug, generators
#754 - probe: assess 🤗 repos for potentially malicious files
Pull Request -
State: closed - Opened by leondz 3 months ago
- 2 comments
Labels: probes, new plugin
#753 - only download HF models when generation is required
Issue -
State: open - Opened by leondz 3 months ago
Labels: enhancement, generators, quality-speed
#752 - probe: VisualWebArena-Adv
Issue -
State: open - Opened by leondz 3 months ago
Labels: probes, new plugin
#751 - log the config of each plugin at load time
Issue -
State: open - Opened by leondz 3 months ago
Labels: enhancement
#750 - atkgen forking HF tokenizer despite being non-parallel
Issue -
State: open - Opened by leondz 3 months ago
Labels: bug, probes
#749 - remove extraneous default params for nims that expect conservative pa…
Pull Request -
State: closed - Opened by leondz 3 months ago
- 1 comment
Labels: bug, generators
#748 - nim openai endpoints reject requests with unexpected params in payload
Issue -
State: closed - Opened by leondz 3 months ago
- 1 comment
Labels: bug, generators
#747 - InjectZalgo sometimes has very high prompt counts
Issue -
State: open - Opened by leondz 3 months ago
Labels: bug
#746 - probe: chatgpt pii exfil
Issue -
State: open - Opened by leondz 3 months ago
Labels: probes, new plugin
#745 - feature: generator temp / random seed shuffle
Issue -
State: open - Opened by leondz 3 months ago
Labels: enhancement, generators
#744 - Handle openai 400 errors more gracefully
Pull Request -
State: closed - Opened by leondz 3 months ago
#743 - return value from `batch_decode` is already a list
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
#742 - Refactor `huggingface` config support
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
#741 - only set generations if still default
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
#740 - add probe for "grandma" attack
Pull Request -
State: closed - Opened by leondz 3 months ago
- 1 comment
#739 - add garak preprint paper
Pull Request -
State: closed - Opened by leondz 3 months ago
#738 - require probes to match input modality of the generator
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
#737 - update `glitch` plugin
Issue -
State: open - Opened by leondz 3 months ago
Labels: probes
#736 - Target LM Studio
Issue -
State: closed - Opened by osok 3 months ago
- 1 comment
#735 - add 'complete' message to report at end of run
Pull Request -
State: closed - Opened by leondz 3 months ago
Labels: reporting
#734 - probe: disguise & reconstruct
Issue -
State: open - Opened by leondz 3 months ago
Labels: probes, new plugin
#733 - Linting - 20240612
Pull Request -
State: closed - Opened by leondz 3 months ago
#732 - ensure report is printed in `utf-8` for Windows
Pull Request -
State: closed - Opened by jmartin-tech 3 months ago
- 1 comment
#731 - CUDA out of memory
Issue -
State: closed - Opened by kbmlcoding 4 months ago
- 1 comment
#730 - utilize GarakException in cli
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
#729 - restore `req_template_json_object` lost in refactor
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
#728 - BEAST Attack Implementation
Pull Request -
State: closed - Opened by erickgalinkin 4 months ago
#727 - add attack vector & impact info to probes
Issue -
State: open - Opened by leondz 4 months ago
Labels: probes
#726 - add "done" entry to report.jsonl
Issue -
State: closed - Opened by leondz 4 months ago
Labels: reporting
#725 - `garak.analyze.report_digest` html output skips mention of probes used if the module score is 100%
Issue -
State: open - Opened by leondz 4 months ago
Labels: bug, reporting
#724 - give more friendly message when `openaicompatible`/`nim` named model not found
Issue -
State: open - Opened by leondz 4 months ago
Labels: generators, ux
#723 - don't do a `tqdm` when generating just one item
Issue -
State: closed - Opened by leondz 4 months ago
- 3 comments
Labels: generators, cli
#722 - InferenceEndpoint Generator fails for AWS because of missing Authorization header
Issue -
State: open - Opened by erikinfo 4 months ago
- 4 comments
Labels: generators
#721 - Contributing to examples or tutorials
Issue -
State: open - Opened by cyberholics 4 months ago
- 2 comments
#720 - add hooks in `base.Detector`
Issue -
State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors, quality-speed
#719 - stop instantiating detectors in `analyze.report_digest`
Issue -
State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors
#718 - Remove support for openai api v0
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: generators
#717 - lazi-fy huggingface, langchain serve, litellm loading
Pull Request -
State: open - Opened by leondz 4 months ago
- 2 comments
Labels: generators, quality-speed
#716 - octo generator loads slow
Issue -
State: open - Opened by leondz 4 months ago
Labels: generators, quality-speed
#715 - generators: move read timeout up to global/`base.Generator` config item
Issue -
State: open - Opened by leondz 4 months ago
Labels: generators
#714 - refactor detectors.dan
Issue -
State: open - Opened by leondz 4 months ago
Labels: enhancement, detectors
#713 - tests: check plugin modules for classes that aren't picked up by plugin enumeration
Issue -
State: open - Opened by leondz 4 months ago
Labels: tests
#712 - doc & inheritance fixes to DanInTheWildMini
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: probes
#711 - Feature: configurable plugins
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
- 2 comments
Labels: enhancement
#710 - additional `coherechat` generator to reflect `/chat` endpoint
Issue -
State: closed - Opened by GangGreenTemperTatum 4 months ago
- 1 comment
#709 - remove `openai_v0` plugin
Issue -
State: closed - Opened by leondz 4 months ago
Labels: enhancement, generators
#708 - Improve cloze data for `leakreplay` probe
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: probes
#707 - speedup report generation by using class attribs instead of instantiating
Pull Request -
State: closed - Opened by leondz 4 months ago
- 3 comments
Labels: reporting, quality-speed
#706 - add CONTRIBUTING.md, update contributing guide in docs
Pull Request -
State: closed - Opened by leondz 4 months ago
- 1 comment
Labels: documentation
#705 - feat: start new cohere chat generator
Pull Request -
State: closed - Opened by GangGreenTemperTatum 4 months ago
- 4 comments
#704 - probe or buff: non-standard unicode
Issue -
State: open - Opened by leondz 4 months ago
Labels: probes, new plugin, buffs
#703 - Update the quality taxonomy
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: probes
#702 - generator & detector structure validation
Pull Request -
State: closed - Opened by leondz 4 months ago
#701 - patch ambiguity around generate() filtering or not filtering `None`s
Pull Request -
State: closed - Opened by leondz 4 months ago
- 1 comment
#700 - probe: polyglo toxicity prompts
Issue -
State: open - Opened by leondz 4 months ago
Labels: enhancement, probes, new plugin
#699 - reporting: update `quality` taxonomy & probes
Issue -
State: open - Opened by leondz 4 months ago
Labels: reporting
#698 - reporting: separate data and presentation
Issue -
State: open - Opened by leondz 4 months ago
Labels: reporting
#697 - typo in racial slur prompt
Pull Request -
State: closed - Opened by leondz 4 months ago
#696 - Split NVCF generator into completion chat
Pull Request -
State: closed - Opened by leondz 4 months ago
- 2 comments
Labels: generators
#695 - test: generator methods should return objects of correct type
Issue -
State: open - Opened by leondz 4 months ago
- 1 comment
Labels: generators, tests
#694 - Set generator `_call_model()` and `_generate()` type hints; amend functions
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: generators
#693 - rest generator response must be a list
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
- 1 comment
#692 - generators: openrouter
Issue -
State: open - Opened by leondz 4 months ago
Labels: generators, new plugin
#691 - generator: phi-3 multimodal nim
Issue -
State: open - Opened by leondz 4 months ago
Labels: generators, new plugin
#690 - Catch generic exception from transformers in huggingface
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
- 1 comment
#689 - stablize openai parallel
Pull Request -
State: closed - Opened by jmartin-tech 4 months ago
#688 - Tutorial for contributing a generator
Pull Request -
State: closed - Opened by leondz 4 months ago
Labels: documentation
#687 - Bump/black 24.4.2 -- and apply formatting
Pull Request -
State: closed - Opened by leondz 4 months ago
#686 - bump & fix black version
Pull Request -
State: closed - Opened by leondz 4 months ago