OpenNMT/CTranslate2 issues and pull requests

#1695 - fix regression flash attention

Pull Request - State: closed - Opened by minhthuc2502 5 months ago

#1694 - opus-mt-en-zh does not respect the end token

Issue - State: open - Opened by l3utterfly 5 months ago

#1693 - Can't hide GPUs to get_cuda_device_count()

Issue - State: open - Opened by mattiadg 5 months ago - 5 comments

#1692 - How to compile from source on windows 11?

Issue - State: open - Opened by ivanhe123 5 months ago - 3 comments

#1691 - Unexpected inference results from Flan-T5 XXL converted to ctranslate2 with version 4.2.1 and 4.1.1 (using tensor parallel)

Issue - State: open - Opened by gk-kd 5 months ago - 4 comments
Labels: bug

#1690 - example of chatting using llama3 with a pyside6 GUI

Pull Request - State: closed - Opened by BBC-Esq 5 months ago - 3 comments

#1689 - target_prefix latency

Issue - State: open - Opened by SimonBenhamou 5 months ago - 2 comments

#1688 - [SOLVED] Running Llama3 with Ctranslate2

Issue - State: open - Opened by BBC-Esq 5 months ago - 4 comments

#1687 - add rope alibi to encoder

Pull Request - State: open - Opened by vince62s 5 months ago - 16 comments

#1686 - Dynamic LoRA switching

Issue - State: open - Opened by MrigankRaman 5 months ago - 2 comments
Labels: enhancement

#1685 - Problem converting Phi3-instruct-128k; "su" rope scaling in Phi-3

Issue - State: closed - Opened by BBC-Esq 5 months ago - 3 comments
Labels: enhancement

#1684 - OpenELM Support

Issue - State: open - Opened by jncraton 5 months ago

#1683 - Build Ctranslate2 for android

Issue - State: open - Opened by glory03023 5 months ago - 3 comments

#1682 - New Flash Attention Error

Issue - State: closed - Opened by BBC-Esq 5 months ago - 8 comments

#1681 - [Feature Request] Support Epsilon Sampling

Issue - State: open - Opened by radinplaid 5 months ago
Labels: enhancement

#1680 - Add phi3 converter

Pull Request - State: closed - Opened by minhthuc2502 5 months ago - 14 comments

#1679 - Anomalous T5 results using GPU inference on a 4090 graphics card

Issue - State: open - Opened by taishan1994 5 months ago - 3 comments

#1678 - [Feature Request] add a classification / estimation layer on top of Encoders

Issue - State: open - Opened by vince62s 5 months ago - 1 comment
Labels: enhancement

#1677 - How to run FA

Issue - State: closed - Opened by MrigankRaman 5 months ago - 2 comments

#1676 - BENCHmarking new flash attention!

Issue - State: open - Opened by BBC-Esq 5 months ago - 10 comments

#1675 - Congrats on flash attention, now how do I run it???

Issue - State: closed - Opened by BBC-Esq 5 months ago - 3 comments

#1674 - bump version 4.2.1

Pull Request - State: closed - Opened by minhthuc2502 5 months ago

#1673 - faster-whisper inferencing problem

Issue - State: open - Opened by glory03023 5 months ago - 2 comments

#1672 - Phi-3 support

Issue - State: closed - Opened by Theodotus1243 5 months ago - 9 comments

#1671 - Fix llama 3

Pull Request - State: closed - Opened by minhthuc2502 5 months ago - 2 comments

#1670 - Load/unload encoder/generator/whisper/wav2vec2

Pull Request - State: closed - Opened by minhthuc2502 5 months ago

#1669 - Running Llama3 like a genius (that I am not) [SOLVED]:

Issue - State: closed - Opened by BBC-Esq 5 months ago - 2 comments

#1668 - Does fairseq transform align can returns the alignment result?

Issue - State: closed - Opened by itaybar 6 months ago - 5 comments

#1667 - Use generator as translator?

Issue - State: closed - Opened by Weroxig 6 months ago - 3 comments

#1666 - No kernel image is available for execution on the device

Issue - State: closed - Opened by McCloudS 6 months ago - 2 comments

#1665 - `unload_model` support for `Generator`

Issue - State: closed - Opened by NeonBohdan 6 months ago - 3 comments

#1664 - include/ctranslate2/ops/flash-attention/flash_fwd_launch_template.h(15): error: identifier "__grid_constant__" is undefined

Issue - State: open - Opened by twmht 6 months ago - 3 comments

#1663 - 4.2.0 release build for python 3.9

Issue - State: closed - Opened by anterart 6 months ago - 13 comments

#1662 - bump version 4.2.0

Pull Request - State: closed - Opened by minhthuc2502 6 months ago

#1661 - Asking about the return_scores during generation

Issue - State: open - Opened by freyaya123 6 months ago - 7 comments

#1660 - fix gemma bug

Pull Request - State: closed - Opened by minhthuc2502 6 months ago

#1659 - Fix to enter BestSampler route when temperature is 0.

Pull Request - State: closed - Opened by masa-oi 6 months ago - 1 comment

#1658 - [Gemma] GELU should be approx tanh not exact

Issue - State: closed - Opened by NeonBohdan 6 months ago - 2 comments

#1657 - Error when converting NMT model with ALiBi or RoPe

Issue - State: open - Opened by randomicity 6 months ago - 15 comments
Labels: enhancement, question

#1656 - I try to pytest but it complains that "AttributeError: module 'ctranslate2' has no attribute 'get_cuda_device_count'"

Issue - State: closed - Opened by otakutyrant 6 months ago - 5 comments

#1655 - int8 quantization not working

Issue - State: closed - Opened by homink 6 months ago - 1 comment

#1654 - When building from source missing C++ libraries

Issue - State: closed - Opened by anterart 6 months ago - 6 comments

#1653 - Deploying with model and tensor parallelism

Issue - State: closed - Opened by subhalingamd 6 months ago - 4 comments

#1652 - Trigger github action

Pull Request - State: closed - Opened by XomaDev 6 months ago - 1 comment

#1651 - Flash attention

Pull Request - State: closed - Opened by minhthuc2502 6 months ago - 3 comments

#1650 - Benchmarking common LLMs on ctranslate2, llama.cpp, and bitsandbytes

Issue - State: open - Opened by BBC-Esq 6 months ago - 1 comment

#1649 - Support for Zephyr and other "StableLmForCausalLM" models?

Issue - State: open - Opened by BBC-Esq 6 months ago - 2 comments

#1648 - Whisper batch generation is not faster than loops

Issue - State: open - Opened by evanarlian 6 months ago - 5 comments

#1647 - Fix 'ValueError' in 'ct2-opennmt-py-converter' for Unsupported '--self_attn_type'

Pull Request - State: closed - Opened by chingizof 6 months ago - 2 comments

#1646 - `translate_iterable` is not properly handling `max_input_length`

Issue - State: open - Opened by winstxnhdw 6 months ago - 2 comments

#1645 - Support Translator with persistent cpu cache

Pull Request - State: closed - Opened by anterart 6 months ago - 2 comments