Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / cohere-ai/tokenizer issues and pull requests
#41 - Update README.md
Pull Request -
State: closed - Opened by daulet about 1 year ago
#40 - Bump gopkg.in/yaml.v3 from 3.0.0-20200313102051-9f266ea9e77c to 3.0.0
Pull Request -
State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies
#39 - Stop ignoring case in the pre-tokenizer.
Pull Request -
State: closed - Opened by Fraser-Greenlee over 1 year ago
- 1 comment
#38 - Can Cohere tokenizer tokenize for GPT-3 as well?
Issue -
State: open - Opened by wjkoh almost 2 years ago
#37 - End of sequence token for Cohere models
Issue -
State: open - Opened by teetone about 2 years ago
#36 - Fix string representation return
Pull Request -
State: closed - Opened by mkozakov about 2 years ago
#35 - Bump version of golangci-lint
Pull Request -
State: closed - Opened by mkozakov about 2 years ago
#34 - Return string representation of tokens
Pull Request -
State: closed - Opened by mkozakov about 2 years ago
#33 - Improve Encoding Efficiency with Proper Slice Capacities
Pull Request -
State: closed - Opened by michael-james-holloway over 2 years ago
- 1 comment
#32 - Update README.md
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#31 - EncodeWords public
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#30 - ignore .dist in gitignore
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#29 - WordSplit public bc ocean needs it
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#28 - changed name -- was confusing bc another file has BPE and this one ha…
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#27 - added tests & files from ocean/lib/bpe
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#26 - testing changing to uint64
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#25 - actually removed the file...
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#24 - didn't add folders to remove in last pr
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#23 - changed name of folder
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#22 - delete cokenizer :(
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#21 - python c extension tests
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#20 - changed all non-struct / encode / decode functions to private
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#19 - initial VM gopy draft
Pull Request -
State: closed - Opened by choicallum over 2 years ago
- 1 comment
#18 - Revert "added GetTokens and ListTokens"
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#17 - Build script for compilation / release
Pull Request -
State: closed - Opened by alexguo247 over 2 years ago
#16 - added benchmarks for 1k tokens encoding + decoding
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#15 - added GetTokens and ListTokens
Pull Request -
State: closed - Opened by choicallum over 2 years ago
- 1 comment
#14 - updating README
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#13 - odd decoding -> encode behaviour that doesn't occur when encoding -> decoding
Issue -
State: open - Opened by choicallum over 2 years ago
#12 - draft for tests & benchmarks
Pull Request -
State: closed - Opened by choicallum over 2 years ago
#11 - Tests for Encode and Decode
Pull Request -
State: closed - Opened by alexguo247 over 2 years ago
#10 - Basic refactoring + cleanup
Pull Request -
State: closed - Opened by alexguo247 over 2 years ago
#9 - build into c extensions for python and node sdks
Issue -
State: open - Opened by kipply over 2 years ago
#8 - use this package in go sdk
Issue -
State: closed - Opened by kipply over 2 years ago
#7 - add feature to list built in tokenizers
Issue -
State: open - Opened by kipply over 2 years ago
#6 - remove tokenizer from ocean, and have all our internal services import this repo for tokenization
Issue -
State: open - Opened by kipply over 2 years ago
#5 - document speed
Issue -
State: closed - Opened by kipply over 2 years ago
#4 - write examples for the library
Issue -
State: closed - Opened by kipply over 2 years ago
#3 - add release management for the library
Issue -
State: closed - Opened by kipply over 2 years ago
#2 - do a general code cleaning pass on encode.go
Issue -
State: closed - Opened by kipply over 2 years ago
#1 - write tests for encode and decode
Issue -
State: closed - Opened by kipply over 2 years ago