aws-samples/foundation-model-benchmarking-tool issues and pull requests

#285 - Custom predictor: Taking a Ollama Custom Server for TinyLlama 1.1b for example

Pull Request - State: closed - Opened by madhurprash 6 days ago - 1 comment

#284 - Update README.md with new python version

Pull Request - State: closed - Opened by madhurprash 7 days ago

#283 - Bring your own judge functionality

Issue - State: open - Opened by madhurprash 7 days ago

#282 - bug fix in the template.yml file

Pull Request - State: closed - Opened by madhurprash 9 days ago

#281 - Dataset handling if any of the keys have missing fields

Issue - State: open - Opened by madhurprash 14 days ago

#280 - pricing fallback bug fix

Pull Request - State: closed - Opened by madhurprash 16 days ago

#279 - Deepseek and uv changes

Pull Request - State: closed - Opened by aarora79 18 days ago

#278 - Config file for Deepseek deployment on DJL VLLM on SageMaker

Pull Request - State: closed - Opened by madhurprash 19 days ago

#277 - VLLM GPU support

Pull Request - State: closed - Opened by dheerajoruganty 25 days ago

#276 - Model accuracy result file doesn't show correct candidate model name

Issue - State: open - Opened by Yiwen-Zhang 26 days ago

#275 - chore: improve no latency metrics data available messaging

Pull Request - State: closed - Opened by tonyksong 27 days ago

#274 - prompt template naming fix for deepseek

Pull Request - State: closed - Opened by madhurprash 27 days ago

#273 - Ollama fix + deepseek prompt & config file

Pull Request - State: closed - Opened by madhurprash 28 days ago

#272 - Adding sonnet3.5 & llama3.3 as judges

Pull Request - State: closed - Opened by madhurprash 29 days ago

#271 - Ollama end to end support

Pull Request - State: closed - Opened by dheerajoruganty about 1 month ago

#270 - Add benchmarking for training models

Issue - State: open - Opened by madhurprash about 1 month ago

#269 - Qwen + llama3.3 tests for accuracy & readme + doc updates

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#268 - The pricing fetching logic only works for EC2

Issue - State: open - Opened by Yiwen-Zhang about 1 month ago

#267 - Experiment config file's model name doesn't match with pricing_fallback.yml file's model name

Issue - State: open - Opened by Yiwen-Zhang about 1 month ago

#266 - qwen generic config file + prompt template

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#265 - Fix for the hf token content

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#265 - Fix for the hf token content

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#264 - parameterizing tp depree, instance type and batch size

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#264 - parameterizing tp depree, instance type and batch size

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#263 - open orca, dolly dataset config files and prompts for nova

Pull Request - State: closed - Opened by madhurprash about 1 month ago

#263 - open orca, dolly dataset config files and prompts for nova

Pull Request - State: open - Opened by madhurprash about 1 month ago

#262 - Integrate Ollama deployment into FMBench

Issue - State: closed - Opened by dheerajoruganty about 1 month ago - 1 comment

#262 - Integrate Ollama deployment into FMBench

Issue - State: open - Opened by dheerajoruganty about 1 month ago

#261 - parameterize configuration files

Issue - State: open - Opened by madhurprash about 1 month ago

#261 - parameterize configuration files

Issue - State: open - Opened by madhurprash about 1 month ago

#260 - Bug fix for production variant name addition

Pull Request - State: closed - Opened by madhurprash about 1 month ago - 2 comments

#260 - Bug fix for production variant name addition

Pull Request - State: closed - Opened by madhurprash about 1 month ago - 2 comments

#259 - Handling for byoe sagemaker version without list_ep/describe_ep sagemaker permissions

Pull Request - State: closed - Opened by madhurprash about 2 months ago

#258 - ARM CPU benchmarking support

Pull Request - State: closed - Opened by dheerajoruganty about 2 months ago

#257 - llama3.2 1b g6e.2xl config file for smaller prompts

Pull Request - State: closed - Opened by madhurprash 2 months ago

#256 - HF dataset loading bug fixes + new config file

Pull Request - State: closed - Opened by madhurprash 2 months ago

#255 - Summarization Prompt Template

Pull Request - State: closed - Opened by dheerajoruganty 2 months ago

#254 - Update template.yml

Pull Request - State: closed - Opened by antara678 2 months ago

#253 - NOVA models config file + bedrock converseAPI predictor

Pull Request - State: closed - Opened by madhurprash 2 months ago

#252 - Update Fallback Pricing YAML Path

Pull Request - State: closed - Opened by dheerajoruganty 3 months ago

#251 - chore(ec2): update byoe yml and docs

Pull Request - State: closed - Opened by tonyksong 3 months ago

#250 - Update EC2 Metrics

Pull Request - State: closed - Opened by dheerajoruganty 3 months ago