google/praxis issues and pull requests

#85 - [NVIDIA] Support GQA with `jax.nn.dot_product_attention`

Pull Request - State: open - Opened by kaixih about 1 month ago

#84 - Use TE dpa for grok mqa

Pull Request - State: open - Opened by hx89 about 1 month ago

#83 - Fix unindex tuple error for pipeline parallelism

Pull Request - State: closed - Opened by wenscarl 2 months ago - 2 comments
Labels: pull ready

#82 - Bump tensorflow from 2.9.3 to 2.12.1 in /praxis/pip_package in the pip group across 1 directory

Pull Request - State: open - Opened by dependabot[bot] 2 months ago
Labels: dependencies

#81 - lingvo removed

Pull Request - State: closed - Opened by rajatsen91 2 months ago

#80 - [NVIDIA] Use separate FP8 einsum instances in MoE

Pull Request - State: closed - Opened by kaixih 3 months ago - 3 comments
Labels: pull ready

#79 - [NVIDIA] Fix FP8 QDQ calls in praxis

Pull Request - State: closed - Opened by kaixih 3 months ago - 1 comment
Labels: pull ready

#78 - Bump the pip group in /praxis/pip_package with 2 updates

Pull Request - State: closed - Opened by dependabot[bot] 3 months ago - 1 comment
Labels: dependencies

#77 - Fix checkpoint for Grok

Pull Request - State: closed - Opened by hx89 4 months ago
Labels: pull ready

#76 - Fix checkpoint policy for Grok

Pull Request - State: open - Opened by hx89 4 months ago

#75 - Bump the pip group across 1 directory with 3 updates

Pull Request - State: closed - Opened by dependabot[bot] 4 months ago - 1 comment
Labels: dependencies

#74 - [Experimental] Use TE dpa for grok mqa

Pull Request - State: closed - Opened by hx89 4 months ago

#73 - Bump the pip group across 1 directory with 2 updates

Pull Request - State: closed - Opened by dependabot[bot] 4 months ago - 1 comment
Labels: dependencies

#72 - [draft] Fp8 inference experimental

Pull Request - State: open - Opened by wenscarl 4 months ago

#71 - Quantize dispatch gemm to fp8

Pull Request - State: closed - Opened by hx89 4 months ago
Labels: pull ready

#70 - Grok fp8

Pull Request - State: closed - Opened by hx89 4 months ago
Labels: pull ready

#69 - Support fp8 direct quantization

Pull Request - State: open - Opened by wenscarl 4 months ago - 2 comments
Labels: pull ready

#68 - Support for loading fp8 checkpoint

Issue - State: open - Opened by wenscarl 4 months ago - 12 comments

#67 - Bump the pip group across 1 directory with 2 updates

Pull Request - State: closed - Opened by dependabot[bot] 5 months ago - 1 comment
Labels: dependencies

#66 - Enable dp in grok

Pull Request - State: closed - Opened by hx89 5 months ago - 1 comment
Labels: pull ready

#65 - Bump tensorflow from 2.9.3 to 2.11.1 in /praxis/pip_package in the pip group across 1 directory

Pull Request - State: closed - Opened by dependabot[bot] 5 months ago - 1 comment
Labels: dependencies

#64 - Fix Grok config

Pull Request - State: closed - Opened by hx89 5 months ago - 1 comment
Labels: pull ready

#63 - Add activation checkpoint offloading

Issue - State: open - Opened by jon-chuang 5 months ago

#62 - Bump the pip group across 1 directory with 3 updates

Pull Request - State: closed - Opened by dependabot[bot] 5 months ago - 1 comment
Labels: dependencies

#61 - [NVIDIA] Add support for LoRA PEFT

Pull Request - State: closed - Opened by ashors1 5 months ago - 1 comment
Labels: pull ready

#60 - Add support for glam with repeated_layer=False

Pull Request - State: closed - Opened by ashors1 5 months ago
Labels: pull ready

#59 - Fix typo

Pull Request - State: open - Opened by ppwwyyxx 5 months ago

#58 - Create grok.py

Pull Request - State: closed - Opened by abhinavgoel95 6 months ago - 1 comment
Labels: pull ready

#57 - Bump idna from 3.6 to 3.7 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 6 months ago - 1 comment
Labels: dependencies

#56 - TransformerLm docs say `start_time_step` should be `prefix_len` but LanguageModel uses `prefix_len-1`

Issue - State: closed - Opened by DCtheTall 6 months ago

#55 - Adding support for expert parallelism

Pull Request - State: closed - Opened by abhinavgoel95 6 months ago
Labels: pull ready

#54 - Bump pillow from 10.2.0 to 10.3.0 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 6 months ago - 1 comment
Labels: dependencies

#53 - [NVIDIA] Add a custom layer for cudnn flash attention

Pull Request - State: closed - Opened by kaixih 6 months ago - 7 comments
Labels: pull ready

#52 - Cross-layer attention weight sharing fails in different scopes

Issue - State: open - Opened by mqyqlx 7 months ago - 3 comments

#51 - Add offloading checkpoint policy

Pull Request - State: open - Opened by jaro-sevcik 7 months ago

#50 - [NVIDIA] Add a custom model which supports evaluation on the BoolQ dataset

Pull Request - State: closed - Opened by ashors1 8 months ago - 3 comments
Labels: pull ready

#49 - Incorrect conversion from tf dtype to jax dtype

Issue - State: open - Opened by backpropper 8 months ago

#48 - Add support for checkpoint policies in MoE models

Pull Request - State: closed - Opened by abhinavgoel95 8 months ago - 1 comment
Labels: pull ready

#47 - Moe expert parallel support

Pull Request - State: closed - Opened by abhinavgoel95 8 months ago - 3 comments
Labels: pull ready

#46 - [Feature Request] Need Matmul Attention layer instead of Einsum to support GPU running

Issue - State: open - Opened by MoFHeka 8 months ago

#45 - [Feature Request] Need ZeRo-1/2 to cooperate with PP+TP+DP. Which may more faster than FSDP sometimes.

Issue - State: open - Opened by MoFHeka 8 months ago

#44 - Support custom FP8 dtype in Pipelined Transformer

Issue - State: closed - Opened by kaixih 8 months ago - 1 comment

#43 - Bump notebook from 7.0.6 to 7.0.7 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies

#42 - Bump jupyterlab from 4.0.10 to 4.0.11 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies

#41 - Bump jupyter-lsp from 2.2.1 to 2.2.2 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies

#40 - [NVIDIA] Use custom grad accumulation for FP8 params

Pull Request - State: closed - Opened by kaixih 9 months ago - 2 comments
Labels: pull ready

#39 - Bump jinja2 from 3.1.2 to 3.1.3 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies

#38 - Bump jupyter-server from 2.11.1 to 2.11.2 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 10 months ago - 1 comment
Labels: dependencies

#37 - Modify GQA for compatibility with Praxis' transformer

Pull Request - State: closed - Opened by ashors1 10 months ago
Labels: pull ready

#36 - Add Transformer Engine support to Praxis

Pull Request - State: open - Opened by ashors1 10 months ago - 1 comment

#35 - [NVIDIA] Use the fast accumulation for FP8 matmul

Pull Request - State: closed - Opened by kaixih 11 months ago - 2 comments
Labels: pull ready

#34 - Added draft for grouped query attention

Pull Request - State: open - Opened by abhinavgoel95 11 months ago

#33 - Bump urllib3 from 2.0.6 to 2.0.7 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] 12 months ago - 1 comment
Labels: dependencies

#32 - Praxis 1.2.0 release

Pull Request - State: open - Opened by chandrasekhard2 12 months ago - 1 comment

#31 - Bump urllib3 from 1.26.16 to 1.26.17 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] about 1 year ago - 1 comment
Labels: dependencies

#30 - gpu_fast_attention not passing segment_ids to jax pallas attention mha

Issue - State: open - Opened by Cjkkkk about 1 year ago

#29 - [NVIDIA] Add overwrite_with_gradient collection

Pull Request - State: closed - Opened by kaixih about 1 year ago - 2 comments
Labels: pull ready

#28 - [NVIDIA] Add FP8 quantization ops

Pull Request - State: closed - Opened by kaixih about 1 year ago
Labels: pull ready

#27 - Add alternate method to apply mask to allow XLA to detect MHA pattern

Pull Request - State: open - Opened by ashors1 about 1 year ago

#26 - Add FP8 custom op for NVIDIA Hopper GPUs

Pull Request - State: closed - Opened by kaixih about 1 year ago - 1 comment

#25 - Bump tornado from 6.3.2 to 6.3.3 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] about 1 year ago - 1 comment
Labels: dependencies

#24 - [Duplicate] Enhancements for transformer engine support

Pull Request - State: closed - Opened by ashors1 about 1 year ago
Labels: pull ready

#23 - Bump certifi from 2023.5.7 to 2023.7.22 in /praxis/pip_package

Pull Request - State: open - Opened by dependabot[bot] about 1 year ago
Labels: dependencies

#22 - Praxis layers don't support user-specified collection names

Issue - State: closed - Opened by kaixih about 1 year ago - 2 comments

#21 - Enhancements for transformer engine support

Pull Request - State: closed - Opened by ashors1 about 1 year ago - 1 comment
Labels: pull ready

#20 - Adds SKIP_HEAD_INSTALLS as an environment variable for installation

Pull Request - State: closed - Opened by terrykong over 1 year ago - 3 comments

#19 - adding alternate method to apply mask to allow XLA to detect MHA pattern

Pull Request - State: open - Opened by abhinavgoel95 over 1 year ago - 1 comment

#18 - adding alternate method to apply mask to allow XLA to detect MHA patt…

Pull Request - State: closed - Opened by abhinavgoel95 over 1 year ago - 1 comment

#17 - Remove microbatch nan check

Pull Request - State: closed - Opened by abhinavgoel95 over 1 year ago - 4 comments
Labels: pull ready

#16 - Adds a dev extra to make installing from git optional to not override an already installed fiddle/jax

Pull Request - State: closed - Opened by terrykong over 1 year ago - 6 comments

#15 - Bump requests from 2.30.0 to 2.31.0 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies

#14 - SineReLU activation function added

Pull Request - State: closed - Opened by sleepingcat4 over 1 year ago - 2 comments
Labels: pull ready

#13 - Bump tensorflow from 2.9.3 to 2.11.1 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies

#9 - Bump tensorflow from 2.9.3 to 2.11.1 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago
Labels: dependencies, pull ready

#8 - use lax.cond operator for gradient accumulation optimizer

Pull Request - State: closed - Opened by shawnwang18 over 1 year ago - 14 comments
Labels: pull ready

#7 - Any publicly available document?

Issue - State: open - Opened by Lekja00160612 over 1 year ago - 1 comment

#6 - Perform gradient clipping on global batch when using gradient accumulation

Pull Request - State: open - Opened by ashors1 over 1 year ago
Labels: pull ready

#5 - Report Strict Accuracy Metric

Pull Request - State: closed - Opened by ashors1 over 1 year ago - 3 comments
Labels: pull ready

#4 - FP32 LayerNorm for GPU

Pull Request - State: closed - Opened by ashors1 over 1 year ago
Labels: pull ready

#3 - update docstring

Pull Request - State: closed - Opened by ashors1 over 1 year ago

#2 - Bump protobuf from 3.15 to 3.18.3 in /praxis/pip_package

Pull Request - State: closed - Opened by dependabot[bot] almost 2 years ago - 4 comments
Labels: dependencies

#1 - add some explicit embedding lookup boundary check as jax does not raise out-of-boundary error

Pull Request - State: closed - Opened by copybara-service[bot] over 2 years ago

GitHub / google/praxis issues and pull requests