Jamie-Stirling/RetNet issues and pull requests

#37 - Is Retnet equivalent to ordinary GPT when the decay is set to 1 ?

Issue - State: open - Opened by xuanyaoming 10 months ago - 3 comments

#36 - Dimensions of forward_recurrent

Issue - State: closed - Opened by Qiu30 12 months ago - 5 comments

#35 - a question about xpos and D of decay mat

Issue - State: open - Opened by DavideHe 12 months ago - 2 comments

#34 - Confusion about "the chunkwise recurrent representation of retention"

Issue - State: open - Opened by CHENHUI-X about 1 year ago

#33 - Can this mechanism be applied to PointCloud data ?

Issue - State: open - Opened by madjid-dx about 1 year ago

#32 - NO LM HEAD

Issue - State: closed - Opened by shnuhw about 1 year ago - 2 comments

#31 - Fix dimension mismatch when hidden size is odd

Pull Request - State: closed - Opened by ilunye about 1 year ago

#30 - How to predict use this net?

Issue - State: open - Opened by GodPCWANG about 1 year ago

#29 - Faster implementation of MultiScaleRetention, adds dependency on einops

Pull Request - State: closed - Opened by draguve about 1 year ago - 1 comment

#28 - The complex theta should cancel out

Issue - State: open - Opened by albertbuchard about 1 year ago

#27 - /src/retnet.py GPU

Issue - State: closed - Opened by Qiu30 about 1 year ago - 2 comments

#26 - Fix math problem in gamma calculation

Pull Request - State: closed - Opened by Jun-depo about 1 year ago - 1 comment

#25 - Assistance on training a new retention network model ?

Issue - State: open - Opened by risedangel about 1 year ago

#24 - Passing Attention Masks

Issue - State: open - Opened by leffff about 1 year ago - 3 comments

#23 - Update retention.py

Pull Request - State: closed - Opened by leffff about 1 year ago - 1 comment

#22 - Q, k and D device difference

Issue - State: closed - Opened by leffff about 1 year ago - 1 comment

#21 - Proposed improvement/collaboration: removing the O(T^2) training cost

Issue - State: closed - Opened by jackd about 1 year ago - 2 comments

#20 - Fixed typo

Pull Request - State: open - Opened by EgoVeroConsisto about 1 year ago

#19 - can retnet be applied in point cloud tasks？

Issue - State: open - Opened by huiyang0613 about 1 year ago

#18 - Changelog of official implementation

Issue - State: open - Opened by donglixp over 1 year ago - 2 comments

#17 - what about cross-attention

Issue - State: open - Opened by aki819 over 1 year ago

#16 - Error when the model is running on GPU

Issue - State: open - Opened by SSamDav over 1 year ago - 1 comment

#15 - Update src/complex/retention.py

Pull Request - State: closed - Opened by MichaelFu1998-create over 1 year ago - 1 comment

#14 - demo example / number of parameter control vs original code

Issue - State: open - Opened by thegodone over 1 year ago - 4 comments

#13 - Chunkwise real

Pull Request - State: closed - Opened by Jamie-Stirling over 1 year ago

#12 - Can you make this repo in available for package installers (pip)?

Issue - State: open - Opened by gaasher over 1 year ago
Labels: enhancement

#11 - RetNet Officially Released

Issue - State: closed - Opened by tiendung over 1 year ago - 1 comment

#10 - Chunkwise retention giving different output

Issue - State: closed - Opened by Jamie-Stirling over 1 year ago - 4 comments
Labels: bug, help wanted

#9 - Minor docs fix

Pull Request - State: closed - Opened by Regenhardt over 1 year ago

#7 - _get_D function very slow for long sequence

Issue - State: closed - Opened by ZuowenWang0000 over 1 year ago - 1 comment

#6 - Real-valued implementation using xPos