THE VIDEO SHELF · state space models
Do we need Attention? - Linear RNNs and State Space Models (SSMs) for NLP
Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.
Why HN readers recommended it
HN commenters recommend it for building intuition about state space models and the tradeoffs between transformer and SSM layers.
This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.
Video & recommendation details
- Creator / channel
- Sasha Rush
- Topic
- state space models
- Format
- lecture
- Level
- advanced
- HN recommenders in dataset
- 3
- HN mentions in dataset
- 3
- Recommendation years in dataset
- 2024–2024
These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.
From the original HN conversations
Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.
To those curious about the tradeoffs between transformer and state space model layers, I highly recommend Sasha Rush's video on it: [VIDEO]
a_wild_dandan on Hacker News · 2024-03
In Jamba: Production-grade Mamba-based AI modelWould recommend Sasha Rush's lecture [VIDEO] , it provides the intuition for state space models via linear RNNs
anomaly72 on Hacker News · 2024-01
In MoE-Mamba: Efficient Selective State Space Models with Mixture of ExpertsHi! Blog author. This was an attempt a couple years ago to understand and write about this paper in a detailed way. Here is a video going through this topic as well: [VIDEO] Section 2 of the blog post is no longer very relevant. A …
srush on Hacker News · 2024-02
In The Annotated S4
Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.