Blog

The story of FLARE: Fast Low-rank Attention Routing Engine Attention, the core mechanism of transformers, becomes prohibitively expensive at large sequence lengths. This post explains the ideas behind FLARE, an attention operator designed to retain global communication while scaling to million-token regimes on a single GPU. Rather than focusing on architectural novelty for its own sake, the goal is to understand attention as a communication operator and ask a simple question: can we keep the benefits of global message passing without paying the full quadratic cost? ...

From POD to neural manifolds Reduced-order modeling (ROM) is ultimately about preserving dominant system behavior with far fewer degrees of freedom. Classical projection methods, convolutional autoencoder ROMs, and modern neural-field formulations all target this same objective, but they make different assumptions about (i) how we represent the state and (ii) how we evolve it in time. This post walks through that progression and focuses on Smooth Neural Field ROM (SNF-ROM): a projection-based nonlinear ROM that uses continuous neural fields as the state representation and explicitly supports physics-based differentiation and time integration during deployment. ...

Why math is interesting The foundation of my interest in math was laid in my freshman year when I was appointed course staff to a sophomore engineering course. As I taught students to model mechanical interactions using forces and moments, I learned not only to communicate my ideas to newcomers in engineering but also how to challenge my own preconceived notions. Explaining why a certain unseen force has to exist to maintain equilibrium is like talking in a different language, in the sense that certain things are obvious (evident enough not to require proving) to me but not to a student. I have to explain that concentrated point forces, reaction moments and all these unseen entities are fictitious objects made by engineers and scientists to model and approximate reality. ...