←
Manvi Agarwal299 posts
Manvi Agarwal avatar

Manvi Agarwal

@ManviTweeteth

Postdoc at Institut de l’Audition (Hearing Institute) | (re)tweets ∈ funny stuff ∪ science-y stuff

◷ Joined December 2015▧ 44 Photos and videos
Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·Aug 16

What in the world is that y-axis scaling!? Differences between points within the same vertical axis are not comparable. Differences between points across different vertical axes are not comparable. Reporting to @GraphCrimes

One of the most under-rated global cultural changes is that upon migrating to England, Indian Hindu and Sikh women's employment converges with natives. My hypothesis is that this is due to caste sanctions weakening overseas.

Post attachment
Sharbatanu Chatterjee | শর্বতনু avatar
⟳ Manvi Agarwal retweeted
Sharbatanu Chatterjee | শর্বতনু@sharbat_c·6 Nov 2025

It should be obvious to French commentators that both the following facts are true : (1) a Zohran Mamdani could not be elected in France because of the lower acceptance and integration of minorities and (2) his economic agenda is pretty normal and not at all radical for France.

Sharbatanu Chatterjee | শর্বতনু avatar
⟳ Manvi Agarwal retweeted
Sharbatanu Chatterjee | শর্বতনু@sharbat_c·18 Apr 2025

Monsieur le Président, j’ai Choosé France for Science (France m’a Choosé for Science, merci MSCA fellowship) et ça passe bien (there are many amazing laboratoires dans ce pays) but sans mesures concrètes pour améliorer la bureaucratie scientifique, il est inutile d'en parler.

Ici en France, la recherche est une priorité, l’innovation une culture, la science un horizon sans limite. Chercheurs, chercheuses du monde entier, choisissez la France, choisissez l’Europe ! Je vous donne rendez-vous le 5 mai. → choosefrance.fr/forscience

Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·8 Apr 2025

3️⃣ We explain why structural information in PE has proven to be empirically successful.  We go back to 1️⃣ and use the content-context connection to show that the higher is the mutual information between data and its positional representation, the better is task performance.

Post attachment
Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·8 Apr 2025

2️⃣ We introduce a new positional encoding method - RoPEPool - that can model causality. How does RoPEPool compare to RoPE and F-StrIPE? Our analysis with a toy example says: RoPEPool isn’t just different, it’s also richer in terms of expressivity.

Post attachment
Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·8 Apr 2025

1️⃣ We show how different families of positional encoding - rotation-based (RoPE) and random fourier features-based (F-StrIPE) - can be compared using kernel methods.  It’s not just vibes - we characterize precisely how queries and keys are affected by positional information.

Post attachment
Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·8 Apr 2025

🚨 We just submitted a follow-up to this work, now available as a preprint: hal.science/hal-05021809

ICASSP 2025 has begun! I will be presenting "F-StrIPE: Fast Structure-Informed Positional Encoding for Symbolic Music Generation" on April 11 in the lecture session "Machine learning for speech, audio and music processing II" at 2 PM. More details below 🧵

Post attachment
Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·7 Apr 2025

In our paper, we show that stochastic positional encoding is, in fact, a noisy version of a well-known kernel approximation technique: Random Fourier Features. We also show how prior knowledge (e.g. related to musical structure) can be used in such linear-complexity Transformers.

Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·7 Apr 2025

However, there was a piece missing: how do you handle relative positional encoding in these linear-complexity transformers? 🤔 Enter Stochastic Positional Encoding! It brings relative positional information back into the picture without going back to quadratic cost.

Manvi Agarwal avatar
Manvi Agarwal@ManviTweeteth·7 Apr 2025

Luckily, there's a solution: you can think of attention as a kernel function and use kernel approximation techniques to reduce the cost from quadratic to linear. ⚡ This was the idea used by Performers, for example.