MIT, Harvard & Northeastern U’s Sparse Probing Aims at ‘Finding Neurons in a Haystack’ | Synced

In the new paper Finding Neurons in a Haystack: Case Studies with Sparse Probing, a research team from MIT, Harvard University and Northeastern University proposes sparse probing, a technique that probes over 100 features to precisely localize the neurons in large language models that are relevant to a specific feature or concept.