SPEX and ProxySPEX: Methods for Identifying Influential Interactions in LLMs at Scale
Researchers from BAIR (Berkeley AI) introduced SPEX and ProxySPEX — algorithms capable of identifying critical interactions among features, data, and components of AI models, including LLMs, orders of magnitude faster than existing methods. The methods are based on the idea of sparsity of influential interactions and use techniques from coding theory and signal processing. Application of SPEX showed that standard attribution methods (e.g., SHAP) can miss complex synergies, as in the case of the trolley problem. ProxySPEX, which exploits the property of hierarchy, achieves the same accuracy with roughly 10 times fewer ablations.
OpenAI

