A recent preprint paper reported by arXiv cs.CL investigates how language model representations handle morpho-syntactic information. Researchers used part-of-speech categories as a controlled test case to observe whether linguistic structures are encoded by individual latent features or by structured groups of features within Sparse AutoEncoders.
Understanding Model Latents
Sparse AutoEncoders are frequently deployed to make neural network representations more interpretable. However, the precise nature of the linguistic structures exposed by these latents has remained an open question. The new findings indicate that part-of-speech distinctions can be recovered successfully from Sparse AutoEncoder activations, demonstrating that this information exists within the latent space rather than merely through simple lexical memorisation.
Open Questions and Future Work
Despite the high recoverability of part-of-speech categories, the study revealed that these distinctions do not map cleanly onto one-to-one relationships between individual latents and specific linguistic categories. Furthermore, the analysis showed that open and closed part-of-speech classes differ in how they are represented. The announcement did not specify the exact architecture of the language models tested or detail the precise mechanisms driving the observed differences between open and closed classes.
Further research will be needed to fully map how broader grammatical structures are distributed across latent feature groups in large language models.
Lead image: Miguel Á. Padriñán / Pexels
