Researchers reporting via arXiv cs.AI have identified a significant flaw in how enterprise AI agents process documents. When transforming Word files or PDFs into tokens—a process known as ingestion—mainstream converters consistently fail to capture visual cues. This “silent cue loss” means that formatting used by human authors to convey meaning, such as bolding, underlining, or specific layouts, is stripped away. Consequently, AI agents tasked with following policies or answering compliance questions may provide incorrect answers with high confidence because they lack the context provided by the document‘s original appearance.
The Impact of Visual Formatting Loss
In enterprise environments, AI agents are often deployed to automate workflows like filing tickets or interpreting complex policy manuals. These agents rely on an ingestion layer to translate human-readable files into a format the machine can process. However, the study found that every mainstream converter tested lost the meaning encoded in the visual presentation of the text. This is not a failure of the AI’s reasoning capabilities but a failure of the data pipeline. By ignoring the Office Open XML (OOXML) properties that define a document’s structure and style, the ingestion layer delivers an incomplete picture to the agent. This results in agents that are “confidently wrong” rather than flagging the information as missing or ambiguous.
Unknowns in Ingestion Technology
The report did not specify which specific mainstream converters were tested, leaving it unclear if proprietary enterprise tools perform better or worse than open-source alternatives. Additionally, the announcement did not name the specific AI models or agent architectures used to demonstrate the confidently wrong behavior. While the researchers utilized a dataset of 76 real-world policy documents to create a recovery task, the full scope of the industries represented in these documents remains unknown. It is also not stated whether specific file formats, such as older .doc files versus modern .docx files, exhibit different levels of information loss. Future developments will likely focus on whether new ingestion layers can preserve OOXML properties to ensure that the visual intent of an author is successfully communicated to the AI agent.
As companies increasingly rely on automated agents for compliance and policy enforcement, the fidelity of document ingestion remains a critical hurdle. Resolving this silent information loss is essential for building reliable enterprise AI systems.
Lead image: brotiN biswaS / Pexels
