A recent study published via arXiv cs.AI evaluated whether frontier large language models can effectively reverse engineer protocol buffer schemas from network traffic generated by iPhone location services. Researchers tested models including Claude Opus 4.8 and Codex GPT Sol 5.6 by attempting to reconstruct binary data schemas working directly from Apple binaries. The evaluation found that the vast majority of AI-generated schemas contained substantial errors. Because these erroneous schemas remained internally consistent, the mistakes proved difficult to detect, rendering direct AI deployment unsuitable for privacy and forensic analysis.
The Limits of AI in Protocol Reverse Engineering
Reverse engineering mobile software data formats has traditionally been a slow and highly specialized task. While employing language models to automate or accelerate this process is an appealing prospect, the findings indicate that current frontier models cannot be relied upon for precise technical parsing. The announcement did not specify the exact methodology used to measure internal consistency or detail the full breadth of the tested protobuf messages beyond those linked to location services.
What Remains Unclear and Next Steps
Several questions remain open regarding this research. The arXiv report did not disclose whether fine-tuning models specifically on binary structures or protocol buffers could mitigate the high error rate. Additionally, it remains unknown if newer model iterations or multi-agent verification frameworks might improve schema accuracy for forensic applications. Security researchers and privacy analysts will need to watch whether toolmaking assistance can be safely partitioned from direct analytical generation.
Ultimately, while LLMs may act as useful toolmakers in auxiliary tasks, they cannot currently replace human expertise in precise mobile software forensics.
Lead image: Soundarapandian MS / Pexels
