Perplexity has launched Photon, a custom-built retrieval and ranking engine designed to optimize the performance of its AI-native search stack. Developed using the Rust programming language, Photon replaces a previously used forked open-source engine. The company has moved all production traffic to this new system, which reportedly reduces p99 latency from 800 milliseconds to 65 milliseconds. According to reporting by MarkTechPost, the engine also powers a new Fast Search mode within the Perplexity Search API, where the company reports single-call latency figures of 160 milliseconds.
Infrastructure Optimization for AI Search
The shift to an in-house engine built in Rust highlights a growing trend among AI companies to optimize the lower levels of their infrastructure for speed and efficiency. Rust is often chosen for performance-critical systems due to its memory safety and execution speed compared to higher-level languages. By replacing an open-source fork with Photon, Perplexity gains more granular control over its ranking algorithms and retrieval processes. This vertical integration of the search stack allows the company to minimize the time spent on data retrieval before the information is processed by a large language model.
Unknowns and Implementation Details
While the performance gains are significant, the announcement did not specify which open-source engine Perplexity was previously using as its base. It is also unclear if Photon will eventually be open-sourced or remain a proprietary part of the company’s internal stack. Furthermore, the reports did not detail the specific hardware configurations or server environments used to achieve these latency benchmarks. The impact on the actual quality or relevance of search results—as opposed to just the speed of retrieval—remains to be seen as the system handles live production traffic at scale.
As AI search becomes more competitive, the transition to custom-built infrastructure like Photon suggests a focus on reducing the latency overhead inherent in large-scale retrieval systems.
Lead image: Brett Sayles / Pexels
