A recent paper published on arXiv cs.AI details a randomized controlled trial investigating whether conversational AI tools genuinely save time in digital product design. As large language models increasingly support prompt-to-design capabilities, both designers and non-designer colleagues can now build prototypes through conversational workflows. To measure these prospective efficiency gains, researchers tested 50 product designers and 50 product managers as they attempted three standardized design tasks using Figma Make.
The Growing Focus on Conversational Prototyping
While software engineering has seen numerous empirical studies measuring developer productivity when using artificial intelligence, equivalent controlled trials for digital product design workflows remain sparse. Conversational interfaces integrated into design software aim to bridge the gap between initial concept and functional prototype, potentially allowing cross-functional team members to generate visual layouts without deep technical skills. This experiment attempts to bring empirical rigor to claims surrounding AI-driven efficiency in creative and structural product workflows.
What Remains Unclear
Despite the execution of the trial with 100 total participants, the specific outcomes, performance metrics, and exact time differences observed during the three standardized tasks were not detailed in the available summary. The announcement did not specify whether product managers derived a greater advantage from conversational workflows compared to professional product designers, nor did it disclose how design quality was measured alongside task completion time.
We will continue to monitor the research as further data from these design trials becomes publicly available.
Lead image: panumas nikhomkhai / Pexels
