Shopify Engineer Shares Practical Lessons from Building Multi‑Agent AI Systems

Paulo Arruda, a staff engineer at Shopify, outlined at QCon AI the operational design and rollout lessons his team learned while moving from GPT‑3.

Paulo Arruda, a staff engineer at Shopify, used a QCon AI presentation to set out practical lessons from building multi‑agent AI systems and explain why design and deployment choices matter for production reliability and developer uptake. He traced Shopify’s internal arc from early GPT‑3.5 experiments to the broader availability of multiple providers in 2024, and argued that operational choices — not just models — determine whether multi‑agent systems move from prototypes into regular use. This matters for engineering teams seeking reproducible, maintainable orchestration patterns that scale beyond ad‑hoc prompts.

Arruda described the concrete prototypes and tools Shopify used while experimenting: chat tools and developer interfaces including LibreChat, VSCode Copilot, and Cursor, together with orchestration artifacts such as Claude Swarm (noted as having 1.4k+ GitHub stars) and its successor SwarmSDK. He presented those projects as hands‑on building blocks that helped the team prototype agent orchestration and evaluate different interaction patterns among multiple AI agents, rather than as finished production platforms.

Real adoption, he said, faced practical resistance. Many engineers lacked time to trial new tooling or were discouraged by poor first experiences with earlier GPT‑era models, generating internal skepticism. Shopify’s hacker culture and executive encouragement gave engineers room to prototype, but day‑to‑day uptake lagged until tooling and workflows matured. A key engineering pivot Arruda highlighted was moving away from monolithic, all‑in‑one prompts toward treating agents as narrow, specialized microservices. That change, he reported, reduced some end‑to‑end task runtimes from hours to minutes in internal experiments and made orchestration and failure handling more tractable for builders.

To address context bloat, Arruda proposed filesystem‑based adapters as a working hypothesis: have agents persist and share state through structured files instead of inflating prompt context, allowing focused models to operate on smaller inputs with clearer contracts. He stressed this as a forward‑looking idea rather than a finalized pattern, one intended to constrain input size and simplify inter‑agent contracts while keeping model calls lightweight and auditable.

On practical safeguards and next steps, Arruda recommended investing early in testing and orchestration: use test generation to catch AI‑originated regressions (an initiative he began exploring around October 2024), favor platform and SDK support for orchestrating agents, and adopt a pragmatic stance that AI should augment developer workflows rather than replace them. His takeaway for builders was straightforward: prototype quickly, keep each agent’s scope narrow, and prioritize testing and orchestration from the start to convert prototypes into reliable production components.