The shipping sprint that exposed the real AI bottleneck

Vibe coding inside a large company is not just fast prototyping with AI. It is a temporary shift in how teams make decisions, share context, and move from idea to shipped software with fewer handoffs. In a month-long experiment at Automattic, small autonomous teams paused normal roadmap work and bui

MMichael SwindellJune 30, 202616 min read

Vibe coding inside a large company is not just fast prototyping with AI. It is a temporary shift in how teams make decisions, share context, and move from idea to shipped software with fewer handoffs. In a month-long experiment at Automattic, small autonomous teams paused normal roadmap work and built real products, showing how designers, engineers, and operators can all contribute in code when the environment supports it. The real lesson is organizational, not cosmetic. When documentation, security, and context access are strong, tools like agent-first development platforms and AI coding workflows can turn nontraditional builders into effective contributors much faster than most teams expect.

At a Glance

A 30-day shipping experiment exposed what actually changes when AI enters real product work.

Core insight: Speed came from autonomy, documentation, and shared context, not from AI alone.
Team shape: Small groups with clear ownership moved faster than larger handoff-heavy processes.
Role shift: Designers were able to build and ship code, while engineers became force multipliers and enablers.
Process change: Working prototypes replaced static design artifacts earlier in the cycle.
Org takeaway: Large companies can create this effect if they invest in training, safe access, and room to experiment.

What happened when 500 people paused the roadmap and built?

A company-wide shipping sprint revealed that AI can compress the distance between idea and production when teams are given autonomy. The most important outcome was not novelty projects. It was evidence that role boundaries, delivery habits, and collaboration models can change quickly under the right conditions.

The experiment ran for 30 days. Participants worked in small teams, paused normal roadmap commitments, and were asked to ship something real. Roughly a third of the company joined the first round, which translated to about 500 people across hundreds of projects.

That scale matters. This was not a single hackathon success story or a polished internal demo. It showed what happens when a large distributed organization creates temporary conditions for speed and lets many people test new workflows at once.

The broader lesson is that AI-assisted building looks very different in a mature software company than it does in a startup or solo side project. Legacy systems, security controls, internal platforms, and team habits all influence what “fast” actually means.

Why did the experiment work in a large, distributed company?

The results were possible because the organization had already invested in foundations that AI tools could build on. Good documentation, asynchronous habits, internal enablement, and secure developer setup reduced the friction that usually blocks non-engineers from contributing code.

Automattic’s distributed and asynchronous operating model created a useful advantage. Years of written decisions, concepts, and process documentation meant important context already existed in durable form rather than in meetings or tribal memory.

That matters for both humans and machines. AI coding tools perform better when they can reference stable, explicit context. People from outside engineering also gain confidence faster when systems are documented well enough to explore safely.

Another key factor was AI training. Employees were encouraged to learn and experiment early, then received more structured role-specific enablement. That meant participants entered the month with some exposure, shared vocabulary, and practical support instead of being told to “go use AI” with no system behind it.

What does this say about the real bottleneck in AI software delivery?

The main bottleneck is usually not code generation. It is organizational friction. Handoffs, unclear ownership, inaccessible context, and fragile setup processes slow teams down far more than typing code does, especially in large companies.

This is why simply adding an AI assistant to an unchanged workflow rarely produces dramatic results. If a designer still has to wait for an engineer to translate intent, if an engineer still has to reconstruct missing context, or if nobody can safely deploy, the tool does not remove the actual constraint.

In this experiment, participants gained speed because they could make decisions and act on them. They were not trapped in long approval loops or blocked by missing access. AI accelerated implementation, but autonomy unlocked the implementation path.

This aligns with broader industry signals. GitHub’s Octoverse has tracked the rapid adoption of AI-assisted development, but adoption alone does not guarantee throughput gains. The Stack Overflow Developer Survey 2024 also shows strong use of AI tools alongside ongoing concerns about trust, correctness, and workflow fit. The hard part is integration into real delivery systems.

How did designers start shipping code instead of stopping at mockups?

Designers moved faster because prototypes became the main working artifact earlier in the process. Instead of spending most of the cycle refining static visuals, they used AI tools to build functioning software first, then returned to visual tuning where needed.

That is a major workflow shift. In many teams, design begins in a tool like Figma and stays there until fidelity is high enough for handoff. Here, the order often flipped. The functional prototype came first, while visual polish and mood-setting came later.

This change matters because a live product answers questions that a mockup cannot. Search behavior, status logic, data connections, authentication, and user flow all become testable earlier. Feedback gets more concrete. Assumptions fail faster.

It also changes the designer’s role. A designer who can shape, build, and iterate a proof of concept directly is no longer limited to specification work. They become much closer to a design engineer, even if they started with limited production coding experience.

Why do engineers become more valuable as enablers, not just builders?

When more teammates can contribute in code, the highest-leverage engineers are often the ones who make others effective. Teaching setup, explaining Git, creating safe defaults, and structuring a project can multiply output more than personally writing another feature.

One early project in the experiment made this especially clear. A mixed-skill group built a working board game session manager in about two hours. The project itself was not the main lesson. The important part was that an engineer created the conditions for everyone else to contribute.

That included setting up the repository, teaching basic commands, clarifying version control, and reducing fear around making changes. Once the environment was understandable, the rest of the team could use AI tools productively instead of getting stuck at the edges of the workflow.

For engineering managers, this is a useful reframing. In an AI-enabled team, senior engineers often create the biggest impact by designing systems, guardrails, and onboarding paths that expand who can ship. Rakuten saw this play out at scale: by running multiple coding-agent sessions in parallel instead of one engineer per task, the company cut average feature delivery time from 24 working days to 5.

What kinds of products were built, and why do they matter?

The projects matter less as isolated features and more as examples of changing capability. They showed that nontraditional builders could move from idea to functioning software across internal tools, public-facing experiences, and multi-surface product concepts in days or weeks.

One project was a lightweight event session manager created in a short collaborative exercise. It included core session actions, participant management, chat, and a stylized visual treatment. The significance was its speed and shared contribution model.

Another project focused on a design system status tracker. It pulled signals from a GitHub repository, Storybook, and Figma, then organized them by status and library. Search worked across the resulting information set, and the output was deployed both internally and in a curated public-facing form.

A third project compressed the cycle even further. In six days, two designers built a working proof of concept for a WooCommerce merchant chat experience on iOS, including authentication, site connection, a customer-facing widget, and an AI-assisted answer flow that could decide when it did or did not have enough information to respond.

Context is what turns AI speed into team speed.

If your team is experimenting with coding agents, a shared control plane can keep plans, tasks, tests, and review in one place instead of scattering them across prompts and chat logs.

See how teams structure agent workflows

How did context management affect collaboration speed?

Shared context sped up both ideation and execution. When project thinking, chat history, and requirements lived in an accessible structure, collaborators aligned faster and AI tools had better material to work from during implementation.

One notable pattern was starting exploration in a project folder where discussions and ideas were recorded as files. That made decisions durable. It also created a reusable source of truth for later building, instead of leaving key reasoning trapped inside temporary chat sessions.

This is exactly where many teams hit a wall with coding agents. A prompt can generate output, but it does not provide ongoing project memory. Once multiple contributors, iterations, and tool calls enter the picture, the team needs something more stable than a conversation window.

One way to handle that is with an agent-first control plane such as Levr’s workflow model, where intent, tasks, automated verification, and human review live in a shared system. If you are coordinating Claude Code, Cursor, Codex, or Copilot across ongoing work, persistent context becomes infrastructure, not convenience.

Why did autonomy matter more than AI fluency?

Autonomy mattered because people could follow momentum instead of waiting for permission. AI helped with execution, but full ownership over decisions, scope, and iteration removed the negotiation overhead that usually slows large organizations.

That was especially visible in the design system tracker example. A proof of concept that might normally require extended discussion was built directly, tested against reality, and refined from there. Questions about maintenance or feasibility became easier to answer once the product existed.

In many companies, promising ideas die in coordination. By the time design, engineering, and operations all align, the original energy is gone. This experiment temporarily removed some of that institutional drag, which made individual initiative far more powerful.

Autonomy does not mean the absence of standards. It means standards are strong enough that people can move inside them without constant gatekeeping.

What process changes should large teams copy first?

Large teams should start by changing environment and behavior, not just tool access. Training, strong documentation, safe development setup, and explicit space for experimentation are more important than mandating any single AI product.

A practical rollout path looks like this:

Provide access: Give teams real time with AI tools, not just procurement approval.
Train by role: Designers, engineers, and product leads need different examples and workflows.
Strengthen setup: Make local environments, security controls, and deployment paths usable by non-experts where appropriate.
Document aggressively: AI and humans both benefit from durable context.
Create champions: Identify people who can teach and unblock others.
Protect experimentation time: Habits do not change if every minute is consumed by roadmap delivery.

Teams using shared agent-first features often formalize this further by connecting project intent to issues, test suites, and review gates. That makes experimentation easier to observe without turning it into chaos.

How do you give coding agents a real project control plane?

Coding agents work best when they can read more than a prompt. They need task context, acceptance criteria, prior decisions, and verification boundaries. A control plane provides that structure so generated work fits into an actual delivery system.

Without a control plane, teams end up with fragmented AI usage. Plans live in one chat, implementation in another, test expectations in a ticket, and review comments somewhere else. That is manageable for a solo experiment, but weak for a team.

One approach in Levr is to define intent in natural language, let agents pick up scoped work, run automated verification, and then require human review before merge or release. That mirrors the kind of disciplined speed the experiment surfaced: fewer handoffs, but not fewer checks.

If your workflow is moving toward multiple coding agents, Levr's agent network layer becomes relevant for shared memory, message passing, and routing. That is useful once work stops being a single-session interaction and becomes coordinated system behavior.

What are the biggest mistakes teams make when copying this kind of experiment?

The most common mistake is treating AI speed as a tool problem instead of a system problem. Teams copy the assistant, skip the training and context work, then conclude that the method is overhyped when the surrounding workflow never actually changed.

Other common failures include:

No safe setup: People are asked to contribute code without usable environments or clear security boundaries.
No written context: Important decisions remain verbal, which weakens both collaboration and AI assistance.
No teacher role: Strong engineers keep building, but do not enable others to build.
Too much fidelity too early: Teams stay in static mockups instead of testing live behavior.
No room to experiment: Existing roadmap pressure prevents process change.

According to McKinsey’s research on generative AI in software engineering, the biggest gains come when organizations redesign parts of the workflow around the tools instead of layering them onto old habits. See McKinsey’s analysis for broader context on where generative AI creates value and where operational changes are required.

Technical Deep Dive FAQ

What is vibe coding in a product team context?

In a product team context, vibe coding usually means using AI coding tools to move from idea to working software with far less ceremony than a traditional handoff-heavy process. It does not mean ignoring engineering discipline. In practice, the useful version combines fast iteration, lightweight planning, and direct building by people who may not all be full-time engineers. The term gets overstated online, but the underlying shift is real: more roles can now participate in implementation if the environment supports them.

What was Radical Speed Month trying to prove?

The initiative was designed to test how quickly real software could ship when normal roadmap work was paused and small teams were given autonomy. The larger question was whether AI-assisted tools, paired with strong documentation and organizational support, could remove delivery bottlenecks. The answer was less about flashy demos and more about capability transfer. People worked across role boundaries, shipped functioning products, and discovered that process constraints, not idea generation, were often the real source of delay.

Can non-engineers safely contribute production code in a large company?

Yes, but only if the organization has already built the right guardrails. That includes secure local setup, documented systems, review workflows, and clear deployment boundaries. Non-engineers do not need to become generalist software engineers overnight. They need constrained paths where contribution is safe, reversible, and observable. AI tools can accelerate learning and implementation, but they do not replace operational discipline. The important distinction is between broad unsupervised access and structured contribution inside a mature engineering environment.

Why does documentation matter so much for AI-assisted shipping?

Documentation reduces ambiguity for both humans and machines. In distributed teams, written context replaces the hallway explanation. For AI tools, it acts as retrieval material, grounding, and decision history. The better the documentation, the less time people spend reconstructing intent or rediscovering system rules. In the experiment, long-standing asynchronous practices created a strong base layer. That is a useful signal for other teams: AI works best when the organization already treats context as an asset rather than a side effect.

How does prototype-first work compare with Figma-first work?

Prototype-first work changes the sequence of learning. Instead of refining visuals until handoff, teams test interaction, data flow, and product logic earlier by building something functional. Figma still has a role, especially for visual direction, mood boards, and interface refinement. The difference is that it stops being the sole source of truth. This can reduce wasted effort because some design questions are easier to answer in a working interface than in static screens. It also shortens the gap between concept validation and implementation.

What role should engineers play when more teammates can code with AI?

Engineers become even more important, but the highest-leverage work often shifts upward. Instead of being the only people who can execute, they create environments where others can execute safely. That includes repository setup, architecture guidance, version control coaching, debugging support, and review standards. In mixed-skill teams, a strong engineer can multiply output by teaching and unblocking. This is especially valuable during transitions, when other functions are still learning how to work in code without creating operational risk.

How do you preserve project context across AI sessions and contributors?

Temporary chats are not enough once work spans days, people, and systems. Teams need durable artifacts such as structured project folders, issue histories, specs, test criteria, and logged decisions. Agent-first platforms can help by centralizing intent, tasks, tests, and approvals in one control plane. In Levr, for example, teams can connect project planning with execution and verification so coding agents are not operating from isolated prompts. The goal is continuity. A new contributor or a new agent session should inherit meaningful context immediately.

When does a team need multi-agent infrastructure instead of a single coding assistant?

Single-session assistants work well for narrow tasks. Multi-agent infrastructure becomes useful when work needs routing, persistent memory, or coordinated handoffs across repeated runs. Examples include planning, implementation, testing, and review happening as connected but distinct activities. That is where Levr's agent network layer is relevant. Shared memory, message buses, and intelligent routing are not necessary for every team on day one, but they become important as agent usage moves from ad hoc help to sustained workflow orchestration.

How can engineering managers run a similar experiment without disrupting everything?

Start with a bounded cohort, fixed time window, and clear success criteria. Choose teams with enough support to move safely, then temporarily reduce roadmap pressure so the experiment is not squeezed between normal commitments. Provide tool access, training, and strong documentation ahead of time. Focus measurement on shipped outcomes, process changes, and cross-role contribution patterns, not just lines of code. The point is to learn where your bottlenecks really are. A small, well-supported trial usually produces better insight than a broad mandate.

What should teams measure after an AI shipping sprint?

Measure output quality, cycle time, handoff reduction, and role participation, not just velocity theater. Useful signals include time from idea to prototype, number of contributors outside engineering, amount of rework after review, deployment reliability, and how often project context had to be reconstructed manually. Also look at enablement metrics: who taught others, what documentation was missing, and which tools created friction. These are often better leading indicators than raw feature counts because they reveal whether the organization is becoming easier to build inside.

Key Takeaways

The experiment showed that AI changes shipping speed most when it changes how teams work, not just how they type.

Large organizations can move faster if they redesign collaboration around context, autonomy, and safe contribution.

Speed source: AI mattered, but autonomy and documentation mattered more.
Role evolution: Designers can become builders, and engineers can become multipliers.
Process shift: Working prototypes should appear earlier than static high-fidelity mockups.
Infrastructure need: Shared context and verification are what make coding-agent workflows sustainable.

The strongest lesson is simple: if you want AI speed, redesign the system around learning and ownership.