Building Autonomous AI Agents for Real: Lessons from Henrik Kniberg

Henrik Kniberg (Abundly.ai, ex-Spotify/LEGO) details the shift to autonomous AI agents, viewing them as “digital interns.” He shares the Abundly Thesis on building agents, human-AI collaboration, and 95% productivity gains.
Can AI Agents Be Trusted? The Limits of Autonomy in Artificial Intelligence Can AI Agents Be Trusted? The Limits of Autonomy in Artificial Intelligence

In the fast-evolving world of artificial intelligence, few voices bridge the technical, human, and organizational as elegantly as Henrik Kniberg. Known for shaping how the world understood agile through his famed “Spotify model” whiteboard videos, Kniberg has again stepped into a formative moment, this time at the frontier of autonomous AI agents. As Chief Scientist and Co-Founder of Abundly.ai and author of “Generative AI in a Nutshell,” Kniberg is helping define how we transition from chatbots to truly autonomous digital colleagues.

His conviction is simple but profound: AI agents are not a futuristic abstraction. They’re already here. And building them for real means blending engineering discipline with human empathy, organizational design, and an entirely new mindset for how teams work. On last night’s 165th episode of the weekly AI After Work (AIAW) Podcast, together with Robert Luciani and Henrik Göthberg, he managed to guide us through the details of how to build these agents and more.

From Agile Playbooks to Agentic Reality

Henrik Kniberg’s career reads like a guided tour through modern digital transformation. Spotify, LEGO, Mojang (Minecraft), and now Abundly. At each stop, he has translated complex systems thinking into simple, actionable principles. Agile teams, he once said, thrive when they organize around autonomy and purpose. Now, he argues, the same logic applies to AI Agents.

Kniberg’s journey into agentic AI began as curiosity. After years in product coaching, he found himself captivated by GPT-4’s leap in contextual reasoning. What began as tinkering with chat models quickly turned into an exploration of autonomy. At his cabin, he began building agents that could act, remember, and adapt – one of which, an in-game bot named Eggbert, became his first living experiment.

Eggbert was no ordinary script. It remembered past interactions, cracked jokes, and coordinated across Minecraft and Discord, gossiping about player antics as if it were part of the gang. It wasn’t alive. But it behaved as though it were. “For all practical purposes, this thing was passing the Turing Test,” Kniberg recalled. “It was both hilarious and profound.”

That experience foreshadowed the founding insight behind Abundly.ai: AI agents don’t just augment human workflows. They occupy a new middle space between code and colleagues. They are faster and more consistent than humans, yet more flexible and creative than code. Building them well demands a balance of technical architecture, organizational psychology, and product design.

Are AI Agents Coming Next Year – What it Means for Us?

The Abundly Thesis – Where Agents Live and Work

Abundly emerged not from a business plan but from a revelation: agents need a place to live. Every agent, Kniberg realized, has a life cycle: birth, onboarding, learning, and continuous adaptation. Without a coherent environment to manage that, companies were re-creating the same scaffolding from scratch.

“People need agents, but agents need somewhere to live,” Kniberg explained. That somewhere became the Abundly platform: an operating system for AI agents inside organizations.

In Abundly, each agent begins as a blank slate. A digital intern. The user describes the agent’s purpose (“Check our GitHub and flag potential security issues”). The agent then asks questions: What permissions do I have? How should I contact you? Slack or email? It writes its own instruction manual, updates it after feedback, and over time, becomes a dependable collaborator.

Kniberg’s preferred metaphor, “the AI intern”, is deliberate. It captures the psychological shift leaders must make. If the agent behaves unexpectedly, that’s not a failure of AI; it’s a failure of management. As Kniberg puts it, “If the intern does something dumb, it’s probably your fault. You didn’t give the right context or feedback.”

That humanistic framing contrasts sharply with traditional automation thinking. Most enterprise tools model work as linear workflows. A predictable sequence of steps. But as Kniberg notes, real organizations are nonlinear, dynamic, and full of exceptions. Agentic systems, by design, thrive in that ambiguity. They don’t execute scripts; they pursue goals.

The Agentic Frontier — From Code to Colleague

To understand why agentic systems matter, Kniberg draws a mental continuum. On one end: deterministic code, which is fast, reliable, and utterly unintelligent. On the other hand, humans who can be slow, creative, and deeply contextual. Agents occupy the middle ground.

They are not fully predictable, but they are consistent enough to trust. They are not conscious, but they demonstrate reasoning-like behavior. And critically, they can be taught.

At Abundly, the development process mirrors coaching rather than coding. Engineers and domain experts co-create prompts, iterate on behaviors, and refine the agent’s decision logic through live feedback. Over time, the agent internalizes context much as a junior hire would. The result isn’t just automation. It’s a collaboration.

That human-in-the-loop principle underpins every Abundly deployment. Kniberg rejects the notion of fully autonomous systems operating without oversight. Instead, he envisions teams where humans and agents co-own tasks. Humans retain accountability; agents handle diligence and execution. It’s not a replacement. It’s amplification.

From Experiments to Enterprise Use Cases

Kniberg’s early proof points came through experimentation, including with a Swedish broadcaster, where Abundly helped develop an agentic co-worker for media research. But the company’s most telling example came from a private investment firm.

The firm faced a repetitive, error-prone task: reviewing thousands of potential portfolio companies each year. Analysts spent weeks classifying prospects into yes, no, or maybe categories. Abundly’s team co-designed a “screener agent” that replicated their reasoning criteria. After a few rounds of iteration, the agent could analyze the entire dataset in minutes, surfacing recommendations for human review.

The impact was startling: a 95 percent reduction in manual time, but also an unexpected rise in quality. When the analysts compared the agent’s classifications to their own past decisions, they found that, more often than not, the AI had been right.

“The agents weren’t smarter than the humans,” Kniberg clarified. “They were just more diligent. They never got tired. They read everything.”

This pattern, the time savings coupled with diligence-driven accuracy, is now repeating across industries. Abundly has built agents for research synthesis, quality assurance, support ticket triage, and compliance monitoring. Each case follows the same arc: small, low-risk pilots that evolve into trusted digital colleagues.

Industry analysts echo this trend. Gartner’s 2025 Emerging Tech Hype Cycle places AI agents at the inflection point of productivity transformation, projecting that by 2027, over 40 percent of enterprise software interactions will occur via autonomous agents rather than direct UI navigation. McKinsey’s mid-2025 report similarly highlights agentic systems as a top driver of AI productivity compounding, where individual efficiency gains multiply across organizational layers.

AI Agents: Are They Becoming the Digital Genies of Tomorrow?

Teaching the Interns. A New Managerial Skill Set

The rise of agentic AI redefines what it means to be a senior engineer or leader. Kniberg notes that many experienced developers initially struggle. Their instinct is to treat AI as a tool. To control it with code rather than lead it with language. But the most effective practitioners act more like managers or coaches.

“You might be a senior developer, but if you don’t know how to give instructions, you’re a beginner again,” Robert Lucianni observed. Prompting, Henrik argues, is leadership: defining goals, providing context, and iterating based on performance.

That framing aligns with a broader shift in AI literacy. As LLMs evolve from assistants to actors, organizations must cultivate what researchers at Stanford’s Institute for Human-Centered AI call agent management literacy, which is the ability to design, delegate to, and supervise autonomous systems. It’s a blend of prompt engineering, governance, and team dynamics.

Abundly codifies this through structured onboarding rituals. Each agent is created by a human owner, given goals, tools, and permissions, and then trained through iterative feedback. Over time, it becomes a participant in the team’s Slack channels, ticketing systems, and project reviews. A co-worker, not a black box.

This design philosophy helps avoid the common pitfall of AI theater, and that is the temptation to deploy technology for novelty rather than impact. Real value, Kniberg insists, comes when agents are embedded in the team’s rhythm, measured by outcomes, and continuously improved.

Building for the Messy Middle

Kniberg often reminds clients that organizations are not perfect workflows rather living systems. Decisions are nonlinear, responsibilities overlap, and authority is distributed. Trying to build perfect automation on top of that chaos often fails because reality is inherently messy.

Agents, however, can thrive in the messy middle. Unlike traditional automation, they can handle exceptions, negotiate ambiguity, and adapt on the fly. This makes them ideal for bridging gaps between rigid systems and flexible teams. “You can’t RPA curiosity,” Kniberg quipped. “But an agent can explore edge cases and learn from them.”

In this way, the agentic revolution mirrors the original agile movement: a shift from rigid process control to adaptive learning systems. Where agile teams iterated toward better software, agentic teams iterate toward better intelligence.

Still, the implications go far beyond engineering. Organizationally, companies must rethink accountability, ethics, and even governance. If agents can make recommendations or take actions autonomously, who owns the outcome? How do you audit decision chains that involve both human and synthetic reasoning?

Kniberg advocates a principle of co-responsibility. Agents should never operate without a human counterpart. Every task has a designated human owner who remains accountable for outcomes. This structure ensures traceability while enabling speed.

The Future Workplace: Co-Workers, Not Code

For Kniberg, the most exciting frontier isn’t the technology itself—it’s the culture that surrounds it. The organizations that will thrive, he believes, are those that treat agents not as automation scripts but as collaborators.

In this emerging paradigm, every team might include a handful of digital colleagues. One agent handles market scanning. Another summarizes daily Slack threads. A third drafts compliance reports or coordinates releases. Each is autonomous within defined boundaries, constantly learning from feedback and interacting with other agents and humans alike.

This future isn’t speculative. It’s already visible in high-performing teams across design, software, and research. At Abundly, the internal culture reflects this philosophy: humans and agents share dashboards, task boards, and retrospectives. Kniberg describes it as a new kind of organizational fabric—one where delegation flows naturally across biological and digital intelligence.

But there is still art in the craft. As Kniberg emphasizes, “Building agents for real means respecting both the power and the limits of AI.” Agents don’t replace human judgment, empathy, or creativity. They amplify them. The goal is not to eliminate work but to eliminate friction.

Beyond the Hype: Pragmatism and Possibility

The excitement around AI agents in 2025 rivals the early days of mobile apps or cloud computing. Every week brings new frameworks. From OpenAI’s o1 series to Anthropic’s expanding tool-use capabilities, and new promises of autonomous productivity. Yet participants in the episode remain pragmatic.

“Every time people say, ‘This is it, we’ve reached the next level,’ I think: we’re still just beginning,” Henrik said. “It’s not about hype cycles; it’s about building things that actually help people work better.”

That perspective echoes through Abundly’s philosophy: start small, stay curious, and build for learning. As enterprises race to implement AI copilots and autonomous workflows, Kniberg’s approach offers a counterweight, one grounded in empathy, experimentation, and human agency.

The agile coach who once taught teams to trust small, empowered units is now teaching organizations to do the same with AI agents.

A New Era of Collaboration

As we enter what many now call the “Agentic Era,” the lessons from Henrik Kniberg and Abundly resonate far beyond software development. They touch leadership, design, ethics, and organizational identity. Building autonomous agents for real is not merely an engineering challenge; it is a societal shift in how we define work and intelligence.

Kniberg’s message is both humble and hopeful: embrace these AI systems as co-workers, not competitors. Learn to lead them as you would a talented but inexperienced teammate. Build environments that let them thrive. A safe space to learn, fail, and improve.

In doing so, we may discover that the most profound outcome of the AI revolution isn’t artificial intelligence itself, but the emergence of augmented humanity.

Ten Steps to Building Real AI Agents

Drawing on Henrik Kniberg’s experience, these ten steps come to my mind on how organizations can move from ideas to functioning autonomous agents.

Step 1. Start with a simple, valuable problem. Choose a task that is repetitive yet cognitively rich enough to demonstrate agentic value, such as triaging support tickets or scanning reports.

Step 2. Define the agent’s purpose clearly. State what success looks like in plain language. Agents thrive on clarity and context, not abstract metrics.

Step 3. Create a safe environment. Build your agent inside a sandboxed workspace where it can test, fail, and learn without disrupting live systems.

Step 4. Assign human ownership. Every agent needs a responsible human counterpart who provides feedback, oversight, and continuous training.

Step 5. Establish communication channels. Let the agent integrate with your existing collaboration tools like Slack or Teams. The goal is seamless inclusion in daily work.

Step 6. Iterate through dialogue. Treat training as a conversation. Correct mistakes through prompts and reflection, allowing the agent to update its own guidance over time.

Step 7. Measure diligence, not brilliance. The greatest value often lies in consistency and persistence, not creative spark. Agents excel at unrelenting thoroughness.

Step 8. Combine human and machine intelligence. Design workflows where agents handle the heavy lifting and humans refine insight, creating a reinforcing feedback loop.

Step 9. Scale through templates. Once an agent works well for one use case, abstract its structure into a reusable pattern that others in the organization can adopt.

Step 10. Keep improving through feedback. Agentic systems are living systems. Continuous learning, refinement, and transparency sustain long-term value.

Extra step if in doubt: Go to Abundly.ai and go through their Agent Design Canvas, which is a practical tool created by the Abundly team to help you systematically design AI agents, or contact the team for more info and assistance.

To listen or view the full episode, go to www.aiawpodcast.com

This article was enhanced with the help of AI tools, drawing on the podcast transcript, complementary online research, and my own perspectives. To go deeper into the source material, I encourage you to listen to the full episode and make your own learnings.

Add a comment

Leave a Reply