Two Important Conversations
Recent remarks by Masayoshi Son, Chairman and CEO of top OpenAI investor SoftBank, described a future where 100 trillion autonomous agents replicate, improve, and surpass human cognitive capabilities at scale. His explanation suggests humans may no longer occupy the highest position within the hierarchy of intelligence. He also stated that the age in which humans are the highest form of intelligence is ending.
Simultaneously, voices such as Tristan Harris, a former Google design ethicist and the co-founder of the Center for Humane Technology. He is one of the most influential public voices on the societal risks of AI and emphasizes the risks associated with rapidly advancing systems. repeatedly argued that the United States and China must pursue stronger coordination to avoid a destabilizing race toward increasingly powerful AI systems. His concern centers on the possibility of the uncontrolled race of achieving Artificial Super Intelligence (ASI), especially following the latest news about OpenAIs release of GPT-6 Astra.
Their work focuses on alignment, international coordination, technological power concentration, and competitive dynamics that push humanity toward outcomes it cannot control. Both discussions carry weight. One addresses the trajectory of intelligence, while the other addresses the risks of reaching that destination. Like hundreds of voices, they start to react more thoroughly; both reveal a far broader reality.
The Question Beneath Artificial Intelligence
Assume the trajectory described by Son proves correct, with autonomous systems collaborating, creating new intelligence, and outperforming humans across cognitive domains. Assume international coordination also navigates immediate risks successfully. A fundamental question still remains regarding the kind of civilization that emerges on the other side. Public dialogue concentrates heavily on capability itself.
Comparatively little attention examines the conditions required for human civilization to remain coherent when intelligence becomes abundant, autonomous, and independent of human cognition. This inquiry concerns neither capability nor safety alone, but targets the fundamental conditions of human agency.
The Missing Layer of the Discussion
The growing debate surrounding artificial intelligence concentrates heavily on capability and safety. The first inquiry explores how rapidly intelligence advances and what occurs when machines outperform humanity across complex domains. The second explores how societies coordinate development, reduce risks, maintain stability, and prevent outcomes that undermine human interests. While both conversations matter, the emergence of machine intelligence introduces a third question that remains underexplored. 
Humanity must examine what happens to civilization when intelligence stops being society’s defining human advantage. The implications extend far beyond technology, affecting the foundations upon which societies organize participation, authority, responsibility, legitimacy, and collective decision-making. Human cognition provided the primary source of direction across society, with knowledge creating influence, expertise creating authority, and judgment creating responsibility. As intelligence expands beyond human capability, that foundational assumption changes. It now requires humanity to uncover how agency, sovereignty, identity, and accountability shall function within environments shaped by machine intelligence.
The Difference Between Events and Conditions
Transformations happen through shifting underlying conditions rather than isolated events. While visible innovations capture immediate attention, lasting significance emerges from changing structural conditions. Artificial intelligence follows this exact trajectory.
Current debates focus on performance, investment, competition, policy, and regulation to influence development speed. In this context, essential questions emerge when intelligence becomes abundant, when cognition embeds throughout global infrastructure, and when human beings interact with reality.
The Expansion of Intelligence
Framing intelligence strictly as a tool remains useful but less relevant in contrast to its impact. The emerging trajectory suggests intelligence is becoming infrastructure, espscially as recursive self-improvement capabilities increasingly escape any measures of human control.
The explosion of intelligence simultaneously influences every aspect of civilizational achievements. As intelligence integrates irreversibly into the operational fabric of society, it transforms the environments through which individuals participate and collaborate. These changes arrive through gradual shifts that become obvious only after conditions have permanently changed. Examples include surveillance or the massive use of AI in warfare.
The Shift of Agency
As machine intelligence expands, the relationship between humanity and intelligence evolves into one of the most significant shifts of the intelligent age. When intelligence resides within the primary systems driving society, defining the role of human agency becomes critical. The central challenge extends beyond creating capable systems to ensuring human beings remain empowered participants within environments shaped by greater intelligence.
Discussions about consciousness, model breakthroughs, safety incidents, and regulatory responses remain important because they help us understand the advancing frontier of artificial intelligence. This is keeping us occupied while a broader transformation may already be unfolding.
The emergence of increasingly capable intelligence slowly begins to change the conditions through which knowledge, decisions, institutions, and human agency operate. Understanding that transition may ultimately prove more important than determining whether machines ever become conscious in the human sense. This is not about assumptions; this is reality.
A Different Civilizational Question
While my previous book, A Billion Times Smarter, examined the trajectories, consequences, and scenarios surrounding the expansion of machine intelligence, my upcoming book, The Second Genesis (release date early October), begins from a different starting point. Instead of focusing on the capabilities of intelligence, it asks what civilization can become.
The shift in decision authority and agency is substantially impactful and happening at an uncontrolled speed. It involves an intelligence capable of outperforming human capabilities at large scale. It implicitly changes the creation of knowledge, decisions, systems, institutions, warfare, and how we individually and collaboratively perceive reality.
The rise of the intelligent age introduces a reality where analysis, optimization, discovery, and coordination emerge from systems exceeding the capabilities of their creators. An astonishing example is that AI and agents now create more than 50% of the content on the Internet. This trend will hold on, and humans will soon play a much less important role in the use of this important civilizational achievement.
Beyond the Technological Transition
For this reason, my upcoming book The Second Genesis approaches artificial intelligence primarily as a civilizational transition rather than a technological challenge. Its focus rests on the impact of this transition and the structural conditions required to empower and protect every individual and humanity at individual and collective scales.
Important questions of sovereignty, trust, identity, legitimacy, participation, knowledge, accountability, and institutional design become paramount as intelligence integrates into the operational fabric of society. Whether the future unfolds through the projections of Masayoshi Son or a different path, the relationship between humanity, intelligence, and agency is changing fundamentally. Understanding that change becomes vital, and designing the conditions through which civilization navigates it becomes essential.
Looking Beyond the Horizon
The definitive question facing society extends beyond whether intelligent systems continue to improve. The focus must turn to the world taking shape around that progress. Defining the institutions, trust models, legitimacy, and individual sovereignty required to sustain human participation requires looking past immediate technical breakthroughs, investment cycles, and policy initiatives. The future will be influenced by visible events, but the transformations occurring beneath them will ultimately define the architecture of civilization.
Closing Perspective
Intelligent systems advance at remarkable speed, driving public discussion toward breakthroughs, announcements, risks, and opportunities that clarify immediate changes. Alongside these events, a broader dialogue becomes essential to examine how intelligence embedded within daily operational environments alters participation, influence, and decision-making. The overarching challenge requires more than creating superior intelligence, demanding the preservation of meaningful human agency within a transformed civilization. Each upcoming edition in this series explores a central dimension of this underlying thesis, examining how deliberate system design can bridge the gap between machine capability and human sovereignty.
This series and my new book invite you to explore a deeper and more responsible way to respond to the changes we have not agreed to. How these changes will unfold is impossible to determine. To ignore the ongoing and escalating shift of agency is therefore an irresponsible reaction, especially as there are opportunities to avoid potential appalling developments.
This series introduces the core ideas of The Second Genesis, with each upcoming edition exploring a central dimension of the book’s underlying thesis.
Do you consider this as a challenge for humanity? Share your thoughts and leave a comment.
Are you interested in exploring the significance of agency and how we should respond to it? Join me in my upcoming article about the amplification of agency.
About the author
Ingo Paas is a board member at Svenska kraftnät and a former CIO/CDO with a proven track record of leading and executing enterprise-wide digital transformation across multiple industries.
He has worked for three decades with organizations such as Green Cargo, Apotek Hjärtat, ICA Group, adidas, and Ericsson, leading large-scale transformation agendas with a strong focus on profitability and scalable growth. His experience includes restructuring fragmented environments into coherent, high-performing operating models and effective,data-driven operations.
In his current board assignment, Ingo contributes to strategy, infrastructure, risk, and long-term value creation, particularly in areas where AI, technology, and innovation reshape investment priorities.
Ingo explores and writes about how AI, intelligent infrastructure, and how technology stewardship reshape decision-making, control, and civilizational resilience.
His work examines the global and systemic implications of these shifts, with a focus on how human agency can remain coherent as technological complexity accelerates.
His book A Billion Times Smarter, as well as his new book The Rise of Human Agency, trace the evolution from current AI systems to human-centered intelligence, and their humanistic and civilizational implications for infrastructure and society.
*The views and opinions expressed by the author do not necessarily state or reflect the views or positions of Hyperight.com or any entities they represent.
