What does it take to deploy artificial intelligence at scale across one of the most complex, asset-heavy industries on earth?
For data scientists, machine learning engineers, and AI practitioners, theory is abundant, but actual production-grade case studies are rare. In a fascinating technical presentation, Lando de Weerdt from Shell’s Generative AI and Machine Vision team, shares insights on how a global energy giant is leveraging custom computer vision models, autonomous robotics, and generative AI to solve massive, real-world operational challenges.
If there is a question on how deep learning behaves when deployed in hyper-specialized, data-scarce industrial environments, this presentation is a must-watch. Here is a glimpse of what can be heard:
Machine Vision for Asset Inspection: From 3D Digital Twins to Subsea Domains
Industrial assets are vast. A typical facility can contain over 10 million meters of piping and more than a million distinct flanges. Inspecting this infrastructure manually for rust, corrosion, and structural anomalies is an immense, time-consuming logistical hurdle.
Shell’s solution is Autonomous Integrity Recognition (AIR). In the presentation, you will learn how the team orchestrates an end-to-end pipeline that takes thousands of drone-captured images, processes them via custom cloud-based machine vision models, and converts them into a 3D asset model.
The presentation dives into several cutting-edge extensions of this core inspection framework:
- AIR Compare: A spatial-temporal approach that tracks the precise severity and evolution of corrosion over time.
- Flange Face Inspection: Automating defect detection under the intense time constraints of facility turnarounds.
- CUI (Corrosion Under Insulation): Combining physical equipment properties, operational constraints, 3D modeling, and Natural Language Processing (NLP) of historical logs to predict a notoriously difficult-to-detect threat.
- Subsea Inspection: Transforming hours of live deep-sea video feeds into an intelligent, offline triage system that automatically flags structural debris and anode conditions.
Machine Vision for Operations: Robotics and Autonomous Rounds
For ML practitioners interested in edge deployment and robotics, the discussion around Operator Rounds by Exception (ORBEX) provides a compelling blueprint.
Instead of forcing human operators to conduct repetitive visual checks on pressure gauges, flow meters, and safety equipment, Shell deploys autonomous ground robots. These robots navigate facilities on pre-programmed routes, capturing high-resolution imagery and streaming it to the cloud.
Lando walks through the data pipeline: from raw image capture to the extraction of precise telemetry data (such as gauge readings and valve orientations), which is then mapped directly into an internal time-series database. For engineers, the highlight of this segment is the glimpse into Shell’s custom machine vision dashboard, designed to balance automated alerting with human-in-the-loop validation.
Overcoming the “Minority Class” Problem with Generative AI
Every data practitioner knows the pain of data scarcity. In a highly optimized industrial setting, dangerous anomalies rarely happen. For example, because Shell’s operators maintain equipment meticulously, the team has almost zero real-world data of failing, empty oil cups. How do you train a robust classifier when you have no negative examples?
This is where the presentation gets highly innovative. The team reveals how they leverage the broad internal knowledge of foundation generative models, fine-tuning them on specialized industrial data to synthesize rare anomaly classes. You will see the exact metrics on how blending this synthetic data yielded a 23% increase in classifier accuracy, offering a practical playbook for handling highly skewed, real-world data distributions.
Expanding the Horizon: From Smart Retail to Wildlife Protection
The scope of the team’s work extends far beyond heavy engineering. The presentation touches upon a diverse portfolio of unexpected computer vision applications:
- Automated P&ID Digitization: Transitioning away from fragile, hard-coded Python parsing scripts toward a flexible collection of vision models capable of mapping complex Piping and Instrumentation Diagrams into standardized data formats.
- Generative Branding Platforms: Overcoming the limitations of off-the-shelf text-to-image models by embedding custom color-template distance metrics directly into the loss function to enforce corporate style guides.
- Retail & Consumer Insights: Utilizing spatial footprints and planogram compliance apps to optimize gas station management in the emerging era of Electric Vehicles (EVs).
- Environmental Conservation: Implementing high-resolution bird species recognition near wind farms to temporarily halt turbines for protected species, highlighting the critical engineering challenge of detecting small targets from vast distances.
Beyond the sheer variety of use cases, this talk addresses the core architectural dilemmas that every senior AI practitioner faces:
- How to structure an internal AI team to serve upstream, downstream, and retail branches simultaneously?
- When to commit to the heavy R&D of building a custom in-house solution versus buying a vendor product and engineering the final 10%?
- How to handle severe environmental data constraints: from low-contrast underwater footage to microscopic anomalies?
Whether looking to optimize deployment pipelines, solve severe data imbalance using generative AI, or simply see how world-class machine vision is engineered for the field, this presentation provides invaluable, battle-tested insights.
This preview is part of our Premium Videos.
Sign up to see the whole presentation about: Machine Vision Applications at Shell
Presented by: Lando de Weerdt, Shell
