Every data scientist dreams of the moment their model hits production. But what happens when that dream turns into a 300% error rate nightmare?
At a recent industry talk, Abdullah and Miguel, two data scientists from Xeneta, the world’s leading ocean and air freight benchmarking platform, shared a refreshingly honest look at their journey through the highs and lows of deploying AI in the volatile world of global logistics. Their story is more than successful algorithms, it’s a masterclass in the unattractive side of AI: the planning, the failures, and the eventual redemption.
Abdullah shares the first act of the story:
“How Xeneta’s biggest failure has evolved into their minor success and later to be their major success”.
Act I: Excitement That Led To The Crash
This Act involves three main parts with one being the plan, which, to them, was optimistic and excited, then the second part: the build, where the morale was high and the third one: the crash, where devastation led to growth.
The presentation kicks off with a cautionary tale involving a character named Brandon (a “lemon farmer” used as a clever analogy for their real-world stakeholders). The mission seemed simple: predict the price of “iced lemonade” based on the abundance of “warm lemonade” data.
As junior data scientists, the team was fueled by delusional optimism. They followed the standard playbooks and even achieved a staggering 5% mean absolute percentage error (MAPE) in the lab. They were ready for glory.
But then, the crash happened.
Within months, that 5% error exploded to 300%. The culprit was a massive shift in market dynamics that their model simply wasn’t built to survive. In the video, Abdullah breaks down exactly how they missed the warning signs of data drift and why their “dirty” Slack-based monitoring solution failed to save them.
This is one of the best examples of data drift.
Act II: The Path to Redemption
Fast forward a year, and the team faced a new challenge: forecasting the actual shipping container market. This time, they weren’t going in blind. They swapped their notebooks for whiteboards and their optimism for pragmatism.
Miguel takes over the narrative to explain how they rebuilt their strategy from the ground up. This wasn’t about using the flashiest tools on the market; in fact, they discarded industry-standard tools like MLflow because they were “overkill” for their two-person team. They ended up building something leaner, faster, and more effective.
Strategic Insights
If someone has struggled to explain model decay to a stakeholder or wondered how to build a robust MLOps pipeline on a tight deadline, this talk will be of help. Abdullah and Miguel dive deep into:
- The “Life After Research” Philosophy: Why the job isn’t done when the model works in a Jupyter Notebook.
- Pragmatic MLOps: How they used S3 as a “dirty but effective” model registry to keep things moving.
- Building Trust through Transparency: Moving to automated dashboards that stakeholders actually understand.
- The Power of Retraining: The specific architecture they used to tackle concept drift before it destroys ROI.
The shipping industry, with its 700 million contracts and constant global disruptions, is one of the hardest environments for AI to survive. This presentation is a rare look behind-the-curtain at how a leading tech company failed fast, learned faster, and eventually mastered the art of time-series forecasting.
You can watch the full video to see the specific technical frameworks and cultural shifts that turned a “lemon” of a project into a cornerstone of Xeneta’s AI strategy and explore all the videos before the upcoming event: Data Innovation Summit.
