Data Innovation Summit turns five next March. Along the way, we have had fantastic speakers unselfishly sharing their knowledge on stage with their peers. Without them, this journey would be impossible.
This interview is part of an interview series dedicated to humanising Data and AI innovation and celebrating speakers who have presented at the Data Innovation Summit. The emphasis lies on the Data/AI people/practitioners, their professional journey and their stories.
Reinforcement learning (RL) is one of the three machine learning paradigms which is all about teaching an agent how to act and interact with the environment in return for the desired reward. It has found many practical applications, one of which is FinTech for the purpose of debt collection.
Jorge Davila Chacon discussed the RL application for debt collection at Data Innovation Summit 2017. But a time-lapse of almost 3 years is a lifetime in the machine learning world. This was the reason we invited Jorge to talk about recent developments in AI and ML, and particularly in reinforcement learning, not only from a technical side, but also from a social aspect of the adoption of these technologies.
Hyperight: Hi Jorge, we are glad to have you with us today and have the chance to catch up. You were a speaker at Data Innovation Summit 2017. To refresh our memories and introduce yourself to our readers, please tell us a bit about yourself and the company you are coming from.
Jorge Davila Chacon: Hi Ivana, thank you for your invitation to share our thoughts about recent developments in AI and our expectations for the following decade. Ten years ago I started studying AI in the Netherlands with a focus on Cognitive Robotics and afterwards obtained a PhD degree in Germany in the field of Computational Neuroscience. Five years ago I began working in a company-builder form Otto Group, where my job was to design machine learning (ML) systems for various startups, and in 2018 I co-founded Heldenkombinat Technologies (HKT) together with an amazing business partner, Olaf Erichsen. In HKT we implement ML solutions for the industry and also Olaf and I are lecturers at the Hamburg School of Business Administration.
Hyperight: Next year we are celebrating our 5th anniversary. A lot has changed with data and AI during these 5 years. From your point of view, where do we see the biggest changes and advancements we have had?
Jorge Davila Chacon: Indeed, a lot has happened in the last 5 years! On the technology side, algorithms have improved in practically all areas of ML. Better optimization methods started helping algorithms to learn with fewer data and generative models improved tremendously. Like StyleGAN, that generates shockingly realistic human faces. On the industry side, the release of pre-trained models by the big players in IT made it possible for companies that have access to much smaller datasets, to adapt such models to all sorts of applications with the use of transfer-learning. Even more importantly, the constant evolution of high-level software libraries increased the adoption of ML in many companies, as it is no longer necessary to be a niche expert to start using ML.
Hyperight: Probably you’ve gone through some changes on a professional level yourself. Could you share how your career path has advanced during these 5 years?
Jorge Davila Chacon: In the last five years, I have had the opportunity to work in various industry sectors applying ML to text, images, audio and planning. Some of the most interesting learnings, however, came from the social aspects related to the adoption of technology in corporations. The love-hate relationship that media has with AI and the obscurity of the technology itself has created an environment where many executives are deeply curious about the possibilities that ML has to offer, while at the same time being particularly cautious before venturing into the use of ML. This scepticism can increase when managers realize for the first time that the implementation of ML systems is an iterative process with new types of uncertainties. First, one has to explore the data, understand its quality and the complexity of the patterns that the algorithms are required to learn. Afterwards, it is necessary to asses the amount of data, the algorithmic architecture and the training time required to achieve the desired performance. For these reasons, ML projects can have less precise estimations than more traditional IT projects. As you see, there is a chicken-egg problem here: one needs access to data in order to provide accurate estimates about the requirements of a project, while at the same time, one needs to provide good estimates in order to be granted access to the data. Nevertheless, once companies become more familiar with the typical evolution of ML projects, they usually become more open to further developments.
Compared to other ML subfields, RL has its own type of strengths and challenges when it is used in the industry. It is a powerful tool to complete “pictures” of the environment where it is deployed.
Hyperight: In your presentation, you talked about reinforcement learning and its application in debt collection. You also referred to reinforcement learning as one of ten breakthrough technologies of the year in 2017. Where are we with reinforcement learning almost 3 years later?
Jorge Davila Chacon: Compared to other ML subfields, RL has its own type of strengths and challenges when it is used in the industry. It is a powerful tool to complete “pictures” of the environment where it is deployed. For example, sometimes when the data is incomplete and heterogeneous RL can be used to fill in the blanks. The idea is to obtain the missing information by interacting with the end-user, sending a message and observing the reaction. In this respect, RL can be seen as the “brain” of an artificial agent, but it is crucial to understand that the “body” of such agent is indispensable and determines what can be learned from the environment. Such a body requires actuators that can modify the environment (like hands that can throw objects or communication platforms that can send emails) and sensors that can detect the changes that the agent’s actions produced in the environment (like eyes to see what happens when the hand throws a ball towards a basket or a cloud service that tells the agent when and how often an end-user opens its web account).
Often it is the body, rather than the brain, which requires the largest effort to design and build. This is what means to have an AI-centered project, rather than simply sprinkling an existing project with ML. Certainly, it is a more expensive investment but in the long run, it will pay back with considerably larger returns.
Another challenge for RL comes when managers understand that RL algorithms are not very well suited to learn from simulations and need to be trained by interacting with real users. Not every company is willing to give away a certain percentage of their revenue-generating users to train obscure algorithms and with good reason. However, it is useful to notice that even if people understand that algorithms are working within constrained decision boundaries, they feel the need to have transparency at the deepest levels. Imagine if part of the recruitment process of human candidates would include an open-brain surgery to understand how the neural networks of the applicant are connected!
Typically, understanding that such candidate has a proven track record in increasing company revenue is enough and, to some extent, RL agents can safely be assessed with the same approach. Ultimately, even if we may not be experts in car engines, we still like when the sales agent opens the hood to show us the complex machinery inside. For this reason, researchers are still working towards data-driven RL to facilitate learning from models of the environment and to minimize the amount of real-world data required to train the algorithm. In the case of the RL project that I presented with you in 2017, we designed the agent in a modular way that allowed us to test different RL variants and to compare them against the performance of human operators.
Furthermore, we were also constraining the families of algorithms that we used to those that allow the most transparency, such that once an agent would reach high performance, we could look inside and understand not only what works best, but also why. The most transparent algorithms tend to be the simplest ones, and we are already seeing progress in making more transparent the more complex types of RL algorithms.
Hyperight: Talking about the decade to come, what are your predictions when it comes to AI and reinforcement learning in particular for 2030?
Jorge Davila Chacon: Certainly, we will see considerable progress in the technology side, both in the algorithms and in the computing power accessible to more people. However, it is also exciting to foresee that a larger proportion of executives will be data-ready and familiarized with the requirements of ML projects. In the past decades, when robotic systems started being adopted in the manufacturing sector, managers were aware of the complexity behind such technology, but trusted external experts and opened their doors to let them understand their internal processes. In this way, such experts were able to determine the optimal points to implement automation, which could be different from the ones initially thought by the manufacturer.
In the case of ML, instead of automating repetitive hand-labour with robotic hardware, we are automating repetitive mental-labour with robotic software. This means that corporations can be more reluctant to reveal internal, more sensitive, processes to external service providers and place the burden of determining the points of automation on the shoulders of internal managers. Understandably, managers often do not have the necessary expertise to determine the points that will bring the highest return on investment. Nevertheless, we see many students in the business school that are very excited about understanding these new technologies and we expect to see an acceleration in the adoption of ML in the following decade. As Brynjolfsson and Mcafee nicely put it in their 2017 HBR article, “Over the next decade, AI won’t replace managers, but managers who use AI will replace those who don’t.”
Add comment