Filipa Castro specialises in creating digital products and taking them to the next level with AI. She’s a data scientist at Continental, where she’s responsible for the development of the machine learning components of an AI-powered product for golfers. Prior to her role at Continental, Filipa was part of the Lead Team at Social Good Portugal and an AI Researcher at Abyssal.
In our latest post, Filipa examines the recent advancements in AI, and the impact these have for business. Using her company Continental as a case study, Filipa reflects on how businesses can navigate this new era to utilise AI’s full potential:
In November 2022, ChatGPT was released and made available to the public, marking a significant milestone in the world of artificial intelligence. Within a mere five days, the model had already amassed one million users, and within two months, it had reached an impressive 100 million users. This unprecedented level of success has positioned ChatGPT as one of the most successful products in history, catching many of us by surprise with its sudden rise to prominence.
Although ChatGPT has garnered significant attention, it is essential to note that it is merely a product that utilises Large Language Models (LLMs) technology. The rapid evolution of LLMs in recent years has undoubtedly contributed to the current revolution in the field. OpenAI, the company that owns ChatGPT, released the first LLM in 2016 and 2018. Other significant tech players, including Google, Microsoft, Meta, AWS -and more recently Hugging Face- have been announcing new research breakthroughs and releasing new LLM models to the world.
According to Sam Altman, the CEO of OpenAI, the usability of ChatGPT is what sets it apart from other models. While GPT-1, Transformers, and LaMDA were relatively unknown to the general public, ChatGPT’s accessibility has marked a new era in the field. ChatGPT is an excellent representation of the direction that AI is likely to take, with an emphasis on making it accessible and useful to anyone, regardless of technical expertise. OpenAI’s success in creating the most successful AI product ever by making it user-friendly and practical for the general public serves as an example and inspiration for other companies in different fields.
The rate at which technology is evolving can be overwhelming. In such a fast-paced environment, it is understandable that many companies may be struggling with how to leverage the full potential of large language models, where to invest the time and effort and stay ahead of the curve in this ever-evolving field.
There are various options available. Commercial tools such as the ones provided by OpenAI-now also available via Azure, Github Copilot, or Amazon Code Whispererare one such option. However, it remains unclear whether these tools are the ideal and only solution, or if developing our own models and utilising opensource options is a feasible alternative. Some of the questions that persist include data confidentiality, cost, responsibility, and provider-dependency.
Data confidentiality is a crucial issue. It is then important to consider whether the services being used share fine tuned models with other customers or use data to improve their own models. While some services claim not to do so, there are others that clearly state that data might be used for further product development. For instance, OpenAI’s initial free subscription included a disclaimer that user data could be used to improve existing models or answer other users. This has resulted in several instances where proprietary data, such as code, has been made available to everyone. OpenAI has since introduced the ability to turn off chat history which, according to them, prevents your conversation from being used for training their own models. In regards to privacy, hosting your own models in private servers brings the advantage that inferences can run locally, without the need for sending out your data to external services.
Regarding costs , it is essential to evaluate how much and in which context the company will use the technology. Software services, such as the ones provided by Microsoft or AWS, should be compared against deploying open-source models; which require paying for the underlying infrastructure. Such a comparison needs to be continuously assessed, given that both sides (paid services and open-source alternatives) are in continuous change. As an example, OpenAI switched from a free service to a paid subscription, and later announced significant price reduction, in just a few months.
Another crucial issue is responsibility for biased and wrong information. As deep learning models are still black boxes, they can learn incorrect and biased concepts from uncurated datasets, making them prone to hallucination and providing misleading information. Companies that integrate such models into their products or use them in development should be aware of who is responsible for such outputs and their consequent outcomes. A good practice is to start with use cases where we can directly evaluate the output of the model and easily identify any errors.
Lastly, provider-dependency is a consideration that needs to be addressed. Being agnostic is often a wise choice in technology, thus the same applies to LLMs. Relying on one product makes companies dependent on the model’s performance and prone to external service outages with little control. Running local models adds complexity, but it makes companies more agnostic, as such models can be deployed in any cloud provider.
Given the fast-paced nature of the technological landscape, policies and costs are continually changing. It takes time for companies to evaluate and mitigate risks, make decisions, and close contracts with service providers. That’s why, at Continental, we like to explore several alternatives: paid services and in-house model development. This approach allows us to move quickly and learn together.
In such a large organisation, with around 200,000 employees in 57 countries, there is a wealth of curiosity and expertise about large language models (LLMs). To share this knowledge, we have created an internal opensource initiative to combine efforts in learning about exploring, and deploying, open-source LLMs. We are also interested in fine-tuning these models to meet our specific needs.
We were able to quickly set up the initiative thanks to the existing toolset that our employees are familiar with: our own platform for open-source projects, tools for code versioning and collaboration, and communication.
As an example, we made use of our private social network to communicate about the initiative, which helped us to quickly grow a community of over 100 people in just a few days and almost 500 people in two months. People are invited to share the latest news and models on generative AI, but mostly to exchange their ideas and on-going user experience with use cases.
The initiative has been a great success so far. We have gained practical knowledge about the potential and limitations of LLMs and made significant progress towards deploying an internal GPT-like model. We are now able to fairly compare such an approach with the available services out there, both in terms of cost and performance. This makes us more knowledgeable, more responsible, and mindful as an organisation.
Once again, this was unlocked by the existing technology stack and in-house AI expertise. This means expertise from hundreds of experts in data science and AI, as well as the required tools ready to use for model training and evaluation, data versioning, experiment tracking and model comparison, deployment of models in several cloud providers, among others.
After the first deployment was made available for a small group of people, the overall interest started to increase as well as the exchange and learning experience from using it. The first challenge was clear: scaling the application would be required to make it available to several users simultaneously without hurting the user experience. We worked with our software architecture experts to build a scalable and elastic solution. This experiment allowed us to measure costs and usage, and to study what setup really makes sense for each use case.
The second challenge is data privacy . As in any big organisation, there is some data that only certain teams can access. This realisation led us to the key idea of having separate sessions, models and knowledge bases for different users.
Naturally, new requests for new features came from the community. The most popular ones are Question & Answers (Q&A) for documents and databases, content generation, and coding assistance. In parallel, it’s important to work together with legal, compliance and cybersecurity teams, among others. We must also be mindful and discuss the expected impact of this and similar tools in the workplace.
