Francesco Gadaleta PhD is a seasoned professional in the field of technology, AI and data science. He’s the founder of Amethix Technologies, a firm specialising in advanced data and robotics solutions. He hosts the popular Data Science at Home podcast, and over his illustrious career he’s held key roles in the healthcare, energy, and finance domains.
Francesco’s professional interests are diverse, spanning applied mathematics, advanced machine learning, computer programming, robotics, and the study of decentralised and distributed systems.
Rust is a new programming language that’s causing a real stir in the data science community. Does this new language live up to the hype, and is it set to replace Python? In this post, Francesco investigates the key advantages Rust offers, and its potential limitations:
In a recent tweet, Elon Musk claimed that Rust is going to be the language of artificial general intelligence. Some people might think this claim is an exaggeration (to be taken with a pinch of salt, like many of his tweets), but in my opinion, Musk’s tweet confirms the speculation surrounding this amazing language.
It’s no longer a secret that Rust is a language of the future. My company, for example, has migrated most of the commercial projects from the usual Python stack (Python C, C++ stack) to Rust, solving many issues of code safety and efficiency. And that’s exactly the point of Elon’s tweet: Rust has many advantages over Python. In this article, I’ll evaluate the various uses of Rust and the ways it has the potential to overtake standard programming language.
When it comes to artificial intelligence and machine learning, Python has been the standard for about 20 years. In the fields of data science and machine learning, for example, many Python frameworks are available for data transformations and data manipulation pipelines, as well as for more sophisticated things like deep learning frameworks, computer vision and more recently large language models (LLMs).
But this is about to change. The latest big trend of large language models is to impose some form of optimisation when it comes to memory storage and compute, for example. And because Python isn’t the best language for efficiency and optimisation, developers have been working on alternatives.
A language called Mojo has recently been designed by Chris Lattner, one of the field’s most talented developers; Lattner is the inventor of Clang and Swift, and also the LLVM suite of tools for compiling from a high-level language to an intermediate representation (many of the low-level architectures that we are used to dealing with today). Lattner demonstrated that Python could be up to 35,000 times slower than compiled languages, and he created Mojo to rectify this.
Mojo hasn’t been released yet, so it’s not fully accessible to all. However, it’s already proving to be one of the best combinations between Python (or interpreted languages in general) and compiled languages. Mojo’s been designed specifically for AI tasks and AI workflows, and it features many capabilities that traditional computer programming languages don’t have.
Rust is another new language on the scene; having been the underdog for a while, Rust has recently become wildly popular, and I believe – as perhaps Elon doesthat Rust is set to be the best replacement for Python and Python workflows. Why? For one, Rust has key aspects that set it apart from standard languages: it’s compiled, rather than interpreted Python. It’s also a runtime, again, unlike Python.
COULD RUST BE THE FUTURE OF AI?
Rust does have its drawbacks: it’s a notoriously cryptic language (Python, on the other hand, is easy to understand). That said, there are more challenging languages out there: I don’t find Rust as difficult to deal with as Perl, for example. Even if you’re starting from scratch with Rust, I recommend persevering with it because there are many paradigms of programming in this language that are new, and will prompt you to discover new concepts about concurrency and safety.
Back when LLMs had yet to emerge and optimisation was not strictly mandatory (due to the reasonable size of machine learning models), we used to start working with Python (or another compact language), especially in commercial environments. We’d use Python to prototype a new idea and iterate it to work with the business. When everything seemed to work as expected, we’d migrate that code base from Python to a better language, usually a compiled one only for very demanding sectors.
But in the last few months, with the advent of LLMs, several billions of parameters are no longer an exception. This means optimisation has become a must, and we have to start with a language that’s performant enough to deal with these new billion-parameter models.
Data engineers encounter problems with Python whenever they want to increase the performance of their code and workflows. Code snippets often get thrown over the fence by Python developers. Consequently, data engineers have to resolve some very nasty optimisations to make the system work, as they usually do not deal with the machine learning algorithm, or whatever the NLP underneath is. In other words, they choose to trust the underlying NLP is built to perform, even though it’s usually not.
For example, docker containers tend to get used to containerise very badly listed dependencies. This makes life easier for the developer, but it’s a nightmare for the poor engineer who must deal with these massive containers. A pure computer scientist would try to eliminate that obstruction, and find a different solution: usually something increasingly close to the bare metal of the machine. And this involves compiling your code for that particular architecture; using so-called native binaries that are specifically optimised for each operating system and specific hardware. And this is where Rust excels. In fact, because Rust is born as a language that’s compiled, it can be optimised as needed. For the record, Rust is compiled to low-level machine code that is optimised for this or that hardware architecture. Such optimisation is left in the hands of rustc , the official Rust compiler.
There are several levels of optimisations with Rust. For example, using crates (the crate is the equivalent of a Python package or a C++ library) that are specifically designed for certain operating systems and hardware. Rust also provides security, portability, and performance. These are three features of the best programming languages you don’t want to ignore, especially for commercial projects.
One problem with Rust, however, is that safe code (code that, for example, doesn’t allow the developer to fall into the trap of nasty bugs like memory violation or buffer overflows) comes at the cost of having native binaries. It can be hard work to learn how to avoid these types of bugs, and to learn the computer programming paradigm specific to Rust. So, don’t expect to become familiar with Rust overnight.
Further, when using compiled languages, or native binaries executed on a system, they could potentially crash the entire system. You no longer have a Python script that runs into a runtime sandboxed environment. When working with Rust or C/C++ and compiling code into native binaries, it’s crucial to be aware that these executables can severely impact your system, potentially causing significant issues if there are bugs present.
There are also portability issues. Native binaries can be specific to a particular operating system; this can be a benefit, because of the optimisation part, but also a problem because you reduce portability by compiling, or using specific crates that are for that particular operating system. So, the problem of portability is a double-edged sword that can play in your favour, or against it.
When we weigh up the advantages and drawbacks of Rust, performance is the biggest elephant in the room: Rust is in fact one of the most performant languages currently available. It’s usually compared to C, and it’s actually proven to be better than C, especially in terms of safety and concurrency.
When it comes to the large language model application stack, there’s another important runtime for Rust applications, which is WebAssembly (WASM). There are several languages that can compile to WASM: C++, TypeScript, and of course, Rust. This combination between Rust and WASM is what I believe Elon Musk meant when he claimed that Rust is going to be the language of AGI.
Rust is in fact one of the most performant languages currently available.
I contend that Rust plus WASM is one of the best combinations when it comes to the large language model application stack.
When you consider the LLM application stack, you typically have agents connecting or connected to the internet. And of course, they receive asynchronous events. They can connect to databases, they can call web services around your cloud, or even out of your cloud. Rust and WASM provide a very interesting stack for high-performance agent apps, especially when it comes to a sync code, asynchronous code, and non-blocking input-output for high-density applications.
Going downwards in the LLM application stack, we move from the agent to the so-called inference layer. The inference layer tends to perform some of the most CPU-intensive tasks: pre-processing data to post-process numbers, for example, into sentences, or structured JSON data, and so on. In terms of pre-processing words and sentences – especially when it comes to LLM applications – that’s the inference layer where the CPU is doing the job of performing the prediction of the next wording, a text generative task, and so on. There are some great examples of Rust excelling here, the most famous being MediaPipe Rust, which is written Mediapipe-rs.
Finally, there’s the TensorLayer where all the GPUintensive tasks are passed from WASM to the native tensor libraries: for example, LLaMA, CPP, PyTorch. TensorFlow, and the famous MediaPipe Rust.
MediaPipe Rust is a Rust library for transferring MediaPipe tasks into WASM Edge and WASI-NN. It’s a very interesting piece of software because it’s a low-code API, very similar to the MediaPipe Python library. It’s very flexible because users can use custom media bytes as input. A host of tasks can already be supported by MediaPipe Rust: in particular, object detection, image segmentation and classification. There’s also gesture recognition, hand landmark detection, image embedding, face detection, text classification, text embedding, and audio classification.
Further, if you look at the README of the official repository on GitHub, you’ll see some interesting and accessible code snippets. Although you must know how to read Rust, the API is very clean, and there are very interesting examples of an image classifier, as well as an object detector, or a text classification that can be done with less than 10 lines of code of Rust. The complexity that these 10 lines of code of Rust can hide is impressive.
To conclude, Rust and WASM could be one of the best replacements for the Python ecosystem as we know it today. They integrate very well with CPU tensor libraries, as well as GPU tensor libraries, which are also written in C/C++ and Rust. They efficiently complete that stack from the frontend, or from an API for a developer down to the backend, which is already optimised. Also, Rust and WASM are more efficient in implementing application-specific pre and post processing data functions. All the tasks where inference is involved are obviously faster than Python, or a Python equivalent.
Further, they have container image sizes – if you ever want to use docker or docker containers – which are much smaller than the Python equivalent; usually several megabytes when you use a Rust-based docker image, for example. By contrast, Python-based images span several hundreds of megabytes. Rust and WASM are also safer objectively speaking, not only because of the language (which is known to be a safe language by design), but because of the impossibility of writing certain nasty bugs. In fact, the attack surface is much smaller than when you deal with a Python-based container where the amount of dependencies that you build or import in your docker is significantly bigger. Using Rust and WASM significantly lowers the risk of bugs, which in turn lowers the chance of sabotaging the entire workflow if you ever use that particular docker image.
Of course, the above point is increasingly superfluous now Rust and WASM are more efficient in implementing all the networking-intensive and long running tasks that are usually required for LLM agents. Remember, we’re moving from a standalone way of inferring and generating text (the ChatGPT we were used to in March this year), to something more complex: the concept of an agent that eventually can go online, download live resources, embed these resources live and generate a conversation using a dynamic context.
Thanks to prompt engineering techniques, the concept of the agent is becoming something that, from a computational complexity perspective, is increasingly complex. A more complicated infrastructure is therefore needed to serve these beasts now; these are no longer standalone models. And so again, Rust has all the properties that one would expect from a highly performant language which is, while not easy to learn, a powerful tool. Even fast cars are not easy to drive; there is a learning curve. But guess what? They go faster than all the others. So, yes, Elon, I think this time, you’re right. Rust is the language of the future.
“To conclude, Rust and WASM could be one of the best replacements for the Python ecosystem as we know it today.”
