uv: Towards a unified vision for Python tooling

A long-term Python user’s perspective on getting productive with uv#

As many readers of this blog will know, I’ve been coding in Python for what feels like forever, having written my first line of Python in 2009. Over the years, I’ve seen and used countless tools to manage dependencies, projects, virtual environments, packages and more, which are collectively known as the Python tooling ecosystem. It’s clear that this ecosystem has always been fragmented, seemingly beyond hope. There are so many tools that do similar things, many of which overlap in functionality, and it’s not clear to even experienced users which one to use when collaborating with others on a project. Having worked with countless users who are just entering the Python ecosystem, I’ve seen first-hand how confusing this fragmented tool chain is for newcomers. Recall that Python has only grown in popularity over the years, and is now the most popular language on GitHub, so user-friendliness in the ecosystem is crucial for the language’s long term success.

The arrival of Astral, and their tools: ruff and more recently, uv, has proven to be a breath of fresh air to the entire Python community. Early in 2024, uv’s release blog post¹ created quite a buzz across the ecosystem, but as I reflect on my own usage of uv over the course of this past year, I felt the need to write this post as a tribute to uv’s success, and to describe how it’s markedly improved my productivity when working with Python.

A lot of you reading this post are probably like me – i.e., you are a user of Python libraries and frameworks written by other people. In this blog post, I’ll summarize a bit of history of prior tools that did each did their individual jobs well, but uv is now changing the game by unifying the benefits of these tools into one.

TL;DR

As an AI Engineer, I typically use uv in two distinct “mindsets” during my development workflow.

• Experimentation: In a project’s early stages (when I make and break quickly), I use uv in combination with ipykernel to iterate on ideas and experiments via interactive execution, directly in my editor (I generally avoid using Jupyter Notebooks).
• Distribution: Once the overall flow of the project is clearer, I use uv to execute the workflow end-to-end via the command line, while seamlessly translating my initial working code to something that’s production-ready or for sharing with others.

If you want to see examples and a timing comparison, skip ahead to this section to see the commands I regularly use in my workflows.

Armin Ronacher, creator of Flask and the rye package manager for Python, explains the reasons for his excitement towards Astral’s toolchain very well in his blog post. With his vast experience in the Python ecosystem, he approaches uv from a library developer’s perspective. This particular comment from his blog post stands out:

Domination is a goal because it means that most investment will go into one stack. For me uv is poised to be that tool. It’s not quite there today yet for all cases, but it will be in no time, and now is the moment to step up as a community and start to start to rally around it
– Armin Ronacher²

Unraveling the messy Python tooling ecosystem#

The early days: easy_install#

When I first began using Python in early 2009, the Python Package Index (PyPI) and pip weren’t mature (pip had only just come out a few months before), and the only way to install packages was by either manually downloading the source and running setup.py, or use the easy_install utility. easy_install had some benefits, but came with a host of problems³. It didn’t have a way to uninstall packages, and it also didn’t have a way to reproducibly and reliably pin the right versions of dependencies like pip did via a requirements.txt file when it came out.

pip was a huge step forward for the Python community, and it quickly became the de-facto package manager for Python. It was so good that it was eventually incorporated into the Python standard library, shipping with every default distribution of Python native to unix operating systems.

The conda affliction#

The package manager conda was developed to solve dependency resolution and library compilation challenges in the scientific Python ecosystem, which made sense when key libraries like numpy and scipy simply didn’t compile on certain operating systems via pip. However, in hindsight, as pip became more and more mature, it’s clear that conda further fragmented the Python user community because it introduced a new package and environment manager totally distinct from pip. It also used an entirely different repository for new, up and coming libraries (conda-forge), so every library maintainer who released a new package on PyPI to be installable via pip had to also release the same package on conda-forge to be installable via conda. If you’ve worked on large Python codebases, especially those that involve scientific computing, you likely faced the issue of conda environments and pip environments not playing well together, forcing your to either migrate your entire codebase into one ecosystem or the other.

I’ve always found it tragic that an entire generation of scientific Python developers came into the Python ecosystem circa 2010-2020 and only knew conda as their go-to Python dependency management tool. In all my time using Python over the last 15 years, I’ve found that “regular” software engineers or backend developers never wanted/used conda, preferring other alternatives, and so I’ve seen my fair share of teams that are a mix of scientific developers and software engineers who have to painstakingly stitch together their workflows because they simply cannot agree on a single toolchain.

Why poetry rose to prominence#

poetry is a tool that a lot of developers (especially library maintainers) swear by, and for good reason – it’s a great tool for managing environments and dependencies via locked dependency files. While pip relied on the requirements.txt file to manage dependencies, poetry introduced a much more structured pyproject.toml (and its associated poetry.lock file), which results in deterministic builds across different platforms and far fewer cases of failure in resolving dependencies.

However, there’s no denying that poetry is noticeably slow. In most machine learning or scientific computing projects where you’re dealing with a ton of dependencies, poetry’s dependency resolver can take a long time to resolve the dependency graph and get a project set up. The justification has always been that it at least “works”, and reduces the burden of portability on library developers who are distributing packages to a large number of users across a variety of platforms. I’ve seen a lot of mature projects in the Python ecosystem adopt poetry for these reasons, and it’s more or less become the de-facto standard package manager for projects involving larger teams or huge open source projects where collaboration is of essence.

A host of other tools#

I won’t go into the details of a whole host of other tools that address specific painpoints in Python tooling, but each one of them such as pyenv, pipx, pipenv, pip-tools, pixi and many others, tried to bring their own perspective to the table, to solve everything from dependency management, from managing Python versions to creating virtual environments. The pipx documentation has an entire page to just list the differences between theirs and a ton of other tools. Mind-numbing stuff, to say the least.

uv: The solution to all these problems#

Since its launch in February 2024, uv has exploded in popularity, and many large Python projects have completely migrated to it in their CI/CD as well as local build/execution pipelines. Below, I’ve summarized a few key reasons why I think uv has become this much of an unstoppable force in such a short time since its launch:

• Performance: uv is fast. It’s ~10x-100x faster than pip and poetry in realistic environments, partly because its dependency resolver is written in Rust 🦀, but also because it implements a lot of smart caching and optimizations that are laser-focused on providing the best performance possible. See this video by Astral’s CEO Charlie Marsh for a deep dive into some of these optimizations.
• Usability: uv is easy to use. It’s a command line tool that’s easy to install, and it’s very easy to set up a new project and get started with it, because it’s also so well-documented.
• Learned lessons from Rust 🦀 packaging: Just like the Rust programming language has the Cargo ecosystem for a single unified toolchain, uv brings a lot of the best practices that Rust developers have come to appreciate, into Python.
• Learned lessons from Python 🐍 packaging: Charlie Marsh, Astral’s CEO, has been open about how he and the core team that developed uv spent a lot of time studying the history of the Python tooling ecosystem (including what worked, and what didn’t), and worked closely with the creator of rye, Armin Ronacher – so this is a tool that’s been built with a lot of prior thought and consideration put into it.
• Engagement with the Python community: A very important aspect of Charlie’s and Astral’s approach is humility, by not pushing their solution on the community and continually engaging with long-term users of other tools in the Python ecosystem. And because the project is completely open source, we get direct insights into how they’re learning from the community.

Usage#

Alright, with that bit of background out of the way, I hope it’s clear why so many Python developers other than myself are just as excited about uv as I am.

What makes a good Python project setup?#

Let’s first start by asking ourselves what the requirements are for a good Python project setup.

• Python version management: How do you manage the Python version for your project? How do you ensure that the user executing your code is using the same Python version as you intended?
• Dependency management: How do you manage dependencies for your project? How do you ensure that the dependencies are locked and that you can reproduce the environment on another machine?
• Virtual environments: How do you manage virtual environments for your project? How do you ensure that the virtual environment is isolated from the system Python installation?
• Interactive code execution: Some times, you may want to run exploratory code to test out ideas without needing a full-fledged Python project. How do you link your interactive code execution tool (e.g., Jupyter notebook) to your project’s dependencies?
• Project packaging: How do you package your project for distribution? How do you ensure that the package is built correctly and that it can be installed on another machine?
• Ease of use: How do you ensure that the tools and workflow you applied to your project is easy enough for you, yourself, and then to train others who are new to the project?

As you read this post and go through the uv docs, you’ll see that it addresses all of these requirements (and more), while being a one-stop-shop when you may have earlier resorted to using multiple other tools.

Create project#

The following steps assumes that you’ve installed uv globally on your machine. To begin, let’s set up a new uv project.

uv init uv-demo
cd uv-demo

This will create a new directory called uv-demo and initialize a new uv project with some basic files in it. The directory structure should look like this:

uv-demo/
├── .python-version
├── .gitignore
├── pyproject.toml
├── hello.py
└── README.md

The hello.py file is a simple Python script that prints “Hello, World!” to the console. You can either delete this file or leave it in there, but it’s a good starting point to begin writing some code. A .python-version file is created to specify the Python version that was used for this project. The pyproject.toml file is the main configuration file for the project, and it’s where you can specify the dependencies for your project – initially, it’s empty.

Set up environment#

Once of the best things about uv is that you don’t need to worry about virtual environments. uv will automatically create a virtual environment for your project, while also ensuring that code that it executes in the current project directory is executed from within this virtual environment – no more worrying about whether you’re in the right virtual environment or not!

For a demo of how fast uv is, let’s install the following dependencies:

kuzu==0.7.1
lancedb==0.17.0
llama-index==0.12.8
llama-index-llms-openai==0.3.12
llama-index-embeddings-openai==0.3.1
llama-index-graph-stores-kuzu==0.6.0
llama-index-vector-stores-lancedb==0.3.0
numpy==2.2.1
polars==1.18.0
pyarrow==18.1.0
python-dotenv==1.0.1

These requirements contain a non-trivial set of dependencies if, say, you want to build a Graph RAG system using Kùzu as the graph store, LanceDB as the vector store, and LlamaIndex as the orchestration framework. These tools/frameworks themselves depend on libraries like pyarrow, numpy, polars, etc., and there needs to be a way to ensure that all of these dependencies are installed a) reproducibly and portably across different machines, and b) without any conflicts in the dependency graph that brings together all of these libraries in a working project.

Performance comparison#

All tests below are run on my 2023 M3 Macbook Pro. We will install these dependencies using three different tools: pip, poetry and uv, and compare the performance of each when starting “cold”, i.e., with a clean cache. Let’s first purge the cache for all the tools on our local machine:

uv clean cache
pip cache purge
poetry cache clear --all .

pip#

Let’s first run the following three commands to install the dependencies via pip:

# Create a new virtual environment
python -m venv .venv
# Activate the virtual environment
source .venv/bin/activate
# Time the execution of the pip install command
time pip install -r requirements.txt

Installing the dependencies via pip took 18.3 seconds. Note that when using pip, you needed to manually keep track of the local .venv directory, and activate it so that the dependencies are installed in the correct environment. It’s all too easy to forget to activate the virtual environment, which leads to the dependencies being installed in the system Python environment instead (not at all a good practice!).

poetry#

Let’s remove the virtual environment via rm -rf .venv and run the same command again, but now let’s install the dependencies via poetry:

# Create a new virtual environment and activate it
poetry init
# Accept the default values and then enter the poetry shell
poetry shell
# Install the dependencies
time poetry install

Installing via poetry took 6.3 seconds. Again, note how the steps are multifold – you needed to initialize a new poetry project, enter the poetry shell, and then install the dependencies. To reuse the environment created, you have to enter the poetry shell every time.

To clean things up for the next step, let’s remove the virtual environment via poetry env remove uv-demo-_xPlEE72-py3.12, and delete the poetry.lock and pyproject.toml files.

uv#

Let’s now install the same dependencies using uv:

# Create an empty pyproject.toml file and initialize a uv project
uv init
# Install the dependencies
time uv add -r requirements.txt

Installing the dependencies via uv took just 2.3 seconds! Upon completion of this command, we now have a .venv directory in our project directory, which contains the virtual environment for this project. We do not need to worry about its contents, or about activating it, as we will see below. uv manages this directory and its usage entirely for us. Indeed, you may as well not even be aware that the .venv directory exists.

The ease of use is already evident here – we simply run uv add -r requirements.txt and uv handles the rest.

To summarize, here’s a timing table for this simple dependency resolution task, done via a cold cache using each of the three tools:

uv was 8x faster than pip and 3x faster than poetry in this relatively simple dependency resolution task. In a realistic scenario, you’re likely dealing with a lot more dependencies that are more complex, and the performance gap between uv and the other tools would be even more pronounced.

Interactive code execution#

In the beginnings of a project, I typically experiment with ideas and iterate quickly, during which I often rapidly update or modify my dependencies. uv is great for this, because it allows me to seamlessly link my interactive code execution tool (e.g., the Cursor IDE) to my project’s dependencies. Modern IDEs like VS Code and Cursor have great support for interactive Python code execution because they embed ipykernel the core utility that powers Jupyter Notebooks, but they make the execution workflow with virtual environments far more seamless than using Jupyter notebooks. It’s amazing how easily a uv-managed virtual environment can be linked to an interactive code session in VS Code or Cursor.

uv has the concept of dependency groups, which allows you to clearly separate the development dependencies from the project’s core dependencies. Because ipykernel is something only I may use on my local machine – my collaborators may prefer to use their own tools to rerun my code – I normally install ipykernel in the dev dependency group. When executing code interactively in Cursor, I simply type in Shift + Enter to run the code, and Cursor will automatically link the ipython kernel to the local virtual environment directory, .venv, that uv created for us when we ran uv add -r requirements.txt.

# We ran this command earlier to install dependencies from requirements.txt
uv add -r requirements.txt
# Add ipykernel to the dev dependency group
# to be able to interactively execute code in Cursor
uv add --dev ipykernel

See below for a demo of how simple it is to use a local uv-managed virtual environment to interactively execute code in Cursor or VS Code.

The above approach is in stark contrast to Jupyter Lab or Jupyter notebooks, which typically require you to install a custom ipython kernel that’s stored and managed separately from the project’s virtual environment that may have been created via pip or poetry. This leads to a disconnect between the environment on your machine and the environment that someone else may use on their machine, because it requires them to perform a bunch of extra steps to ensure that the kernel is correctly installed and linked to the correct virtual environment on their end, which you as the developer who distributed the code have no control over.

uv solves the above problem by allowing you to use the same tool and virtual environment for both interactive code execution and command line execution, and clearly packaging it all in a self-contained way that can be easily distributed to others.

Command line execution#

Let’s look at how uv handles command line execution of Python code.

The .venv directory in which uv manages the virtual environment is created when you run uv add or uv sync (which looks for a local pyproject.toml file and creates a .venv directory that holds the virtual environment, if it doesn’t already exist). If you’re using uv in a CI/CD pipeline, you can run uv sync to ensure that the virtual environment is created at the location where the git repository is cloned.

❯ uv sync
Resolved 113 packages in 0.75ms
Audited 108 packages in 0.09ms

Let’s run the same code as shown in the interactive execution section above, but this time, we’ll run the Python script via the command line. Here’s the code we’ll run:

import polars as pl

df = pl.DataFrame(
    {
        "a": [1, 2, 3],
        "b": [4, 5, 6],
    }
)

print("Hello from uv-demo!")

We can run the hello.py script using uv run as follows:

❯ uv run hello.py
Hello from uv-demo!

Although it may not be immediately obvious, this very simple command does the following under the hood:

1. Installs Python if it’s not already installed
2. Creates and activates the virtual environment
3. Installs the dependencies
4. Runs the code

This sequence of steps is very powerful, as it simplifies the thought process of the developer while also ensuring that the code is executed in a consistent environment across all platforms. To ensure this, uv also creates a uv.lock file that locks the exact versions of the dependencies that were used to run the code, which is similar a poetry.lock file, but is created much faster.

As a developer, all I need to do is focus on writing good code that translates my thought process into something that can be executed, and uv will handle the rest!

Useful commands#

In summary, here are some of the most useful commands that I more frequently use when working with uv:

• uv init: Initialize a new uv project
• uv add <dependency>: Add a specific dependency to the project
• uv add -r <requirements.txt>: Add dependencies from a requirements.txt file (which is a common way to distribute dependencies in a Python project)
• uv sync: Sync the project dependencies with the existing pyproject.toml file (if it doesn’t exist, simply run uv init to create it)
• uv run myfile.py: Run a Python file in the project (similar to python myfile.py)

Tool usage#

Another very useful feature that I’ve begun using more in my workflows is the uv tool run command, whose shorthand is uvx. This command allows you to run the ruff formatter on your code via a similar workflow to uv run. Again, uv does the necessary background work to download and install ruff on the fly, if it’s not already installed on your machine⁴.

uvx ruff check .
uvx ruff format *.py --line-length 100

This eliminates the need for older tools like black, isort and flake8 that were necessary to format code in Python projects (these tools were also far slower than ruff). Using uv in combination with ruff makes the entire process of writing clean, well formatted code just so seamless and convenient!

Takeaways#

uv can obviously do a lot more than what I’ve shown here, and I encourage you to read the docs to deep dive into its functionality. Other benefits include managing multiple Python versions, executing standalone scripts without the need to create a virtual environment, and more. This is only the beginning!

In my Python project workflows (both for experimentation and distribution of clean, reproducible code), uv has completely replaced the following tools:

• pip
• pyenv
• poetry
• venv
• pipenv

Depending on what background you’re coming from, you may have other tools that you’ve used, that are also likely easily replaced by uv.

I’m very passionate about communicating these benefits of unification in the Python tooling ecosystem, which in my opinion, brings Python on par with other languages like Rust, Ruby and Go – languages that have long been lauded for their excellent tooling. I also believe that sometimes, these ideas are best communicated via live demos, so I highly recommend watching the following video for a live coding tutorial that I did on this topic with YourTechBud. Feel free to add your thoughts in the video’s comments section!

If you’ve made it this far, thanks for reading! Please help spread the word about uv and begin trying it out for yourself. Your future self will thank you for it, and there’s no going back!

Example code#

The example code and requirements.txt file used in this post can be found here. Feel free to clone the repo and compare uv’s performance to other tools in your own projects!