Skip to content

Python virtual environments

venv

venv handles virtual environments.

Install venv with:

pip3 install venv

Create a new virtual environment with this command:

python3 -m venv env

Enter the virtual environment with:

source env/bin/activate

Exit with deactivate

pipenv

pipenv handles virtual environments and dependencies. It is a successor to the requirements.txt file.

To install, type in:

pip3 install pipenv

Spawn a shell in a virtual environment to isolate the development of this application:

pipenv shell

To close your virtual environment and shell, just type in exit.

This will create a virtual environment if one doesn’t already exist. Pipenv creates all your virtual environments in a default location.

You can force the creation of a Python 2 or 3 environment with the arguments --two and --three respectively. Otherwise, Pipenv will use whatever default virtualenv finds.

Sidenote: If you require a more specific version of Python, you can provide a --python argument with the version you require. For example: --python 3.6

Now you can install any 3rd party package you need, f.e. flask. Oh, but you know that you need version 0.12.1 and not the latest version, so go ahead and be specific:

pipenv install flask==0.12.1

Let’s say you also have some unit tests for this awesome application, and you want to use pytest for running them. You don’t need pytest in production so you can specify that this dependency is only for development with the --dev argument:

pipenv install pytest --dev

Providing the --dev argument will put the dependency in a special [dev-packages] location in the Pipfile. These development packages only get installed if you specify the --dev argument with pipenv install.

Okay, so let’s say you’ve got everything working in your local development environment and you’re ready to push it to production. To do that, you need to lock your environment so you can ensure you have the same one in production:

pipenv lock

This will create/update your Pipfile.lock, which you’ll never need to (and are never meant to) edit manually. You should always use the generated file.

Now, once you get your code and Pipfile.lock in your production environment, you should install the last successful environment recorded:

pipenv install --ignore-pipfile

This tells Pipenv to ignore the Pipfile for installation and use what’s in the Pipfile.lock. Given this Pipfile.lock, Pipenv will create the exact same environment you had when you ran pipenv lock, sub-dependencies and all.

The lock file enables deterministic builds by taking a snapshot of all the versions of packages in an environment (similar to the result of a pip freeze).

Now let’s say another developer wants to make some additions to your code. In this situation, they would get the code, including the Pipfile, and use this command:

pipenv install --dev

This installs all the dependencies needed for development, which includes both the regular dependencies and those you specified with the --dev argument during install.

When an exact version isn’t specified in the Pipfile, the install command gives the opportunity for dependencies (and sub-dependencies) to update their versions.

Pipenv will attempt to install sub-dependencies that satisfy all the requirements from your core dependencies. However, if there are conflicting dependencies (package_a needs package_c>=1.0, but package_b needs package_c<1.0), Pipenv will not be able to create a lock file and wil output an error like the following:

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches package_c>=1.0,package_c<1.0

As the warning says, you can also show a dependency graph to understand your top-level dependencies and their sub-dependencies:

pipenv graph

This command will print out a tree-like structure showing your dependencies. Here’s an example:

Flask==0.12.1
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.10]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.14.1]
numpy==1.14.1
pytest==3.4.1
  - attrs [required: >=17.2.0, installed: 17.4.0]
  - funcsigs [required: Any, installed: 1.0.2]
  - pluggy [required: <0.7,>=0.5, installed: 0.6.0]
  - py [required: >=1.5.0, installed: 1.5.2]
  - setuptools [required: Any, installed: 38.5.1]
  - six [required: >=1.10.0, installed: 1.11.0]
requests==2.18.4
  - certifi [required: >=2017.4.17, installed: 2018.1.18]
  - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
  - idna [required: >=2.5,<2.7, installed: 2.6]
  - urllib3 [required: <1.23,>=1.21.1, installed: 1.22]

From the output of pipenv graph, you can see the top-level dependencies we installed previously (Flask, numpy, pytest, and requests), and underneath you can see the packages they depend on.

Additionally, you can reverse the tree to show the sub-dependencies with the parent that requires it:

pipenv graph --reverse

This reversed tree may be more useful when you are trying to figure out conflicting sub-dependencies.