Python Pipenv

To make dependencies easier in dev and production, I use pipenv.

Here is an example of my production VM build script with pipenv at the bottom

# Python 3.8.2 comes with Ubuntu 20_04 but we want newer
sudo add-apt-repository ppa:deadsnakes/ppa -y

sudo apt update -y

# get Python 3.9.14 (11th Oct 2022)
sudo apt install python3.9 -y

# to stop WARNING: The scripts pip, pip3 and pip3.8 are installed in '/home/dave/.local/bin' which is not on PATH.
export PATH=/home/dave/.local/bin:$PATH

# need pip to get pipenv 
sudo apt install python3-pip -y

# update pip to 22.2.2 (11th Oct 2022)
pip install --upgrade pip

# install pipenv
pip install --user pipenv

Pipfile

# create a new Pipfile and Pipfile.lock if not there
# otherwise get all dependencies
pipenv install

# useful to purge the virtual env
pipenv --rm

# update all packages
pipenv update

Add a dependency to pipfile eg https://postmarkapp.com/send-email/python which I use for sending emails

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
postmarker = "*"
numpy = "*"
requests = "*"

[dev-packages]

[requires]
python_version = "3.9"

Sample Python

import sys
import os
import requests
import numpy as np

# https://postmarkapp.com/send-email/python
from postmarker.core import PostmarkClient
import postmarker

# 3.9.14
print("Python intepreter version:")
print(sys.version)

# 2.28.1
print("requests module version:")
print(requests.__version__)

# 1.23.3
print("numpy module version:")
print(np.__version__)

# 1.0
print("Postmarker version:")
print(postmarker.__version__)

# token = os.environ['POSTMARK_TOKEN']
token = os.environ.get('POSTMARK_TOKEN')

# send to black hole (will succeed if no errors)
if token is None:
  token = 'POSTMARK_API_TEST'

postmark = PostmarkClient(server_token=token)
postmark.emails.send(
  From='help@osr4rightstools.org',
  To='davemateer@gmail.com',
  Subject='Postmark test',
  HtmlBody='Successful test - this is html'
)

Run

# check to make sure all dependencies are installed
pipenv install

# run inside the virtual environment
pipenv run python email-test.py

VSCode

Ctrl Shift P - Python select Interpreter, then select the service-firemap or whatever your directory is called. Or press on the 3.9.14 64-bit icon on bottom of screen

If you don’t see the correct pipenv, close and reopen vscode.

Ready to debug! With all dependencies installed in the pipenv.

Psycopg2

https://pypi.org/project/psycopg2-binary/

[packages]
postmarker = "*"
pandas = "*"
fiona = "*"
SQLAlchemy = "*"
# this fails
#psycopg2 = "*"
# workaround
psycopg2-binary = "*"

I couldn’t get the python postgres client to build from source which is advised for production.

So unfortunately I’m using this only in dev, and for prod I’m manually running pip files. https://github.com/osr4rightstools/osr4rights-tools/blob/main/fire-map-infra/create_firemap_webserver.sh#L188

Conclusion

Using a virtual environment to control versions of packages is super important in python to avoid clashes in package versions! Pipenv works fine.