https://github.com/osr4rightstools/osr4rights-tools is a project where we spin up an Azure VM with a GPU to do face searching for Human Rights investigators. This source contains the Azure CLI build script, and Cloud init deployment script of the VM.
https://github.com/spatial-intelligence/OSR4Rights source for the Python script that scans faces using the built VM above.
Installing on a GPU
We could use docker, and indeed there is a docker compose file in the face_recognition source. However it is quite out of date.
As you can see from the build script it is quite involved to install, and getting the correct versions of the libraries working is tricky.
- face_recognition Python lib that we use to identify faces in pictures which uses dlib
- dlib C++ toolkit containing machine learning algorithms
Only spinning up when needed / Performance
It is expensive keeping a GPU machine running all the time, so lets only spin one up when needed.
https://github.com/osr4rightstools/osr4rights-tools contains an Azure CLI script to bring up a vm ready to go. However compiling from source takes time - about 12 minutes.
- Create Azure resources (some in parallel with no-wait) - 1:28
- Run bash build script on VM
Creating an Image
see next blog article
apt or apt-get go with the newer apt?
Impossible to initialize nvidia nvml : Driver/library version mismatch
This happened when I did an apt-get upgrade
GPU doing fine, with 2 instances of the single threaded app working using 2 of the CPU cores.
- http://vis-www.cs.umass.edu/lfw/ lfw (170MB)
- http://afad-dataset.github.io/asian face age dataset
Copying files from GPU server to local
# windows to linux
scp 00248.png firstname.lastname@example.org:/home/dave/facesearch/job3
# linux to windows
scp email@example.com:/home/dave/facesearch/job3/. .
Or use WinSCP