How to create an offline self-hosted haveibeenpwned API service

Community Feedback
1

Intro

When you create your account for the first time, or when you want to change your passphrase, passbolt checks if this passphrase is not part of a breach by sending requests to api.pwnedpasswords.com.

If you run passbolt in a very restrictive environment with no access to api.pwnedpasswords.com and still want to check if your passphrase is not part of a breach, or if you don’t want to sent requests to this API, you could be interested by this tutorial.

I will explain here how to self-host an offline haveibeenpwned API service on a Linux server in your local network. I used Debian 11 for my tests but you can use the Linux distro of your choice.

The offline API service is this project from Felix Engelmann: GitHub - felix-engelmann/haveibeenpwned-api: running the have i been pwned api locally, please read the README to fully understand how it works. Use it at your own risk !

Add a DNS entry

We cannot define a custom haveibeenpwned service in passbolt, that’s why you will have to make your local computer (requests to the API are made from the passbolt browser extension) believe api.pwnedpasswords.com is running on your Linux server.

Create a new DNS entry where x.x.x.x is the IP address of the server who will run the self-hosted service.

x.x.x.x → api.pwnedpasswords.com

Create self-signed SSL certificates

api.pwnedpasswords.com is served through https, so you have to create self-signed certificates and make your local computer trust them by importing the certificate in your computer: how to import self signed certificate - Google Search

Create the SSL certificates:

openssl req -x509 \
    -newkey rsa:4096 \
    -days 3650 \
    -subj "/C=LU/ST=Your-country/L=Your-town/O=Your-org/OU=Your-team/CN=api.pwnedpasswords.com/" \
    -nodes \
    -addext "subjectAltName = DNS:api.pwnedpasswords.com" \
    -keyout key.pem \
    -out cert.pem

As you will trust these certificates on your network, you will be able to reach the service with SSL on your local network without issues.

The setup

Here is what you need on your Linux server:

  • nginx to act as a reverse proxy for the offline-pwnedpasswords API and Handle SSL certificates
  • p7zip-full to be able to extract pwnedpasswords archive
  • aria2 to speedup the download
  • docker.io to quickly be able to run docker images
sudo apt install nginx aria2 p7zip-full docker.io

Download the offline archive of pwned passwords, the aria2c command will use 16 connections to speedup the download, it will create a ~15Go file (you can get the list here: Have I Been Pwned: Pwned Passwords ):

aria2c -x16 https://downloads.pwnedpasswords.com/passwords/pwned-passwords-sha1-ordered-by-hash-v8.7z

Extract it (will create a ~25Go file):

7z x pwned-passwords-sha1-ordered-by-hash-v8.7z

This list contains lines like this:

000000005AD76BD555C1D6D771DE417A4B87E4B4:10
00000000A8DAE4228F821FB418F59826079BF368:4
00000000DD7F2A1C68A35673713783CA390C9E93:873

To be able to use the offline API, you must have lines like this:

000000005AD76BD555C1D6D771DE417A4B87E4B4:0000000010
00000000A8DAE4228F821FB418F59826079BF368:0000000004
00000000DD7F2A1C68A35673713783CA390C9E93:0000000873

This python script will help you to get the correct line format:

wget https://raw.githubusercontent.com/felix-engelmann/haveibeenpwned-api/master/scripts/prepare.py

Execute the script to get the correct line format, will create a ~28Go file:

python3 scripts/prepare.py pwned-passwords-sha1-ordered-by-hash-v8.txt 10 pwned.txt

docker image

From there, you have a pwned.txt file, it is your offline hashed passwords list. You can launch an offline pwned password API with docker. You can run the docker image from felixengelmann:

sudo docker run -d --restart always -v "$PWD/pwned.txt:/srv/pwned.txt" -p 127.0.0.1:5000:5000 felixengelmann/haveibeenpwned-api

This image seems to be rebuilt on a regular basis but is quite huge (~1Go :open_mouth:). If you care about security, you can use an image I built. My image is ~60Mo and is a distroless one.
Distroless means you won’t find any shell utility such as bash, cd, or mkdir. This image can run only the pwned-api and nothing else.
You can run it like this:

sudo docker run -d --restart always -v “$PWD/pwned.txt:/srv/pwned.txt” -p 127.0.0.1:5000:5000 --entrypoint /usr/local/bin/python anatomicjc/haveibeenpwned-api run.py

You will the sources of this image here and automated gitlab pipelines here (image is rebuilt once a week)

This will expose your offline API on localhost on port 5000. To really simulate the pwnedpasswords API, you must serve it with https. You can create this nginx configuration file on /etc/nginx/sites-enabled/offline-pwnedpasswords.conf:

server {

  listen 443 ssl http2;

  server_name api.pwnedpasswords.com;

  ssl_certificate /etc/nginx/cert.pem;
  ssl_certificate_key /etc/nginx/key.pem;

  ssl_session_timeout 1d;
  ssl_session_cache shared:MozSSL:10m;  # about 40000 sessions

  ssl_session_tickets off;

  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
  ssl_prefer_server_ciphers off;
  
  location / {
    proxy_pass http://127.0.0.1:5000;
  }

}

This will forward api.haveibeenpwned.com requests on https to the docker container. Verify the path to your SSL certificates, and verify your nginx configuration is valid:

sudo nginx -t

If it is ok, you can restart nginx:

sudo systemctl restart nginx.service

passbolt should now be able to check is your passphrase is not part of a breach with your local haveibeenpwned self-hosted API.

Don’t hesitate to ask if some parts of this tutorial are unclear.

Please enjoy,

1 Like