Chia cryptocurrency farming seems popular these days. Farmers proudly post images of their expensive rigs in social networks, and avidly discuss optimization techniques.

Surprisingly, few are concerned with resilience.

Why is that important?

The software can stop working anytime. In the case of a long lasting outage, the cost of a missed opportunity could be significant.

Sooner or later, everyone needs to leave home to get food, interact with the outside world, and pursue activities unrelated to cryptocurrencies. It’s, therefore, essential to automatically recover the services when they become stale or go down.

Start on boot

First of all, make sure that farming starts after the machine reboots. The probability of such an event isn’t negligible, as most farmers run their services at home without redundant electricity supply.

On Ubuntu, save the following code in the file start_all in your chia-blockchain directory:

#!/usr/bin/bash

cd /home/x/chia-blockchain/
. ./activate
chia start all

Then, make it executable:

chmod u+x /home/x/chia-blockchain/start_all

Finally, schedule it in crontab to run on boot:

@reboot /home/x/chia-blockchain/start_all

You must replace /home/x/chia-blockchain with your installation directory path.

Monitoring

Every now and then, farming gets stuck – probably due to a software bug. After restarting the services, syncing kicks in again.

As this problem can happen anytime, you need to monitor the status and autocorrect.

The following Python monitor.py program queries the farming status, and restarts everything if the status isn’t either “Farming” or “Syncing”.:

#!/home/neno/chia-blockchain/venv/bin/python

import subprocess
import re
from datetime import datetime

out = subprocess.Popen(['chia', 'farm', 'summary'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
stdout, stderr = out.communicate()
stdout = stdout.decode('utf-8')

status = re.search( 'Farming status:\s+(.+?)\n' , stdout).group(1)

if status != 'Farming' and status != 'Syncing':
  f = open("/tmp/chia_hanging", "a")
  f.write( datetime.now().strftime("%Y-%m-%d %H:%M:%S") + ' ' + status + '\n')

  out = subprocess.Popen(['chia', 'start', '-r', 'all' ], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  stdout, stderr = out.communicate()
  stdout = stdout.decode('utf-8')
  f.write(stdout + '\n')

  f.close()

It logs incidents in /tmp/chia_hanging.

monitor.py is called by the shell script monitor:

#!/usr/bin/bash
DIR=/home/x/chia-blockchain/
cd $DIR
. ./activate
./monitor.py

Save both monitor.py and monitor in your chia-blockchain installation directory, replace /home/x/chia-blockchain/ with your path and make both scripts executable.

Finally, schedule the bash script in crontab:

0 * * * * /home/neno/chia-blockchain/monitor > /tmp/monitor.log 2>&1