As a witness, you witness node is expected to be available when it is your turn to witness a block. Depending on your position on the witness list your witness node will be selected to witness a block a few times a day.
If your witness is not available and functioning during that time you will get a "missed block". As a witness, this is a highly discouraged but is ultimately a fact of life.
The typical response is to log into the cli wallet and issue a "update-witness" using a unique signing key to disable your witness. This takes your witness out of the rotation to prevent further missed blocks. This is also the recommended procedure when doing planned maintenance to your witness server. A lot of new witnesses believe missed blocks are absolutely unacceptable but they happen, and how you respond is what is critical.
missed blocks are like battle scars on a body of a proud warrior ( epic music plays in the background ), there's nothing wrong to have them, well, as long as you are a warrior and have a well-shaped body
@gtg aka "Gandalf"
The developer of Steem-Python (@furion) made a really cool toolkit called Conductor that allows you to automatically disable your witness if you miss blocks. This can set this to be 1 block or the default 10 blocks and then it will automatically change your signing key to disable your witness.
You can also do an auto fail-over to a back-up witness. If that wasn't cool enough, it doesn't even have to run on your witness! You can and should install it on a remote machine to monitor your witness in case it loses connectivity or becomes unavailable.
If you thought that wasn't cool enough, it's so easy to set it up there is no reason not to.
Installing Conductor requires a few packages that can easily be installed with your favorite package manager. I have never used Conductor on Windows so I will focus on Linux only.
These instructions are for Debian/Ubuntu distrubution but just change the package manager to what is used on your platform.
How to install & Setup Conductor Kill-Switch
Install Dependencies
sudo apt install libffi-dev libssl-dev python3 python3-dev python3-pip
Install Steem-Python
pip3 install -U git+git://github.com/Netherdrake/steem-python
Install Conductor
pip3 install -U git+https://github.com/Netherdrake/conductor
Setup Wallet
Conductor uses a BIP32 encrypted wallet built into Steem-Python to protect the witness active key. The first time you add a key to the wallet, you will be asked for a password to protect your wallet.
Before doing anything, I recommend configuring Steem-Python for additional RPC nodes.
steempy set nodes https://steemd.steemit.com, https://rpc.buildteam.io, https://gtg.steem.house:8090
Add your active key to the wallet via steempy.
steempy addkey
The next step will vary depending on if it is a new witness. If it is a new witness, you will need to answer some questions about your new witness, otherwise it just completes uneventfully.
conductor init
We will focus on the kill-switch functionality of Conductor. Conductor can be installed on your witness, your backup witness, on a remote machine, your desktop, or all of the above.
To use the kil-switch all you need to do is:
conductor kill-switch
You will be asked for your wallet password so it has access to your active key. You can set the environmental variable "UNLOCK" to your wallet password and it will not ask you anymore.
Once the kill-switch has activated, it will run on it's own and monitor for missed blocks. Once it reaches the threshold (default 10 missed blocks) it will de-activate your witness. You will need to nohup, tmux, or init script the process so it continues to run after you leave.
There are two are only two other options for the kill-switch, both of which I use:
conductor kill-switch -n X
conductor kill-switch -k <BACKUP WITNESS PUBLIC KEY>
The first option changes the default 10 missed block threshold to whatever number you set it to. Typically this is 1, 5, 10 or some number near it. Once this threshold has been reached, the witness will be disabled using the unique singing key STM1111111111111111111111111111111114T1Anm which tells the network your witnes should be skipped.
The other option is where things get really cool. If you have a backup witness, you can add it's key to the wallet and then use the -k parameter to specify another public signing key to fail-over to instead of just disabling your witness. Currently it will not fail back to the primary and it will not notify you of a failure. It is really easy to setup though.
conductor kill-switch -k <BACKUP WITNESS PUBLIC KEY>
You do not have to specify the primary (current) witness node, and you can specify more than one additional backup node. This parameter also works with the -n parameter to specify how many missed blocks before failing over.
Conductor has many other options like a price feed, easy enable/disable, and an easy wizard for updating your witness parameters. The kill-switch is an easy to use but critical safety net.
Thanks @furion!
