Tips and Guidelines for the Noob Witness Setup

Are you planning on setting up a witness node and don't have a lot of knowledge in Linux? Did you setup your node and stumbled on a few snags? Keep reading.

I've been hanging around in the https://steemit.chat/channel/witness chat for a week now, after I started my own witness node. I've seen many new comers like myself bringing their various contributions to the witness community and wanting to setup their own nodes. Many are following @someguy123's guide on setting up a witness server. It's an excellent starting point if you have no experience with linux. It's written by an experienced IT guy who is a witness himself. I've followed his guide personally and I've been up and running in no time.

However, there are a few recurring concerns being voiced in the channel by users during their setups. I would like to address them here, especially for the panicky types. Also I'll share a few simple linux tips for managing and monitoring your node.

Witness Considerations

First and foremost, a witness node is a very sensitive process, it's highly recommended that you do not run anything else on your box, or you will risk missing blocks. Dedicate your box to running the witness only, with the recommended minimal specs. A decent CPU and a stable bandwidth are a must. A Raspberri Pi with 4GB RAM ain't gonna cut it!

Currently the blockchain data is 16.7GB (not a big problem)
For the shared memory file, the current miminal requirements are

Seed node (p2p mode): 5.5GB
Exchange node: 16GB --> that's us (we can comfortably run with 12GB)
Full node: 65GB --> that's the heavyweights like @gtg

For the rest of this post, I'll assume you followed @someguy123's setup guide and you're on Ubuntu.

Monitoring the Witness Logs

The guide included the command ./run.sh logs which is an alias for the command:
docker logs --tail=30 witness

This will show the last 30 lines of the log. If you want to monitor the logs in realtime, add --follow (or -f) option like this:
docker logs --tail=30 --follow witness

You can stop the log monitoring with Ctrl-C. It's a universal linux command that terminates a process running in the foreground.

Monitoring the CPU and RAM Usage

There are two commands that allow you to do that with the commands top or htop. They have different outputs, so use whichever you prefer.
You can install htop on Ubuntu with sudo apt-get install htop.

Common steemd Errors

A common startup error is:
[33m2263080ms th_a main.cpp:133 main ] Error parsing logging config from config file /steem/witness_node_data_dir/config.ini, using default config
It happens with @someguy123's method and it's nothing to worry about.

During the replay, you will see many errors like this:
[33m2416593ms th_a steem_evaluator.cpp:85 do_apply ] Wrong fee symbol in block 3208405
Again, these are normal.

When full synchronization is reached, your node is ready and you will start seeing messages like this:

585054ms th_a       application.cpp:507           handle_block         ] Got 16 transactions on block 13549520 by timcliff -- latency: 54 ms
588382ms th_a       application.cpp:507           handle_block         ] Got 27 transactions on block 13549521 by xeldal -- latency: 382 ms
591143ms th_a       application.cpp:507           handle_block         ] Got 20 transactions on block 13549522 by witness.svk -- latency: 143 ms
594427ms th_a       application.cpp:507           handle_block         ] Got 24 transactions on block 13549523 by roadscape -- latency: 427 ms
597057ms th_a       application.cpp:507           handle_block         ] Got 19 transactions on block 13549524 by jesta -- latency: 57 ms
600292ms th_a       application.cpp:507           handle_block         ] Got 18 transactions on block 13549525 by good-karma -- latency: 292 ms
603441ms th_a       application.cpp:507           handle_block         ] Got 20 transactions on block 13549526 by anyx -- latency: 441 ms

If you're running the replay for the first time (./run.sh replay) and you have an hour to stare at the monitor while you drink coffee, I recommend you follow the logs in realtime just to be sure everything is working OK.

The steemd daemon is single-threaded so don't be alarmed if only one CPU core is operating at 100% during the replay. And don't be surprised if it's taking ages on a Atom CPU!
After the replay is done, the CPU usage is pretty minimal. What's really important is the RAM. If you don't have enough RAM, the replay will freeze.

Once you're synchronized and you broadcasted your witness (update_witness in the cli_wallet), don't forget to run a price feed. You can use Steemfeed-JS. Although be advised, sometimes the price feed will sporadically crash with the following error (you guessed it, monitor it with docker logs --tail=30 --follow feed)

SteemApi error: get_required_signatures [ { ref_block_num: 0,
    ref_block_prefix: 0,
    expiration: '1970-01-01T00:00:00',
    operations: [ [Object] ],
    extensions: [],
    signatures: [] },
  [ 'STM11111111111111111111111111111111111111111111111111' ] ] {}
(node:17) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: SteemApi error:get_required_signatures[object Object],STM11111111111111111111111111111111111111111111111111{}

It's a known bug https://github.com/svk31/steemjs-lib/issues/8 and the workaround for now is to crontab it to restart every two hours, with the following command:
crontab -e
Then add this line to the end of the file and save:
0 */2 * * * docker restart feed

Then cd ~ and start the feed again in the background with:
docker run -itd --rm --name feed steemfeed-js

There was a little mistake in @someguy123's guide, he wrote:
docker run -it --rm --name feed steemfeed-js
which will run the process in the foreground, so add the 'd' to run it detached (in the background).

Setup the Time Synchronization

One last crucial step is to set the server time synchronization (it was mentioned in another guide, don't remember which one). We need to keep in synch with the rest of the nodes!

Open a root shell (I hate typing sudo everytime):
sudo -s

Set your timezone, for me it's EST (optional):
timedatectl set-timezone America/New_York

Install NTP (Network Time Protocol):
apt-get install ntp

Open the config
nano /etc/ntp.conf

Add these settings to the end of the file and save
minpoll 5
maxpoll 7

Or, you can do this quicker with one line:
echo "minpoll 5" >> /etc/ntp.conf ; echo "maxpoll 7" >> /etc/ntp.conf

Then:
systemctl enable ntp ; systemctl restart ntp

Check everything is good:
timedatectl

Should output something like this:

Local time: Sun 2017-07-09 23:27:57 EDT
Universal time: Mon 2017-07-10 03:27:57 UTC
RTC time: Mon 2017-07-10 03:27:57
Time zone: America/New_York (EDT, -0400)
Network time on: yes
NTP synchronized: yes
RTC in local TZ: no

And finally, exit the root shell:
exit

Conclusion

If you have other troubles with your witness setup, you can join us on https://steemit.chat/channel/witness, there are plenty of cool people who can help.

I am @drakos, and I am a witness 😀

_{Proud member and delegator of the @minnowsupport project.

Join us on https://discord.gg/GpHEEhV}

If you like this post, upvote and comment.
If you really like it, resteem.
If you love it, follow.
_{Don't be shy, click that banner and enjoy my blog (gif animation by @justcallmemyth)}