Consensus errors and warns while running a private network

I’m running a private network with 2 nodes in 2 different VM’s on a cloud provider.
The nodes are able to connect but i get errors and warning from the stdo and stderr which i cannot fully understand.

On the genesis validator:
https://pastebin.com/raw/n5zytHzz

{"log":"WARN 2020-05-06 16:56:33 consensus/src/liveness/pacemaker.rs:190 Local timeout for round 380\n","stream":"stderr","time":"2020-05-06T16:56:33.8601336Z"}

{"log":"WARN 2020-05-06 16:56:33 consensus/src/event_processor.rs:510 Round 380 timed out: already executed and voted at this round, expected round proposer was [\"5942d356\"], broadcasting the vote to all replicas\n","stream":"stderr","time":"2020-05-06T16:56:33.8602487Z"}

{"log":"WARN 2020-05-06 16:56:35 network/src/peer_manager/mod.rs:906 Connection from /ip4/168.63.129.16/tcp/58121 failed to upgrade unexpected end of file\n","stream":"stderr","time":"2020-05-06T16:56:35.6872694Z"}

Specially the last one. The port is always random (?) and that ip isn’t the second validator.

On the second validator (the non genesis one )
https://pastebin.com/raw/if3NbuUq

WARN 2020-05-06 16:54:15 consensus/src/liveness/pacemaker.rs:190 Local timeout for round 333
WARN 2020-05-06 16:54:15 consensus/src/event_processor.rs:510 Round 333 timed out: will try to generate a backup vote, expected round proposer was ["5942d356"], broadcasting the vote to all replicas
ERROR 2020-05-06 16:54:16 consensus/src/event_processor.rs:674 Waiting until proposal block timestamp usecs 1588784062505776 would exceed the round duration Instant { tv_sec: 2537259, tv_nsec: 346735594 }, hence will not vote for this round
WARN 2020-05-06 16:54:16 consensus/src/event_processor.rs:626 MaxWaitExceeded
WARN 2020-05-06 16:54:19 consensus/src/liveness/pacemaker.rs:190 Local timeout for round 336
WARN 2020-05-06 16:54:19 consensus/src/event_processor.rs:510 Round 336 timed out: will try to generate a backup vote, expected round proposer was ["5942d356"], broadcasting the vote to all replicas
ERROR 2020-05-06 16:54:19 consensus/src/event_processor.rs:674 Waiting until proposal block timestamp usecs 1588784066105518 would exceed the round duration Instant { tv_sec: 2537264, tv_nsec: 596198827 }, hence will not vote for this round
WARN 2020-05-06 16:54:19 consensus/src/event_processor.rs:626 MaxWaitExceeded
WARN 2020-05-06 16:54:24 consensus/src/liveness/pacemaker.rs:190 Local timeout for round 337
WARN 2020-05-06 16:54:24 consensus/src/event_processor.rs:510 Round 337 timed out: will try to generate a backup vote, expected round proposer was ["5942d356"], broadcasting the vote to all replicas

What can be the cause of the event processor timing out the round?

I’m on the testnet branch and I’m running the validators with

docker run \
    -e CFG_LISTEN_ADDR="$bootstrap" \ //current machine ip if not genesis 
    -e CFG_NODE_INDEX="0" \ // index 1 if non genesis
    -e CFG_NUM_VALIDATORS="$nodes" \
    -e CFG_SEED_PEER_IP="$bootstrap" \
    --publish 8000:8000 \
    --publish 8080:8080 \
    --publish 9101:9101 \
    --publish 9102:9102 \
    "$image"

(to create the docker image I used the build script in validator-dynamic(

2 Likes