Tearing my hair out trying to debug newfound packet loss on my home network.
-
Tearing my hair out trying to debug newfound packet loss on my home network. I updated OpnSense the other day through a year of deferred updates. I thought that was the source of the problem. It was not.
To cut to the chase, here's a nice little iperf log. The === 40001 === is the source port that I have iperf configured to send from
# sh checkem-cmu
===== 40001 =====
[ 5] 0.00-5.02 sec 59.8 KBytes 97.4 Kbits/sec 0.547 ms 0/51 (0%) receiver
[ 5] 0.00-5.25 sec 59.8 KBytes 93.2 Kbits/sec 0.824 ms 0/51 (0%) receiver
[ 5] 0.00-5.03 sec 59.8 KBytes 97.4 Kbits/sec 0.686 ms 0/51 (0%) receiver
===== 40002 =====
[ 5] 0.00-5.02 sec 5.86 KBytes 9.56 Kbits/sec 0.340 ms 41/46 (89%) receiver
[ 5] 0.00-5.02 sec 7.03 KBytes 11.5 Kbits/sec 0.326 ms 39/45 (87%) receiver
[ 5] 0.00-5.02 sec 5.86 KBytes 9.56 Kbits/sec 0.141 ms 36/41 (88%) receiver
===== 40003 =====
[ 5] 0.00-5.02 sec 59.8 KBytes 97.5 Kbits/sec 0.635 ms 0/51 (0%) receiver
[ 5] 0.00-5.06 sec 59.8 KBytes 96.7 Kbits/sec 2.689 ms 0/51 (0%) receiver
[ 5] 0.00-5.02 sec 59.8 KBytes 97.5 Kbits/sec 0.839 ms 0/51 (0%) receiverThis is repeatable. Most 5-tuples work. Particular 5-tuples experience 90% packet loss.
So, naturally, I swapped out my router for my laptop, and managed to repeat the packet loss.
So now is the hard part: Convincing Verizon they've got a bad LAG or ECMP group a little bit inside their network when all I can talk to is a front-line tech guy who's very nice but only knows how to test my ONT and has no idea what iperf or ECMP is.
-
Tearing my hair out trying to debug newfound packet loss on my home network. I updated OpnSense the other day through a year of deferred updates. I thought that was the source of the problem. It was not.
To cut to the chase, here's a nice little iperf log. The === 40001 === is the source port that I have iperf configured to send from
# sh checkem-cmu
===== 40001 =====
[ 5] 0.00-5.02 sec 59.8 KBytes 97.4 Kbits/sec 0.547 ms 0/51 (0%) receiver
[ 5] 0.00-5.25 sec 59.8 KBytes 93.2 Kbits/sec 0.824 ms 0/51 (0%) receiver
[ 5] 0.00-5.03 sec 59.8 KBytes 97.4 Kbits/sec 0.686 ms 0/51 (0%) receiver
===== 40002 =====
[ 5] 0.00-5.02 sec 5.86 KBytes 9.56 Kbits/sec 0.340 ms 41/46 (89%) receiver
[ 5] 0.00-5.02 sec 7.03 KBytes 11.5 Kbits/sec 0.326 ms 39/45 (87%) receiver
[ 5] 0.00-5.02 sec 5.86 KBytes 9.56 Kbits/sec 0.141 ms 36/41 (88%) receiver
===== 40003 =====
[ 5] 0.00-5.02 sec 59.8 KBytes 97.5 Kbits/sec 0.635 ms 0/51 (0%) receiver
[ 5] 0.00-5.06 sec 59.8 KBytes 96.7 Kbits/sec 2.689 ms 0/51 (0%) receiver
[ 5] 0.00-5.02 sec 59.8 KBytes 97.5 Kbits/sec 0.839 ms 0/51 (0%) receiverThis is repeatable. Most 5-tuples work. Particular 5-tuples experience 90% packet loss.
So, naturally, I swapped out my router for my laptop, and managed to repeat the packet loss.
So now is the hard part: Convincing Verizon they've got a bad LAG or ECMP group a little bit inside their network when all I can talk to is a front-line tech guy who's very nice but only knows how to test my ONT and has no idea what iperf or ECMP is.
Oh yeah, this is the hard part. Apparently the front-line techs have no way to receive information via any text format. They had to video and take a shot of my screen to see the traceroutes.
blah blah this could have been an email blah blah and instead I've spent 2 hours on the phone with them.
-
Oh yeah, this is the hard part. Apparently the front-line techs have no way to receive information via any text format. They had to video and take a shot of my screen to see the traceroutes.
blah blah this could have been an email blah blah and instead I've spent 2 hours on the phone with them.
@dave_andersen best wishes!
I often recommend nping for things like this
