On the second-to-last day of vacation, I suddenly stopped being able to reach my home Wireguard server, which I use as a VPN and to access all my home devices/services when I'm away. The neighbours are on the same ISP and they weren't having any issues, so I resigned myself to having to fix things out in a rush on the night of my return before going to bed, since I was going to need the internet for work the following day.

When I finally got home, I saw that it wasn't just internet access: all devices were unable to talk to each other, even locally. I first suspected my old EdgeRouter X, so I connected my laptop directly to it, but everything was fine. I got a DHCP lease and could browse the internet. I suspected a naughty device messing up the network. In the past, I've had some USB-C docks start misbehaving and flood the entire network if the computer they were attached to went to sleep, but I didn't have any of them connected this time, so I was puzzled.

The activity lights on all the switches around the house were blinking frantically, more than would be justifiable for a network of unreachable devices, so I figured some other device must have been misbehaving. I started methodically disconnecting things from the EdgeRouter and noting which one would bring the connection down when reconnected. I "narrowed" it down to a MoCA adapter that brings Ethernet upstairs from my basement through the house's coax cabling, so it was something on the main or upper floor.

I started exploring everywhere and bisecting all network connections between all the switches, until I finally got to a switch in the kitchen. I unplugged one of the cables, and the activity light for the port right next to it turned off too. ...What? I followed the cable and discovered that it went right back into the port next to it, forming a loop.

This is a spare cable that I keep rolled up under the kitchen table if I want to work from there -- what probably happened is that our friend who was housesitting while we were away saw the unplugged end of the cable under the table, thought that perhaps they had accidentally unplugged it, and plugged it back in, promptly bringing the entire network down. Fun!

Previous Post