What are the odds that you buy two motherboards- from the same manufacturer but at different times, and end up with duplicate MAC addresses?
I’m currently in the process of building a new homelab and, without disclosing too many details, bought three Supermicro server boards. Everything is nearly ready and so my vExpert accredited buddy came over to have a look at the new toys and generally talk about nerdy shit while setting things up. I’d gotten to the stage of installing vSphere and vCenter onto USB pens and a separate hard disk respectively.
With everything loaded up, it was time to create a cluster. Add host to cluster, enter credentials, timeout. Hmmmm.
- vSphere client to host connectivity – check
- Laptop ping to host – check
- vSphere Web client to host connectivity – check
- Scratch head – check
vSphere 1 – IT professionals 0
After much coffee drinking it was decided that the issue must lie with with networking on the fresh vCenter install, and so it was reinstalled.
vSphere 2 – IT professionals 0
After more coffee and head scratching, several Zenmap scans and more ping payloads than I’d care to count, we established that the switch must be at fault. Switching cables between ports was causing random IP addresses to show and disappear sporadically. Oh no, I’ll just have to buy that Cisco SG300 I’ve had my eye on (I hate my switch.) Firmware updated, ARP cache cleared, etc, etc, etc. By this point we’d worked out how to get consistent (but still incorrect) results from the switch.
vSphere 3 – IT professionals 0
“What are the odds I’ve got duplicate MAC addresses” I jested. After a bit more head scratching we decided it would be worth a look. IPMI was reporting different MAC addresses. The motherboards were labeled with different MAC addresses. LANScan showed otherwise.
Son of a bitch. I really didn’t want to send this motherboard back across the globe to be replaced.
The next few minutes contained many four letter expletives and defamatory comments about Supermicro’s quality control team. I say defamatory because as my buddy Googled for previous, similar occurrences he said something that made me instantly realise what had happened. I’d installed ESXi on one of the servers then swapped the USB pen to another server. ESXi likes to stick with the MAC address it’s given at the time of install.
vSphere 4 – IT professionals 1
I’m awarding us both a point here. vSphere definitely shafted us (through my own actions, I know) but a quick reinstall of ESXi and we got the last laugh.
Lessons learned:
- I need more hands to count the amount of times I’ve installed vCenter
- Don’t swap boot disks between devices when running ESXi
- Supermicro quality control ain’t so bad
- If turning it off and on again doesn’t work, reinstalling it will
How slim are the odds of duplicate MAC addresses?
Pretty slim.