Summary

As an infrastructure architect, the lab environment plays a significant role to validate design ideas, confirm configuration changes, and build new skills.

My home lab has evolved over the past 20 years, moving from a complete rack of hardware, down to a single server, back to multiple servers, then completely outsourced to the cloud. In recent years the physical servers have returned to run compute hungry workloads like GNS3 and VIRL, as well as ESXi appliances that are easier to host locally than from remote.  

Physical equipment

The lab begins with a 19" rack in the basement. In the rack from top to bottom is the cable modem, Juniper and Cisco routers, switches, and firewalls, Dell and HP servers running ESXi, power strips and a UPS.

This is a working lab, the cables are not pretty, cable management is non-existant, and the rack is not back lit with neon lights. It is changing, new parts are added, old parts are removed, new equipment is tested, cables are rerouted constantly. I prefer flexibility and rapid development over looks.

An undersized UPS supplies the power.  The UPS was originally purchased, when the lab was outsourced to the cloud and the only devices needed to remain up during a power outage were the cable modem, wireless access points, and router. While it is working for a single server, running both is not possible.   Underneath the Dell server are two APC SmartUPS 1000’s, but the batteries are dead:

Example power usage from the Dell server:

Virtual Servers

The 1U HP server is an old ESXi server currently being migrated to a new Dell with significantly more capacity. This system is running all the light workloads:

The 2U Dell is the workhorse of the home lab, all VM servers will be eventually migrated here. Systems like VIRL and GNS3 use a lot of resources.

Remote access

Linux VMs running on the ESXi server is where remote console and development work takes place. tmux allows me to start and stop work while easily picking up where I left off.

OpenVPN running on the home-server and Dynamic DNS provides remote access into the lab:

Configuring OpenVPN is made simple by openvpn-install:

jemurray@home-server:~$ history | grep open
  225  curl -O https://raw.githubusercontent.com/angristan/openvpn-install/master/openvpn-install.sh
  226  chmod 755 ./openvpn-install.sh
  228  sudo ./openvpn-install.sh

Network virtulization and design

GNS3 is used to model network designs.  The server is hosted on the ESXi farm so that there is enough CPU and memory to design large topologies. I legally get images from a Cisco VIRL license and a Juniper support contract.   

While I prefer to use GNS3 for most network modeling projects, I pay for VIRL to get legal Cisco images which work in both VIRL and GNS3.

Monitoring

Smokeping monitors connectivity internally and externally. In the past there were problems with my local ISP, tools like Smokeping make it easier to make a case when dealing with the support personal:

SaaS Services

Digital Ocean hosts my remote Linux servers. Even though I could host these servers at home using dynamic DNS, it is nice to have a well-connected system on the public Internet:

Route53 hosts DNS for my public and lab subdomains. I moved DNS to a hosted solution so that changes in the lab do not affect my personal “production” services:

Even a lab environment should have 2FA enabled. In fact, I have 2FA enabled for every public service with this feature enabled. LastPass authenticator generates the tokens:

Home LAN

Google WIFI supplies connectivity to the lab and the house. This is one area that is constantly evolving from a cheap big box router, to pfSense, vanilla Linux, Ubiquity edge, Cisco ASA, and Juniper SRX, the lab has seen them all. Why Google WIFI?  Because my family is tired of constant changes and complexity. Having a device that someone other than me can troubleshoot is important to my family. Having your wife or kids serial console to a Juniper SRX because YouTube TV is not working is a showstopper:

Electronics

In additional to the network and server hardware, I enjoy experimenting with electronics. Over the years I have collected various sensors, motors, servos, Arduino, Raspberry PI’s, and soldering equipment. I started playing around in this area, while trying to find inexpensive network connected relays for controlling power and sensors to monitor conditions in the house.

Old Parts

Finally, no home lab would be complete without a bone yard of old parts. Over time, the pile has dwindled, the shelves cleaned, and boxes emptied. A few years ago an entire pickup truck load of old equipment was shipped off to an electronic recycling center. Getting rid of this old hardware is one more step to keeping me off the horders TV show: