Description

BACKGROUND

I am an IT guy. I often use my home as a laboratory for the things I do in the office. This allows me to expand my knowledge, test theories, and find out what works before deploying systems in a live business environment. This project resulted from a passion to simplify all of the various network appliances and servers into a single, easy to manage, centralized system. A primary goal was to virtualize my Untangle router/firewall into an ESXi environment. This, obviously, was going to require some advanced planning and network configurations.

So, naturally I converted my recently retired gaming/editing desktop into an almighty all-in-one glorious VMware ESXi server for home use. Everything you see was recycled from previous desktops and workstations.

HARDWARE SPECIFICATIONS

CPU - Intel 5820K (6/12)

Motherboard/Chipset - Intel X99

Memory - 32GB DDR4-3000 in Quad Channel

Storage - 128GB PCIe NVMe, 500GB SATA3 SSD, 6TB mechanical HDD

Network - 1x PCIe 1GbaseT, 4x PCIe 1GbaseT

Power - 650W 80+ Gold

Video - 512MB Quadro (only required for system to boot)

PRIMARY FUNCTIONS

1) Network router/firewall/web filter (Untangle v14.1.0)

2) Network storage for SMB & FTP shares (OpenMediaVault v4.1.22)

3) Plex media server

4) Security camera NVR for 3 IP surveillance cameras (Blue Iris v5)

5) Syslog/SNMP server (Kiwi v9.6.7)

6) Axigen e-mail server (future dated)

7) VPN server and tunnel for privacy

DESIGN CONSIDERATIONS

Perhaps the trickiest part of this build was configuring the network interfaces so that a virtual network router could be implemented. A mis-configuration could open the network to direct internet access, and thus to cyber attack. Here's how I laid it out:

There are five physical network interfaces (NICs) in this machine, not including the Killer interface integrated onto the motherboard, which is not being utilized due to known ESXi compatibility issues. So, of the remaining NICs, one is dedicated to the management interface of the ESXi server itself. We'll call this one Ethernet 0, or eth0. This provides a means to manage the appliance even if the router (which is also the DHCP server) is unavailable.

A second NIC, eth1, is assigned to a physical CAT5e Ethernet connection that spans across the home via the attic crawl space and into the garage, where an 8-port switch mounts to a workbench. This 8-port gigabit switch provides PoE and connectivity to the three IP surveillance cameras located on the exterior of the home via dedicated CAT5e connections to each.

The third NIC, eth2, supplies connectivity to a TP-Link EAP225 wireless access point, passing through a PoE injector on its way to the device. This ceiling mounted access point provides a blanket of 2.4/5GHz WiFi to the home.

The fourth NIC, eth3, provides local network connectivity to a nearby Netgear ProSafe 8-port switch, which interconnects several other appliances located in the home - a printer, a smart TV, a Roku, a gaming PC, and a Fingbox network monitoring appliance. This NIC also provides connectivity to each of the virtual machines through a vSwitch. More on this in a moment.

The fifth and last NIC, eth4, connects to the Motorola/Arris SB6190 broadband modem, providing Internet access to the home.

Each physical connection to the ESXi server had to be configured with independent vSwitches. While this does not allow for any redundancy across interfaces, it does allow for maximum flexibility for various configurations. Utilizing vSwitches among the various virtual machines has an added benefit - 10gbps intercommunication between them. Each physical NIC corresponds to a vSwitch, which then corresponds to a Port Group. Labeling of each component was done in such a way to limit confusion during troubleshooting, and there are five in total. I.e. eth0 >> vSwitch0 >> PortGroup0

VIRTUAL MACHINES

The Untangle virtual router is the heart of the network. It has four network connections, both physically and virtually. The first port is for WAN connected to the modem. Local connectivity to the switch is on port-2. While all of the physically connected devices in the home have static IP addresses, Untangle provides DHCP for any and all WiFi devices over port-3. Connectivity to the IP cameras and the garage workbench is done via port-4. Untangle also serves several other functions including a firewall, web filter, SSL packet inspector, intrusion prevention, bandwidth monitoring, OpenVPN server for remote connectivity to the home, and always-on full tunnel VPN relay for homeowner privacy.

The various network devices are provided with 2 terabytes of shared cloud storage using OpenMediaVault 4 (OMV), which is based on debian Linux v4.19. This NAS appliance provides SMB files shares to the Windows laptops/desktops in the home, as well as PLEX movie storage and hosting through a plugin. In addition, FTP can be reached from the Internet via port-forward rules set in the router.

Home and property surveillance is provided by three Dahua HD cameras with PTZ and IR capabilities. They are managed, and recorded to a Windows Server 2016 based NVR system called Blue Iris, which was recently upgraded to version 5 just recently released. A web browser interface is accessible from the Internet, again via port-forward rules set in the router. This helps control and restrict access to the IP cameras themselves, providing a secure method for remote access.

Lastly, a virtual network monitoring system was deployed onto a Windows 7 environment, and utilizes Kiwi software (now owned by Solarwinds). The system ingests syslog and SNMP data from the various other network appliances and servers. It sends alerts through SMTP using gmail servers based on triggers such as high CPU or memory usage, low disk space, failed logon attempts, and other helpful events.

I currently have plans to configure a personal e-mail server using Axigen. This server could not only provide private e-mail handling and storage, but could be used as a local SMTP relay. The biggest hurdle at the moment is that my Internet provider (xFinity) blocks these ports inbound to my network.

CONCLUSION

This server has been running for a couple weeks now without any issues. I love having a central piece of hardware for any and all maintenance. I really love being able to spin-up a virtual desktop or server environment to test my various hair-brain ideas. When I tell my friends and coworkers that I run my home on a virtual router, it always makes for interesting conversation. I really enjoyed building this system, and would highly recommend the challenge to anyone with some basic skills with networking and virtualization.

Questions/comments welcome. Thanks for looking!

Log in to rate comments or to post a comment.

Comments

  • 7 months ago
  • 2 points

yes

  • 7 months ago
  • 2 points

Still rocking that 5820K, nice

  • 7 months ago
  • 2 points

Absolutely. It's a power house!

  • 7 months ago
  • 1 point

Truly amazing build you have. Very nice. If it's okay with you to ask, what kind of edition of VMware ESXi and your routing software did you use? Would a free version be possible or would you recommend to get the paid version?

  • 7 months ago
  • 2 points

VMware ESXi 6.7 is free software for private use. You can download from their website after creating a free account.

[comment deleted by staff]
  • 7 months ago
  • 0 points

Comments like this are not appropriate for this site.