We spend so much time working toward a goal, delivering on an expectation, that it becomes business as usual. People trust me to do my job, I trust others to do theirs. Together we make great teams, great divisions, and a great company for our customers.
In the last two weeks, two major companies have lost my trust. One, Anthem, an insurance company, that had 80M customer medical records stolen and the other Lenovo, a computer manufacture, that installed software on their consumer laptops and desktops that intercepted TLS/SSL encrypted traffic using a self signed encryption certificate and embedded it into the operating system.
I trusted these companies like I know my company’s customers trust us. How can we prevent becoming the next untrustworthy company?
For Anthem, the problem was not following the basics of data security. Encrypt your data at rest, in flight, and protect your keys at all cost. Restrict data access to only those who need it – and ONLY the data they need. What a logistics nightmare to coordinate that among the entire company’s applications – but not as bad as the nightmare they’re living with now.
Regardless if you store your data in a colocation with insane physical and electronic boarder security or your own onsite datacenter with James Bond proof security – thieves don’t have to get out of their underwear to make off with a billion dollars with of data.
Lenovo had a simple task of maintaining their tradition of building computers that people want. Someone made the decision to install some extra software (presumably for profit) on their consumer computers that gathers “We thought [Superfish] would enhance the shopping experience…“. I call bullshit – the software was harvesting their customers data by decrypting encrypted browser traffic using a “Man in the Middle” attack. Lenovo decided to make an extra dollar and deceive their customers into thinking the lock on their browser actually meant they were secure. It took five months for the word to get out on the Lenovo malware, but in the week it hit the media – I’ve read about it everywhere. I’m watching LNVGY to see what happens when the stock holders finally figure out what this means.
It means companies large and small that buy one or thousands of their products in bulk may begin to question the integrity of Lenovo. If they installed privacy busting malware into their consumer goods – what kind of nefarious tricks were in the latest ThinkPad or X laptop in my business? Is my CIO’s bank account going to be hacked, company secrets leaked to a Lenovo partner in China, or my laptop used as a portal for Chinese hackers to run rampant in my network? A secret decryption chip and hook into the network hardware to leak secrets back to home base? Sure, it’s far fetched… or is it? Stranger things have been dreamed up.
Anyone can prevent these – stand up and say something. Call out bad ideas for what they are. Identify security risks when you see them. These are what makes great people, great. They take on the challenge instead of letting the big sleeping dragons sleep. Eventually they’ll wake up, on their own or with the help of someone, and lay waste to your kingdom.
In a previous post about the Sophos UTM, I’ve received some comments about Xbox Live.
I’ve recently rebuilt my Sophos UTM and found that my backup configuration files were encrypted with a password I couldn’t locate… my bad. So I’m rebuilding all of my firewall rules again.
For the Xbox 360 Live and Kinect, there are some quirks that I hope these steps help you overcome.
- Setup a DHCP reservation for your Xbox so that your UTM will create a network definition and the IP will remain the same.
- Create new service definitions for each of the ports listed in the base article here
- Port 88 (UDP)
- Port 3074 (UDP and TCP)
- Port 53 (UDP and TCP)
- Port 80 (TCP)
- Port 1863 (TCP and UDP)
- In my UTM, I’ve named them Xbox Live UDP 88, Xbox Live TCP & UDP 3074, etc. This helps keep your definitions clean and searchable.
- Create a definition group and call it Xbox Live and Kinect (or whatever helps you keep organized).
- Create a new Firewall rule
- Source Xbox 360 -> Services Xbox Live and Kinect -> Destination Any
- Create a new NAT rule
- Rule Type: DNAT (Destination)
- Matching Condition
- For Traffic from: Any
- Using service: Xbox Live TCP and UDP 3074
- Going to: External (Address)
- Change the destination to: Xbox 360
- Automatic Firewall rule: Checked!
Save and don’t forget to turn on your Xbox 360 firewall rule before testing.
Here are some screen captures that may help show you what these all mean.
I hope this helps someone but I’ve been chasing a wifi problem in my house for a few days and finally got to fixing it.
Equipment: 4th Gen Airport Extreme (802.11 a/b/g/n; 2.4GHz & 5GHz)
Symptoms: poor range, slow speeds outside of the room the airport was located.
Configuration: I have it configured auto everything across the board, no unique SSID for the 5Ghz network, and nothing customized other than DNS – thanks to DDOS on Charter’s DNS servers I swapped in Google and Level 3’s DNS servers.
I loaded a free app on my MacBook called WiFi Explorer that displays wifi signals, noise, and occupied channels but the one built into OS X works equally well. I noticed that the 2.4Ghz network was dropping off for 30-45 seconds every minute even though there are devices on my network that require 2.4GHz.
5GHz was solid but as expected it’s range was poor and signal strength at the distances I needed it to work in my house were very low.
Started with power cycling the router. No difference. Soft then Hard reset, no change. 2.4GHz just wouldn’t stay on.
Launched Airport Utility and reset the AirPort to the factory setting, no difference. So now I’m thinking hardware issue. Of course, this is not under warranty anymore.
The last trick I had available was to roll back the firmware. In the airport utility click on the AirPort Extreme to display the serial number and firmware version. Option click on the version and pick 7.6.3 from the list. The utility downloads and installs the firmware. Really couldn’t be more simple.
Bam all is good after the reboot. Both radios functioning at expected levels. So is it a firmware issue or a glitch? So I decided to upgrade the firmware to the latest. The latest does have some good fixes.
After the reboot everything has been solid again. WiFI Explorer shows 5GHz and 2.4GHz on solid and never dropping off. I’m going to chalk it up to a glitch in the firmware that was cleared by reloading the firmware. The only way to do that for these is to rollback then upgrade again. Luckily though in true apple fashion – the utility does all the hard work and maintains your configurations.
Also the iOS utility offers the same functionality, so it’s easy to repair these.
Wrapping up master images has become something virtualization engineers of all product disciplines have to become familiar with. A bad master image can be deployed dozens or hundreds of times – only to find out a simple tweak could have saved you thousands in necessary hardware costs.
Here’s a new hidden gem I found and I hope to add to this list as more arrive.
Installing or updating Dot Net
Almost all Microsoft patching includes some form of a dot net update. When this product is updated, it likes to recompile a lot of code to help speed up launching dot net applications – pre-compiling actually does help user perception of application launch speeds.
Typically you run windows update on a server or workstation and dot net installs its updates and queues items in a work list that dot net executes later. This typically happens later in the day or evening and almost always pegs your CPU for a minute to 1/4 of an hour while is pre-compiles code.
Microsoft is pretty clear about this process in this MSDN Blog post.
The problem is, when you’re patching master images – you don’t want to leave the queued items for each deployed VM to have to execute. Deploy a dozen servers, and now you have a dozen servers with queued dot net jobs waiting to flog your CPUs.
For Windows 2008 R2 and Windows 2012 servers, you can easily kick off these queued items before you wrap up your images for templates by following these simple steps:
- Run a comand prompt or powershell prompt with administrative privlegdges.
- Run this command:
- Wait for the compiling to finish
The blog post above contains other paths for other versions of Windows, but hopefully that helps others.
KISS, keep it simple stupid. An old saying I try to keep in mind when working on a solution. My home network hasn’t maintained this zen philosophy.
My most recent setup has been driven by a recent discovery that I can keep my Apple Airport’s built in Guest network, as long as I can connect the Airport’s to something that can pickup VLAN 1003, the VLAN Apple decided to isolate the users on the guest network so they wouldn’t be able to see file or print shares on the main network.
So I decided to build a firewall but the only hardware I had was a bit overkill and didn’t want to dedicate the whole thing to this one task. I started with a used Dell OptiPlex 745 micro tower. This little workhorse has a dual core Intel CPU chugging along at 1.8Ghz, it’s got the virtualization instructions and 64bit capable. The Dell came with a tiny hard drive and a few gig of ram, but I grabbed a spare SATA drive and purchased four sticks of 2GB RAM from an ebay seller. I also picked up an Intel gigabit NIC to compliment the onboard gigabit NIC.
The Dell runs VMware ESXi 5.5 with a free license. I’ve configured the VMware host with four networks.
- External network, connected to the vSwitch that uplinks to the Intel Gigabit NIC
- Internal network, connected to the vSwitch that uplinks to the onboard Gigabit NIC
- Guest network, connected to the same vSwitch that uplinks to the onboard Gigabit NIC, but is on VLAN 1003
- DMZ network, connected to a vSwitch that doesn’t connect to anything physical, I’ll use it for test/dev VMs I create on the host.
I signed up for a free account with Sophos, here so I could download their UTM software. They also provide a generous license for home users that will protect up to 50 IP addresses. It’s almost nearly fully functional and includes managed end point protection for up to ten Windows computers. If I really wanted to go nuts, I could buy some Astaro (now Sophos) wireless access points and have them fully managed from this server but I think the Airport Extremes will work just fine.
Sophos offers a prebuilt appliance you can download and just run out of the box, but I think they let their junior assistant’s intern build it. It has poor performing configuration choices… and really you’re just better off building it from scratch. The .ISO is pretty universal – it’ll handle installing on bare metal, virtual, or being installed on Astaro/Sophos hardware appliances.
So going on information from the forums and documentation:
- OS is based on SLES 11 64bit
- 1 CPU, 2 Core
- 4GB of RAM
- 60GB of Disk, using a SAS controller
- 4 x VMXNET3 vNICs – leave all but the internal vNIC disconnected (Intel E1000 vNICs work, too but will consume more resources than the VMXNET3 paravirtual vNICs)
- USB controller (for USB backups)
- Delete the floppy disk drive, serial, and parallel ports
Boot the VM from the .ISO, and go through a few basic questions. It’ll handle the disk partitioning, volume formatting, and even installs VMware Tools for you because it identifies itself as a VMware VM and goes that extra step. Once you do the install, you’ll have to browse to the server’s IP on port 4444. During this final setup you’ll be prompted to upload the license file Sophos sends you and answer a few questions about the preliminary firewall rules.
I would recommend checking all of the basic firewall rules – Email, DNS, Web, etc. This will make your start a little easier. Leave the rest of the monitoring, filtering, and all that disabled, it’s only going to get in the way.
Now comes the tricky part – matching the physical nics with the virtual networks and vNICs on the Sophos UTM virtual machine. Under Interfaces & Routing, click on Interfaces. You’ll see your Internal interface [UP]. Add a second interface, let’s start with the Guest network.
- Name: Guest
- Type: Ethernet Static
- Hardware: Pick one
- IPv4 Address: choose an IP for this interface and a net mask to set the size. This IP will be the gateway for everything on this network.
- No other settings need to be changed or enabled
Once you save the interface, you’ll see that it’s [down]. Edit your VM, and enable the vNIC connected to the Guest network. If the interface doesn’t go up, edit it and pick a different eth in the hardware menu. Once you hit it, follow through on the other networks until you have all four up and running.
Now, Sophos UTM comes out of the box ready to support an internal LAN and external WAN. Additional network will require some more configuration, read on.
DHCP and DNS needs to be configured for at least the Guest network. You can set it up for DMZ too, if you want. Navigate to Network Services, DNS. In the Global tab, click the folder icon and choose your Guest (Network) and DMZ (Network).
Then switch over to DHCP and click New DHCP Server, and add a new one for the Guest network. The gateway and DNS IP addresses will be the interface IPs you setup earlier, they end in .1.
While you’re here – switch to the NTP network service and allow Guest and DMZ to access the NTP server you’ll configure later.
The one setting that got me stuck was the NAT settings. Switch to Network Protection – NAT. You’ll notice a rule already set for Internal (Network) -> External.
Clone this rule and set it up for DMZ and Guest so they can also connect to the outside world.
Now that I had a functional firewall, I needed to get the Airport Extreme devices configured. I launched the Airport utility from my Mac (you can do the same from any iOS device) and configured the Airports for bridge mode and turned off the password on my Guest network – the UTM will handle authentication now. Everything else works fine – one of my AEs is a print server with two printers, and a Time Machine backup target with a 2TB USB disk attached… all work fine.
Pro tip: once the AE is in “bridged mode” ,
you lose the ability to use the WAN port for anything. Just use the LAN ports for connectivity back to the Sophos UTM or other Airport Extreme APs the WAN port becomes a LAN port in bridged mode (I had a different experience in previous firmwares, so something must have changed). If you use a switch – make sure it can forward VLAN tagged traffic, most SOHO switches will not. If you have multiple Airport Extreme APs, daisy chain the AE APs to ensure VLAN 1003 packets get delivered back to the Sophos UTM and hang the switch off of the other ports to provide more access ports to your internal devices.
Okay, this was the icing on the cake for this deployment. Now that I have my guest network isolated on a dedicated interface and VLAN, I can really do some neat stuff with Sophos UTM.
- In the Wireless Protection section visit Global Settings.
- Enable the Wireless Protection, then add the Guest interface to the allowed interfaces.
- Click Apply.
- Visit the Wireless Networks, Access Points, and clean out any auto-configured networks or access points.
- Then click on Hostspots. Enable the Hotspot feature.
- Switch to the Hotspots tab.
- Add a new one, call it Guest Portal or something easy to identify.
- Add the Guest interface to this hotspot and configure the rest of the options for your unique needs.
Tips: The Password of the Day will not be an easy password. It’ll be something like ogaleseh35 (that was yesterday’s password at my house). This password is good for the day.
You can opt for a Voucher system… which is really powerful. You can limit by time and or consumed bandwidth. I can see myself forcing my kids to use a guest network and only handing out vouchers when stuff is done around the house. You can delegate access to other people so they can log into a user portal on the Sophos UTM and print or PDF additional vouchers.
A few other features I turned on:
- Global IPS, I turned off the IPS for now on my internal network until I can isolate my dumb media devices (smart TVs, etc) and exclude them
- Endpoint Protection (free AV software) – sophos managed end point for Windows (OS X is free, but still not managed yet!)
- Uplink Monitoring – I won’t get emailed when it goes down (duh!) but I’ll get the down and up alert when the UTM can send email again.
- User Portal – for people to get VPN setup and vouchers for guest wifi
- NTP – accurate time isn’t an option any more.
Additional Firewall Rules
Sophos UTM is a true firewall. Nothing gets in or out with an explicit rule. This can be very challenging at home when you have a myriad of different devices. Most devices like Apple TV, Vizio TV apps, and the like catch a ride on port 80 or 443, so if you have web enabled, you’re good to go.
Additional rules are needed to allow stuff like MineCraft, Apple Push Notifications, AT&T Microcell, Mumble, Xbox Live, etc… be prepared to spend some time digging around knowledge bases or Googling to find the appropriate ports. Luckily, it’s real easy to build rules.
Tip: Use groups for rules that you may need to add different ports or services to a single rule. I made one called Games. Now when I run into another game (or service like Steam) that needs another port allowed out – I can create the Network Service, and just add it to the existing Games group.
I have a rule for Apple. Holy crap they have a lot going on. Most you don’t need to allow out or are handled by existing rules.
- iCloud DAV services, iChat, FaceTime, Game Center, yuck… Lots of UDP port ranges, but again these are only allowing these apps OUT – not random internet person reach into your network on these same ports. On the topic of Apple, when watching the firewall logs I forgot they have a Class A IP range… so it may be easiest just to create a Network rule and call it Apple’s Network… then you can use that in each rule for apple.
- Xbox Live has a list of UDP and TCP ports that need out, and a NAT rule to allow UDP/TCP 3074 back in
- AT&T’s Microcell needs to have https, ntp, and some IPSec traffic allowed out to femtocell.wireless.att.com and after watching it fail – another IP, 18.104.22.168, in AT&T’s IP range that wasn’t documented. Once that last IP was added to the rule, bam – solid 5 bars.
One last thing that can help your UTM perform better is to disable logging and reporting if you don’t need it. I turned off all the reports but after reviewing the rules – the logs can be retained for no less than 1 day. So I chose to disable logging. This caused a problem with troubleshooting the firewall – I couldn’t view the live log, it wasn’t being generated!
So I turn it on when I want to troubleshoot something, but I’ll leave it disabled. I’m not sure how the 5th amendment fits in to a firewall keeping logs of your internet traffic for a year… but if you don’t have to save all the data about your network activity – why flog your UTM’s storage when you don’t have to.
On the flip side, if somethings not working – almost every feature has a Live Log view so you can watch the blocked packets fly by. This is exceptionally helpful.