Wrapping up master images has become something virtualization engineers of all product disciplines have to become familiar with. A bad master image can be deployed dozens or hundreds of times – only to find out a simple tweak could have saved you thousands in necessary hardware costs.
Here’s a new hidden gem I found and I hope to add to this list as more arrive.
Installing or updating Dot Net
Almost all Microsoft patching includes some form of a dot net update. When this product is updated, it likes to recompile a lot of code to help speed up launching dot net applications – pre-compiling actually does help user perception of application launch speeds.
Typically you run windows update on a server or workstation and dot net installs its updates and queues items in a work list that dot net executes later. This typically happens later in the day or evening and almost always pegs your CPU for a minute to 1/4 of an hour while is pre-compiles code.
Microsoft is pretty clear about this process in this MSDN Blog post.
The problem is, when you’re patching master images – you don’t want to leave the queued items for each deployed VM to have to execute. Deploy a dozen servers, and now you have a dozen servers with queued dot net jobs waiting to flog your CPUs.
For Windows 2008 R2 and Windows 2012 servers, you can easily kick off these queued items before you wrap up your images for templates by following these simple steps:
- Run a comand prompt or powershell prompt with administrative privlegdges.
- Run this command:
- Wait for the compiling to finish
The blog post above contains other paths for other versions of Windows, but hopefully that helps others.
This evening I got an urgent email from a colleague that just got the rug pulled out from under him. His customer decided that the two web servers on the front end of a SharePoint farm he was building couldn’t be 2008 R2 because they require 32bit servers. I stayed far away from the “why” – and just wanted to help deliver the “how” as easy and fast as possible.
The problem he ran into was when he deployed two Windows 2008 32bit Enterprise Edition servers… they never customized from the template. Still had the default password, network settings, and host names, just bit for bit clones of the template with a customized answer file waiting to be applied.
Normally, vCloud Director will clone the template, push an answer file to it that contains all the things that have to change to make it a unique server, then kick off a sysprep (in windows) to make it all happen. Within a few minutes your clone is now a new server with your settings already set – ready to run.
Not this time. And to be honest, I don’t ever think we deployed a 2008 32bit Enterprise server – so the template may have been broken from day -1.
After digging around in the sysprep logs, I find a break…
SYSPRP LaunchDll:Failure occurred while executing 'C:\Program Files\Internet Explorer\iessetup.dll,SysPrep_Cleanup', returned error code 2 SYSPRP RunExternalDlls:An error occurred while running registry sysprep DLLs, halting sysprep execution. dwRet = 2
Okay, so I verify iessetup.dll exists. So why aren’t you executing it.
After more digging and some Googling, I think I find the root of the issue… we’re missing a registry key.
It appears that when .NET is installed and removed before sys prep – it can remove the RunOnce key, which is used by Sysprep to store some commands using iessetup.dll.
I simply launched our template, added the key, and wrapped it back up. Redeployed the two servers – and bam…
Two 2k8 32bit servers deployed without a hitch. It’s now 1:10am and I haven’t even touched my other work tonight.
Source of Solution:
We’re spinning up Windows 2008 R2 Standard servers from templates in our vCloud environment and begin to notice a problem. We can’t join them to a working domain.
Network Path Not Found is the error we got when we attempted to join the domain.
After some troubleshooting, we think it might be DNS issues… but everything works. After further investigation we notice that the service TCP/IP NetBIOS Helper is set to Automatic – but is not running. Attempts to start it, fail.
Nothing around the net is helpful – so we start tearing apart a working server. The one registery key related to the TCP/IP NetBIOS Helper is different.
Start Value should be set to 2 instead our broken server was set to 4.
This also solved the event log entry:
Service Control Manager (7001 – None): The TCP/IP NetBIOS Helper service depends on the NetBT service which failed to start because of the following error: The service cannot be started either because it is disabled or because it has no enabled devices associated with it.
When you launch the HP Network Configuration Utility from the control panel or the system tray and get this error:
An error occurred due to invalid data in the XML file used by this application. The XML file has been corrupted and should be reinstalled from the installation media.
You have no obvious way to resolve it. The recommendations are to disolve the team or reinstall the NCU. The trick is you need the NCU to disolve the team and the NCU doesn’t appear in the Add Remove Programs!
Attention: Before you go banging through these steps to rip out and replace your network connections, please read the steps first. Enjoy not having to learn from your own mistakes because you will have learned from mine!
I had software installed on my server that was licensed according to the MAC address of the Team – and it stayed the same. Your milage may vary so be sure you don’t nuke the team before you know exactly what your software is going to do if the MAC changes.
- Document your NIC team settings (including IP, Subnet Mask, Gateway, DNS servers, and all of the other customized fields) so you can enter them back in again later.
- Download a recent version of the NCU from HP’s website specific to your server’s operating system and place it on the desktop of your server for easy access.
- Connect to your server from iLO or a console – RDP will be useless after you nuke your network connections.
- Navigate to the properties page of one of the physical NICs on your server (see the screenshot I provided).
- Click on Properties
- Click in the “This connection uses the following items:” area on “HP Network Configuration Utility”
- After you’ve confirmed that you’ve done steps 1 and 2 – Click Uninstall
- Confirm all of the confirmation screens and reboot when it tells you.
- After the reboot, run the NCU installer.
- After a successful install of the NCU – launch it.
- Configure your network with a new team using the information you copied down in step 1!
Enjoy your new fully functional NCU. If you need further assistance, HP Support will be your best resource, check out their new support site: www.hp.com/go/hpsc
If were under a technology news rock yesterday you may have missed the news that HP decided it was no longer going to be competing with Apple in the phone or tablet business. The HP tablet prices dropped from $600 and $700 to $100 and $150 Saturday early morning.
A price drop this significant causes some big news. Websites like slickdeals.com picked up on it almost instantly and spread the word far and wide. Leading to a run on web retailers’ websites like bestbuy, officemax, walmart, and others. I even went out to stores on Saturday morning to see the effects and maybe a chance at scoring one, but my luck was poor. Not a poor as the guy working the phones at Staples… That guy was getting nonstop calls and angry walk-ins. I even let a disgruntled customer waiting in the computer isle in on the story and that the kid in the comouter department got sucker punched by HP last night… And on one of the busiest back-to-school shopping days around here… Dickish move, HP. You could have waited a few weeks.
But that’s not the worst of it. HP’s own web store couldn’t handle the traffic. Actually, it utterly crumbled under the load. The entire site was lethargic, more than normal, and even before getting to the actual product page you could tell it was getting hammered. The store was unresponsive and took three minutes to process my intent to purchase. After all that, I ended up with a VB Script error about running out of memory. Very classy.
Let me give you a little side story. HP is betting big in their cloud services, managed services, and high performance data centers. They’re looking to spin off or dump their consumer products like tablets and PCs because they aren’t profitable. They want to sell you servers (good products), network gear (crap), and software (hit or miss) to make it all work. They want you to come to them when you have problems with your network, websites, and business.
And they showed how completely unprepared they were for a simple spike in traffic that equalled, I’m estimating, the traffic any major retailer site gets on black Friday.
So here we are… with everything clearly in perspective. HP is going to take a beating from consumers who have stood by them and defended their consumer products, CTO and CIOs are going to question their professional services and data center designs, and me… well, I just wish Michael Dell would shut the fuck up about it all. His company isn’t doing all that hot in this market either and he’s not doing much to fix it.
It’s a shame really, this is not the company Hewlett and Packard worked their lives to make. The history of the company was one of greatness – a Google of the last generation. If you get a chance to read “The HP Way” give it a read. You’ll see this isn’t HP anymore.