Mike F Robbins

The requirements for the 2013 Scripting Games Advanced Event 4 can be found here. For this event I created multiple functions and I’m going to quote chapter 6, section 1 of the “Learn PowerShell Toolmaking in a Month of Lunches” book written by Don Jones and Jeffery Hicks, published by Manning:

A function should do one and only one of these things:
Retrieve data from someplace
Process data
Output data to some place
Put data into some visual format meant for human consumption

I started out by naming my function to create the audit report properly (New-UserAuditReport). If you having trouble picking a name for your function, it’s probably because you’ve broken the previous rule and your function does more than one thing.

Of course, all of the functions (4 of them) all have their own comment based help.

Here’s the validation that I performed on the filename and extension via a regular expression. This not only enforces the htm and html file extension specified in the requirements, but also ensures that the file name is valid on a Windows system. There’s no sense in allowing the script to process all the way to the end only to have it fail because the file name is not valid on a Windows system. Be proactive, not reactive. This MSDN article explains what is not valid in a Windows filename.

I couldn’t decide whether to use the cmdlets that are part of the Active Directory module or not because they may not be present on the system this is being run from and they only work on Windows Server 2008 R2 or higher unless you have a list of ingredients to include a Leprechaun as discussed in this TechNet blog to make them work with Windows Server 2003 domain controllers. Here’s a “Hey, Scripting Guy! blog” on the same subject that refers to 2008 non-R2 domain controllers as well. The problem is that since the scenario didn’t specify what OS the domain controllers are running, these cmdlets may not work.

Using ADSI (Active Directory Service Interfaces) on the other hand would be something that would work on systems that didn’t have the AD PowerShell module installed and it would also work against older domain controllers.

What I decided is why make a decision? Why not have the best of both worlds? Try the AD cmdlets and I mean literally try the AD cmdlets as shown in the first try catch block in the code below and then if there’s an issue, try ADSI as shown in the second (nested) try catch block in the code shown below as well:

If neither one works then the script exits with a warning instead of continuing to only error out somewhere further along in the code. That’s what the problem variable is for.

There is an issue on the Scripting Games website where the “Plain” tab doesn’t show the inline CSS so be sure to use the “Code” tab when copying and pasting the code to use on your own machine.

I figured those auditors would love some old school Greenbar paper. Here’s an example of six results showing a “Greenbar” style webpage (aka virtual Greenbar paper):

You may be wondering why I would chose to use the “LastLogonTimestamp” property when there are other properties such as “LastLogon” that already have human readable dates to start with? That’s because the LastLogon property is not replicated to all domain controllers in the domain and LastLogonTimestamp is beginning with Windows Server 2003. So it’s either query all the domain controllers in the domain or convert the date. From an efficiency standpoint it’s better to convert the date rather than possibly querying hundreds of domain controllers. See this TechNet blog article for more information about those two properties.

Now for the two functions that “Get” the data. The first is “Get-ADRandomUser”:

As you can see, this is a reusable function. Here’s the output that it creates (which is objects):

The ADSI one was a little trickier because I needed it to return the same data as the AD one. This one is called “Get-ADSIRandomUser”:

Although the type of object that this function produces is different, it’s still objects and it produces the same output from a properties and data standpoint as the previous function:

And finally, there is the “Set-AlternatingCSS” function which is a modified version of a function from Don Jones’s “Creating HTML Reports in PowerShell” book. Visit PowerShellBooks.com for details about this free ebook (Thank You Don!)

µ

I’m seeing a trend with a lot of PowerShell scripters. Many of us have a GUI, non-scripting background and we moved to PowerShell because we didn’t want to be button monkeys clicking through the same options in the GUI day in and day out. I’ve always heard, if you’re going to do something more than once, do it (script it) in PowerShell.

The trend I’m seeing is many are bringing their bad habit of wanting to repeat themselves over and over, day in and day out, to PowerShell. They write the same code over and over. I need a new tool to accomplish a task today. I’ll rewrite the same code that I’ve written every time I’ve had to accomplish a similar task again today so there will be fifty thousand copies of similar code out there, each in a different function to accomplish a similar task. No, no, no! Now you’re thinking like a script monkey instead of a button monkey, the only difference is you’re repeating yourself in PowerShell instead of the GUI.

A new day and a new way. Write reusable code, write it once and be done with it! That way when a modification needs to be made, it only needs to be made once or at least a lot less. Redundant data is a bad thing. What? I thought redundant data was what we wanted in IT? No (period). Redundancy of systems to prevent failure and to keep them online in case of a failure is a good thing. Redundancy in the form of backups is another good thing and all of these will assist you in keeping your job, but having multiple copies of the same data that are all being modified independently and not being synchronized is a nightmare.

The scenario for the 2013 Scripting Games Advanced Event #3 can be found here. I chose to first write a reusable function that would query the logical disk information and return it as generic objects that could be used to accomplish many different tasks. Want to create an HTML report as in this scenario, no problem. Want to display the output on the screen or place it in a text file, or use it as part of a larger project, those aren’t problems either, and there’s no reason to rewrite another function to retrieve the same information, ever. Hard coding this into the rest of the script or function is effectively the same thing and placing formatting cmdlets in you script or function because then you can’t do much else with it. What are you going to pipe the output to if it’s producing HTML files and not objects? Nothing, right? At least I can pipe format cmdlets to out cmdlets.

Here’s the output of that function:

Note, I pulled the computer name from the computer with the other information. This is very important. Getting on my soapbox for a moment. What is PowerShell? Ultimately, it’s a means to an end. At the end of the day if you don’t accomplish the task at hand, it does not matter how perfect your PowerShell code is. The client, customer, or your boss won’t care that you did something really cool and used best practices because ultimately the code didn’t accomplish the task at hand. We should strive for all the above though, really cool, efficient code that accomplishes the goal your trying to accomplish and do it accurately!

Never depend on someone imputing the real computer name into your tool. It’s all too common to have a DNS CNAME record for many machines which is not the true computer name. The IP address or something such as localhost could also be provided. If you’re naming your HTML file and displaying the name that was provided via input on the webpage, there’s a good chance that it’s not the actual “Computer Name”. You’re already querying WMI and at least for the class I chose, Win32_LogicalDisk, the computer name can be retrieved from it as well so there’s not much overhead in retrieving one additional property to achieve a higher level of accuracy. Climbing off my soapbox now.

Ultimately, the Get-DiskInformation function would be placed in a separate module. You can see in the image below that I placed it in the begin block of my advanced script. Never heard of an advanced script? Neither had I. I didn’t even know you could do all those cool advanced function things in a script. Placing the Get-DiskInformation function in the process block would cause it to run one time for each computer that is piped in so don’t put it there because it only needs to be loaded once, then it’s in memory and can be called as many times as necessary without having to reload it.

While we’re on the subject of a script, you’re probably wondering why I chose a script instead of a function for this tool? Think about how it’s going to be used, read between the lines in otherwords. Is someone actually going to manually run this script? Probably not. How fast is the data going to be out of date? Possibly very quick since it’s a static HTML file. It’s probably going to be setup as a scheduled task. That said, if it were a function it would need a script or some code placed inside of the scheduled task to dot source it which could get messy. I thought it would be cleaner to have everything in a script that could be called using PowerShell.exe in task scheduler. To me that approach was cleaner.

Don’t forget about validating the path. That seems to be a common problem. I honestly didn’t know how to do that myself and I took a few hits in event 1 because of it. Here’s an article that Glenn Sizemore wrote that taught me how to validate those types of parameters.

One small detail I did as well, as shown in the remainder of the script in the following image was to change the computer name to lower case for the output file because I prefer HTML files to be in lower case, but that’s just my personal preference. I didn’t change the case of the computer name that’s displayed on the web page though.

Want to know more? I’m speaking this Saturday, May 18th on “PowerShell Fundamentals for Beginners” at SQL Saturday #220 in Atlanta.

µ

I’m sure that most people can easily find any of the blogs of the official judges from the 2013 Scripting Games. I recommend reading those blogs whether you’re competing in the scripting games or not since there’s a wealth of great information contained in them. The best place to find those blogs if you don’t know already is the Judges Notes section under the Scripting Games area on PowerShell.org so there’s no reason to duplicate them here.

There are also a number of people who are competing in the Scripting Games that are writing blog articles of their own blog sites. A couple of the ones that I’m aware of are listed below and while they’re my competition in the advanced class and have links promoting their Scripting Games entries in their blogs (I do the same thing),  I don’t mind promoting their blog articles because there’s some great information to be found in them. I’m actually glad they provided links to their entries because both of these guys are excellent PowerShell scripters and you could learn a lot from viewing their Scripting Games entries. Ultimately the scripting games is all about the community learning more about using PowerShell best practices in a friendly competition that’s just for fun.

Click on the links below to go the Scripting Games category on each of these guys blog:

LazyWinAdmin (Trust me, this guy is not lazy. He was training for a marathon while at the PowerShell Summit).

_Emin_ also known as p0w3rsh3ll (I’ve been impressed with his PowerShell scripting abilities ever since he wrote some functions to set the proxy server settings on Server core – see that article here).

If there’s anyone else blogging about the Scripting Games, leave a comment and I’ll post a link to your blog as well. I believe PowerShell.org is in the process of putting something similar to this up.

Here’s where you can find the blog articles that I’ve written about the Scripting Games.

Update: More blogs about the 2013 Scripting Games will be added to the list below as I’m made aware of them. This is an open invitation to have the “2013 Scripting Games” category of your blog listed here:

Jonathan Tyler is one of the top PowerShell scripters in the advanced competition this year. He won the Iron Scripter competition last year at PowerShell Saturday 002 in Charlotte, NC. Be sure to check out his blogs and Scripting Games entries to learn techniques on how you might become an Iron Scripter in the future.

Dave’s blog articles about the Scripting Games.

Carlo’s blog articles about the Scripting Games.

Chris H’s articles about the Scripting Games.

µ

The majority of the entries I’ve seen for the Scripting Games event 2 are using the TotalPhysicalMemory property from the Win32_ComputerSystem WMI class to determine the total amount of physical memory in a server. The property name is “TotalPhysicalMemory” after all so that’s what it should contain, right? Not so fast, keep reading to discover why.

Your manager needs an inventory of all of your company’s physical servers that are located in three different data centers which are in different parts of the country. He has assigned you the task of inventorying those servers and wants you to physically log into each server or travel to the location where the servers reside to retrieve the information using the GUI.

You’ve asked your manager if you could try retrieving this information with PowerShell first. He is skeptical and thinks PowerShell is some kind of Witchcraft Voodoo stuff that must be like EverQuest or WarCraft because people stay up late at night playing some sort of scripting games with it. Reluctantly, he gives you the go ahead to proceed with the use of PowerShell for this project. This is your chance to sell your manager on using PowerShell at your company!

The requirements that your manager wants just happens to be the same as the Scripting Games event 2 requirements. Since you’re new to PowerShell, you decide to look at the scripts that were submitted in the advanced event #2 and use the method to retrieve the information that most others used, after all, if most people used it and received good scores, then it must be right? You think “This will be too easy, I can copy one of those 5 star scripts and I’ll be the PowerShell hero at my company!”

Well, it just so happens that your company got a good deal on some servers several years ago, before you started with the company, that had 8 gigabytes of RAM in them, but due to application compatibility requirements the prior IT engineers who are no longer with the company had to load an x86 (32 bit) version Windows Server 2003 R2. The company already owned Standard Edition of that OS and they didn’t need more than 4 gigabytes of RAM so they loaded that operating system to keep the project within budget.

The awesome 5 star script that you downloaded used what most others in the competition used to retrieve the total amount of physical memory in the servers (TotalPhysicalMemory property from the Win32_ComputerSystem WMI class).

Here are the requirements your manager provided which just happen to be exactly the same as the requirements in the Scripting Games advanced event #2. As you can see, in black and white, he wants the “Amount of Installed Physical Memory” from the servers:

You’ve completed the task, turned in your amazing results to your manager and saved the company lots of money on labor and travel in the process. Now your manager wants to see you in his office. It must to be because he wants to give you a raise or promotion for doing such an awesome job!

Your manager has checked the manufacturers website for some of the servers you provided the inventory for and he wants to know who stole memory out of the servers to put in their gaming machines at home to play these scripting games because the report shows 4GB of RAM and the manufacturers website shows they shipped with 8GB.

Well, it just so happens that the 5 star script you copied as well as what most of the others are using for the Scripting Games in event #2 to retrieve the Amount of Installed Physical Memory is incorrect! What? Sorry, but it’s true.

As you can see in the image below, the TotalPhysicalMemory property of Win32_ComputerSystem WMI class only shows the amount of memory that the OS has available to it and not the amount of physical memory in the servers:

Since the Windows Server 2003 R2 Standard Edition x86 (32 bit) operating system is only able to access 4 gigabytes of RAM, that’s what the report you turned into your manager had on it, where as you can see using the Win32_PhysicalMemory WMI class and adding up or summing the capacity for all of the memory chips did indeed retrieve the actual amount of physical memory in the server.

Want to know why not to use the Win32_ComputerSystem WMI class for the number of processor sockets? See my blog titled “2013 PowerShell Scripting Games Advanced Event 2 – Attention to Detail is Everything“

Want to see my solution for Scripting Games advanced event 2? Click this link: Advanced 2: An Inventory Intervention. You’ll have to be signed in to see the solution. If you like my script, please vote for it! Click on the star all the way to the right:

µ

Here’s my approach to the 2013 PowerShell Scripting Games Advanced Event 2:

When I start one of the Scripting Games events, I read and re-read the scenario because if you don’t understand the requirements, you can’t write an effect script, function, command, tool, etc. It’s not a bad idea to print out the event scenario and highlight the high-points.

Here’s the scenario for Advanced Event 2 -An Inventory Intervention, I’ll place the items in bold that I would normally highlight on my printout:

Dr. Scripto finally has the budget to buy a few new virtualization host servers, but he needs to make some room in the data center to accommodate them. He thinks it makes sense to get rid of his lowest-powered old servers first… but he needs to figure out which ones those are.

This is just the first wave, too – there’s more budget on the horizon so it’s possible he’ll need to run this little report a few times. Better make a reusable tool.

The phrase in bold in the previous paragraph means it needs to be a script, function, or maybe even part of a module that has error checking to prevent others you may hand this ‘tool” off to from having issues with it (not a one-liner).

All of the virtualization hosts run Windows Server, but some of them don’t have Windows PowerShell installed, and they’re all running different OS versions. The oldest OS version is Windows 2000 Server (he knows, and he’s embarrassed  but he’s just been so darn busy). The good news is that they all belong to the same domain, and that you can rely on having a Domain Admin account to work with.

The important points are support OS’s as old as Windows 2000, some without PowerShell, they’re all in the same domain and we have a domain admin account to use for access. It’s best practice to run PowerShell as a standard user and specify alternate credentials when running tools such as this need additional rights instead of running PowerShell as a domain admin so we’ll need to add a credential parameter to our tool.

The good Doctor has asked you to write a PowerShell tool that can show him each server’s name, installed version of Windows, amount of installed physical memory, and number of installed processors. For processors, he’ll be happy getting a count of cores, or sockets, or even both – whatever you can reliably provide across all these different versions of Windows. He has a few text files with computer names – he’d like to pipe the computer names, as strings, to you tool, and have your tool query those computers.

That’s a lot of information. It’s a good thing I have a few highlighters that are different colors. My first thought: “I’ll need to use WMI to retrieve this information”. The great thing about WMI is that it’s been around forever and you could even find a VBScript that uses WMI to accomplish what you want and pull the WMI class out of it and use it in PowerShell.

Now to determine what WMI classes we need to use:

Windows Version: Win32_OperatingSystem. This one was too easy. You can search WMI using the Get-WMIObject cmdlet to find classes that may contain the data you’re looking for. The class you want is going to begin with win32_ and normally your not going to want the classes that start with win32_perf so filter those out like so:

Based on these results, there’s only two classes that might contain the data we’re looking for: Win32_OperatingSystem or Win32_SystemOperatingSystem

I’ll switch to the Get-CimInstance cmdlet at this point since I’m running PowerShell version 3 and it tab expands namespaces and classes so there’s less typing. I’ll pipe to “Format-List -Property * -Force” because by default only a fraction of the properties are displayed. I added the -Force switch parameter because I want to see the properties where whoever wrote this is trying to protect me from myself. I can see the Caption is the OS name, and the CSName is the ComputerName:

Towards the bottom of the results, the Version property is the OS Version:

Test against all the different OS’s we need to support and validate that class exists and returns the correct data for each OS starting with Windows 2000. I personally ran my commands against over 30 machines so I would receive different results from a wide variety of different hardware and operating systems. Don’t run your commands in your company’s production environment though.

Ok, that one class gives us the server name, and OS version which are two of the items we need. I’ll also include the OS Name since OS’s like Windows 8 and Server 2012 or Windows Server 2003 and XP have the same “version” numbers. That way if this tool is ever run on workstations and servers, we’ll be able to differentiate between the two.

I’ve found in the scripting games it’s generally ok to include a little more information as long as it still meets the requirements and works, but with each item you add that’s not required, you open up another chance for problems. Add something that’s not required and if it doesn’t work, it’s much worse that not adding that item or feature at all.

Now we need the memory and processor information. I’ll do another WMI search to see what classes exist that might contain that information just like before:

The Win32_ComputerSystem class looks promising.

It’s also a good idea to test the classes as you go to make sure they do indeed retrieve the data you want. While the Win32_ComputerSystem appeared to return the correct information on multiple modern operating systems, on Windows 2003 it doesn’t provide the correct information as shown below. I know that this Dell PE 1950 Server has two physical processors with four cores each so the NumberOfProcessors property can’t be used to retrieve the number of physical CPU sockets since this should be 2:

I also have a question for you, first read this again: For processors, he’ll be happy getting a count of cores, or sockets, or even both - whatever you can reliably provide across all these different versions of Windows.

What does reliably mean? The online dictionary I used says “giving the same result on successive trials”. Can you provide a count of processor cores RELIABLY across all the different versions of Windows beginning with Windows 2000? My answer is no so I did not provide this information. Same question for processor sockets: Can you RELIABLY provide sockets across all the different versions? Yes, so I provided that information.

It’s also a good idea to lookup the WMI classes you’ve chosen to use on MSDN to see if there are any notes about the properties that you’re planning to use. Doing this for the Win32_ComputerSystem class and viewing the NumberOfProcessors property would have allowed you to discover that it wouldn’t display the processor sockets correctly for Windows 2000 and Windows 2003 machines:

Image Source: http://msdn.microsoft.com/en-us/library/windows/desktop/aa394102(v=vs.85).aspx

There’s a hotfix to correct this problem on Windows 2003, but not on 2000: http://support.microsoft.com/kb/932370

Here’s a trick to accurately retrieve the correct number of physical processors from any of the systems this tool is suppose to run against:

The TotalPhysicalMemory property in Win32_ComputerSystem is also incorrect. It shows what’s available to Windows after things such as memory for video has been taken out or if the OS doesn’t support the amount of physical memory in the server, it would only show what the OS is able to access.

Using this class and property would only return 4091MB of RAM on the same Dell PE 1950 as shown in the following image. I didn’t know they made physcial memory chips that would add up to that amount (4091MB)? That’s because they don’t. Notice in the second example, it’s actually very easy to determine the true amount of physical memory using the Win32_PhysicalMemory class. You’ll need to Sum or add up all of the memory chips to retrieve this information, but it’s not that difficult:

Oh, and in case he forgets how to use it – make sure this tool of yours has a full help display available.

The help is self explanatory. Run “help about_Comment_Based_Help” in PowerShell to see examples.

Are you casting the amount of memory as an integer in gigabytes? Guess what? It’s not uncommon for a Windows 2000 Server to have less than half a gigabyte of RAM. Cast 256MB as an integer in gigabytes and you’ll end up with zero. Is zero useful information? It’s not. You could either leave a couple of decimal places (don’t cast it as an integer), or cast it to megabytes as I did.

Here’s the process I use to write my PowerShell scripts, functions, commands, tools, etc:

#1 Write pieces of the script, function, command, etc that pulls the necessary data and verify accuracy. If the data isn’t accurate it doesn’t matter how fancy your PowerShell code is (period).

#2 Tune your command for efficiency. Don’t make multiple calls across the network to the same machine unless absolutely necessary. Only go to the well once and get all you need. Need all you get as well (don’t pull data from a remote machine that you don’t need – filter at the source, not the destination). Store the data in a variable and work with the variable instead of constantly creating network traffic back and forth. Same goes for calls to disk. They’re expensive. Get what you need and store it in a variable. Be sure to only retrieve what you need. A wise man once told me don’t buy the whole box of Whitman’s Samplers if all you like is chocolate covered peanuts.

#3 Do step #1 again and make sure you’re data is still accurate (Accuracy counts!)

#4 Apply best practices once you’ve completed #1, #2, and #3. Add error checking etc.

Want to see my solution for Scripting Games advanced event 2? Click this link: Advanced 2: An Inventory Intervention. You’ll have to be signed in to see the solution. If you like my script, please vote for it! Click on the star all the way to the right:

Want to know more or talk about the Scripting Games event #2? Join us for an unofficial meeting of the Mississippi PowerShell User Group tonight at 8:30pm CDT. Use this link to join the Lync meeting. See our Attendee Info page for the requirements if needed.

µ

This is a continuation from my previous blog titled “2013 PowerShell Scripting Games Advanced Event 1 – Parameters Don’t Always Work As Expected“.

This isn’t the exact script, but sections of it. You’ll notice at the bottom of the first image shown below, I retrieve the list of folder names from the files variable to keep from having to make another call to the file system. Going from one variable to another in memory is a cheap operation where as going to disk to retrieve something is more expensive from a resources standpoint. I received this comment from a reviewer: “I thought this was pretty clever: $folders = $files.directory.name | Select-Object -Unique”.

Then I iterate through the list of folders one at a time instead of iterating through each file individually. I build my paths, storing them in variables. I check to see if the folder exists and create it if not, and then move the files a folder at a time. My goal was to make as few calls to the file system as possible.

It’s not only about writing a command that’s syntax is correct, or a command that works, but also a command that’s as efficient as possible. I’ve trimmed out the error checking and verbose output from these examples to make them easier to understand.

Want to see my actual solution? Click this link: Advanced 1: An Archival Atrocity. You’ll have to be signed in to see the solution. If you like my script, please vote for it!

µ

The scenario for this event states the following which has been paraphrased:

Someone allowed log files to build up for around two years and they’re starting to causing disk space issues on the server. You need to build a tool to archive the log files because this has happened more than once. Applications write log files to a sub-folder in “C:\Application\Log”. Examples are C:\Application\Log\App1, C:\Application\Log\OtherApp, and C:\Application\Log\ThisAppAlso. The log files have random file names and a .log extension. After the log files are written to disk, they’re no longer used by the applications.

Your tool should archive log files older than 90 days and move them to “\\NASServer\Archives, maintaining the source sub-folder stucture such as “\\NASServer\Archives\App1″.

The source location,  destination location, and file age to archive should each be a parameter. Any errors that happen during the move should be clearly displayed. Output should not be displayed if the command completes successfully.

Using Get-ChildItem with the -Filter parameter:

I started out working with Get-ChildItem and was almost set on using the -Filter parameter to filter the results down to only log files, the problem is that using -Filter in that manner would also return any file extension that begins with log such as .log1 as shown in the following results:

So unfortunately if you used the -Filter parameter with Get-ChildItem in your solution, it doesn’t meets the requirements of this scenario since it will archive more than just .log files.

Using Get-ChildItem with the -Include parameter:

The other parameter that I thought would accomplish the task was Get-ChildItem with the -Include parameter. The issue I ran into with it is that it only works when using the -Recurse parameter. At first, using the -Recurse parameter didn’t seem to be an issue:

After re-reading the scenario, I decided that I only wanted to move log files in the first level sub-folders and not in the actual “C:\Application\Log” folder and not in sub-folders deeper than the first level sub-folders so as you can see in the previous image, the -Include parameter doesn’t accomplish that task.

I ended up using something similar to the following which retrieves only the files in the first level sub-folders that are older than 90 days without using either the -Filter or -Include parameter:

I’ll continue this blog post tomorrow to discuss how I dynamically retrieved this information since the scenario states the folder names could change. I’ll discuss how I determined the folder names without making an additional call to the file system. I’ll also show you how I used the list of folder names instead of iterating through each file individually to move the files to the correct sub-folder on the destination end.

Want to see my solution? Click this link: Advanced 1: An Archival Atrocity. You’ll have to be signed in to see the solution. If you like my script, please vote for it!

µ