Using Pester and the Operation Validation Framework to Verify a System is Working

If you haven't seen the Operation Validation Framework on GitHub, it's definitely something worth taking a look at. This framework allows you to write Pester tests to perform end-to-end validation that a system is operating properly. Pester is typically used to perform test driven development or unit testing on your PowerShell code, but in this scenario it's used for operational testing.

You need to have Pester installed, but if you're running Windows 10 then you already have Pester. Download the Operation Validation Framework and place it in your $env:PSModulePath just like you would any other module.

Write your operational tests:

 1Describe "Simple Validation of a SQL Server" {
 3    $ServerName = 'SQL04'
 4    $Session = New-PSSession -ComputerName $ServerName
 6    It "The SQL Server service should be running" {
 7        (Invoke-Command -Session $Session {Get-Service -Name MSSQLSERVER}).status |
 8        Should be 'Running'
 9    }
10    It "The SQL Server agent service should be running" {
11        (Invoke-Command -Session $Session {Get-Service -Name SQLSERVERAGENT}).status  |
12        Should be 'Running'
13    }
14    It "Should be listening on port 1433" {
15        Test-NetConnection -ComputerName $ServerName -Port 1433 -InformationLevel Quiet |
16        Should be $true
17    }
18    It "Should be able to query information from the SQL Server" {
19        (Invoke-MrSqlDataReader -ServerInstance $ServerName -Database Master -Query "select name from sys.databases where name = 'master'").name |
20        Should be 'master'
21    }
22    Remove-PSSession -Session $Session

The last test uses a function that can be found in my MrSQL module on GitHub if your interested in it. I'm simply using it to remove the requirement of needing the SQLPS module installed to query a SQL Server with PowerShell.

One thing I would like to figure out is how to skip tests (set dependencies) based on the results of previous tests in my Pester tests since all of the subsequent tests will fail if the first one fails in this scenario and those failures take a lot more time to return results than successes do. Based on some responses to a tweet of mine, skipping tests may not be the best way to approach this, it may be better to run a simple test and only run a comprehensive test if the simple one completes successfully. I noticed that Invoke-OperationValidation has a TestType parameter and the valid values for it are simple and comprehensive with the default being both @('Simple', 'Comprehensive') so I'll plan to research that option.

Now to run the operational tests using the Invoke-OperationValidation function. Omitting the IncludePesterOutput parameter would eliminate the top portion of the results (the top portion of the output is what you would typically see for results when using Pester).

1Invoke-OperationValidation -testFilePath C:\tests\SQLServer\sqlserver.tests.ps1 -IncludePesterOutput


I came up with this scenario because I recently experienced a problem with a SQL Server where all of the necessary services were running and it was listening on port 1433, but it was unresponsive when trying to query actual data from any of its databases.

Operational tests can not only be used to test a single system as you've seen in this scenario, but to also perform end-to-end validation of system operation where you have multiple tiers such as web front-end servers, application servers, reporting servers, and back-end database servers along with other things that systems like that depend on such as directory services and network connectivity.

Just think, you could use these types of operational tests to check the amount of latency a system is experiencing and automatically spin up or remove VM's or containers as needed once a certain threshold is reached. You could also take corrective action based on failing tests. It's my understanding that the default output from Invoke-OperationValidation is returned in a format that can be used by other monitoring systems.

I knew learning how to write Pester tests would pay off, not only in the form of writing better code with fewer bugs in it especially when changes are made to it down the road, but now we're starting to see other uses for Pester tests such as this in the form of operational tests.