OpsMgr

Optimizing Cisco UCS Management Pack for larger(any) environments

Andreas Sobczyk OpsMgr, PowerShell

Some months ago Cisco released a brand new management pack for the UCS hardware platform, this update was more or less a rewrite of the entire solution, including switching all alarms from monitor based to rule based. This was done in order to minimize the number of discoveries needed and therefor the objects created by the management pack, this could have quite a performance impact since every little memory module and fan was discovered as objects, and this could be many fans and memory blocks in a large UCS environment.

Since it is all rule based now we would miss the auto-resolve feature usually used by monitors, but to solve this Cisco created a rule using a simple schedule to close and update alerts as needed based on the event entries on the server running the Cisco UCS monitoring service. But with the first release (4.0.1) there was a problems, it was only checking for alerts with resolution state New (0) and i guess in most production environments many are using resolution states to route alerts to correct teams or set the progress of the alert, therefor when you changed resolution state on your alerts they would never update or close. This problem is now solved in version 4.1.1 but now a new problem appears! Cisco choose to use “Where-Object {$_.ResolutionState -ne 255” to find all the non-closed alerts, using this method in our large environment it takes 10-12 seconds to find all the closed alerts and to make it even worse this command is running for every event collected by the rule “Cisco.Ucs.Watcher.UCSFault.Event.Collection.Rule” for the selected interval meaning that the script would never complete in our case and end on timeout.

To resolve all this I found all the elements needed for the rule “Cisco.Ucs.Library.UpdateAndCloseAlert.Rule” in the original Cisco Management pack and created my own fix for this problem.

Instead of

$scomActiveAlerts = Get-SCOMAlert | Where-Object {$_.ResolutionState -ne 255}

I changed it to

$ActiveAlerts = Get-SCOMAlert -ResolutionState (0..254)

This simple change gives the same result but in 0,5 seconds instead of 10-12. And instead of running the command for each event collected I changed it to run one time for the script (just before the ForEach) and save the result in a variable $ActiveAlerts and use that in the ForEach instead.

This just shows how easy you can improve (and decrease) the performance of your PowerShell scripts.

I uploaded my fix management pack to GitHub, if you import this one remember to disable the original rule.

GitHub Download

 

 

MPTool: Automate Windows Service Monitoring

Andreas Sobczyk Automation, MPTool, OpsMgr

Monitoring a Windows Service running state is a common request at my work, and therefor a obvious thing to automate and win back some free time. Thanks to the MPTool PowerShell module this can easily done with four lines of PowerShell

First you need a management pack to place it in, either use a existing or create a new. In this example we will place all resources, class, discovery and monitor, in the same management pack, for more complex management packs you will usually split into separate management packs.

New-MPToolManagementPack -ManagementServerFQDN scomms01.cloudmechanic.net `
-ManagementPackName "CM.MyApp.ServiceMonitoring" `
-ManagementPackDisplayName "CM MyApp Service Monitoring" `
-ManagementPackDescription "Contains monitor for MyApp Windows service Monitor"

Once you have a management pack to place the monitor in, you need a class for the specific Windows service, you don’t need to add any properties for just basic service monitoring since the name of the Windows service cannot exist more than once on a Windows computer.

New-MPToolLocalApplicationClass -ManagementServerFQDN scomms01.cloudmechanic.net `
-ManagementPackName "CM.MyApp.ServiceMonitoring" `
-ClassName "CM.MyApp.WindowsService" `
-ClassDisplayName "CM MyApp Windows Service" `
-ClassDescription "CM MyApp Windows Service - Created with MPTool"

We now have a class for our Windows service and we can then go ahead and create the discovery to find where the service is existing. All Windows Services has a key under HKLM:SYSTEM\CurrentControlSet\Services\[NameOfTheService] so this can be used to discover the service. Note that this should be the service name and not the display name.

New-MPToolFilteredRegistryDiscovery -ManagementServerFQDN scomms01.cloudmechanic.net `
-ManagementPackName "CM.MyApp.ServiceMonitoring" `
-DiscoveryName "CM.MyApp.WindowsService.Discovery" `
-TargetClassName "Microsoft.Windows.Computer" `
-RegistryPath "SYSTEM\CurrentControlSet\Services\MyApp\" `
-DiscoveryClassName "CM.MyApp.WindowsService" `
-IntervalSeconds 300

Now the final step is to create a monitor for the discovered instances of the Windows service. Here you simply just need to target the newly created class, select the state of the alert if it triggered, Error or Warning, and last again defined the name of the Windows service to monitor the state of.

New-MPToolWindowsServiceMonitor -ManagementServerFQDN scomms01.cloudmechanic.net `
-ManagementPackName "CM.MyApp.ServiceMonitoring" `
-TargetClassName "CM.MyApp.WindowsService" `
-UnhealthyState "Error" `
-ServiceName "MyApp"

Of course now we want to put it all together in a script that we can use for self-service scenarios! I have created one example and placed  my GitHub for MPTool.

Happy automating!

 

MPTool, Automate and Simplify Management Pack Development

Andreas Sobczyk MPTool, OpsMgr , , ,

This is something I have been looking forward to for a long time! The first public release of the PowerShell module MPTool, it is a module for automating and simplify OpsMgr management pack development I have been working on for almost a year together with Martin Dyrlund Arnoldi

Back in the end of 2014 we started introducing automation and self-service into our company and spend the most of 2015 automating common tasks around our daily server provisioning and de-provisioning work. In the end of 2015 we looked into automation for SCOM as we couldn’t keep up with the work the way we did it at the current point. We needed a way to automate our management pack creation to be able to create them faster and more standardized, and so we came by the awesome PowerShell module OpsMgrExtended from Tao Yang. We started to play around with Tao’s module and quickly figured out that the concept of module was exactly what we needed, but also that we required the ability to create more advanced monitoring solutions, so we decided to build our own PowerShell module, and the result is this, MPTool.

MPTool is created with the goal that if you know PowerShell you should pretty easily be able to create management packs. It contains a lot of build-in logic in each function, it automatically adds management pack references if needed, for a PowerShell discovery you only need to create a PowerShell array with the discovery data and the function it self will create the SCOM specific code needed, and so on.

We have now with great success used it internally for more than 6 months and is now ready to share it with the community, I hope you will all find it as useful as us!

Over the next weeks I will post some more articles on how you can use this module in different examples

Please report if you hit any Bugs or troubles so we can improve where it is needed, also if you have any idea for future releases we will take it in and evaluate.

Detailed documentation is available at GitHub

Download Link

GitHub Repository

16 CmdLets is available now, and a few more is already in testing and will be added soon.

New-MPToolManagementPackAlias
Get-MPToolManagementPackReferenceAlias
New-MPToolManagementPack
New-MPToolOverrideManagementPack
Add-MPToolManagementPackReference
New-MPToolApplicationComponentClass
New-MPToolComputerRoleClass
New-MPToolLocalApplicationClass
New-MPToolClass
New-MPToolWindowsEventAlertRule
New-MPToolFilteredRegistryDiscovery
New-MPToolPSDiscovery
New-MPToolPSStateMonitor
New-MPToolWindowsServiceMonitor
New-MPToolDependencyMonitor
New-MPToolHostingRelationship