GSM Displaying Incorrect DNS Response Times

In May this year I discovered DNS Resolution Time problem which was taking around 4 seconds to resolve for GSM tests. 

I raised a case with Microsoft who instructed me to enable a VS web test to help diagnose  however I managed to get to the bottom of the problem before starting the case.Thought I would blog as this may very well catch me out again.

The Graph are four separate GSM tests from London endpoints for DNS Resolution Counter for the same URL ‘www.blahblah.com’  Over a 24 hour period. Two are displaying high DNS and the other two are displaying less than 1 second so no consistency there which also implies an error with the GSM config.

GSM DNS Error

This graph displays the DNS Resolution Counter over a three day period selecting only the GSM tests in the previous screenshot that were below 1 second. You can see that around 10:00 PM ish UK time the problem had gone away for those two test.

GSM DNS Error2

Clearly something wrong here. I ran independent tests with numerous external tools including the obvious human option browsing the site and timing how long the page initially displays. My conclusion was if it is taking 4-6 seconds to wait for DNS as GSM is indicating then it would be incredibly easy to detect and recreate. Needless to say I could not re-create the same times interdependently.

The problem.

Some companies will use devices for DNS name resolution for client requests. So this problem may or may not manifest itself as a problem for all clients. Alongside onsite network staff have identified the cause is due to GSM making checks over IPV6. The Devices are not configured for IPV6.0  The cause to slow DNS server response times are due to the request from the client over IPV6.0 which eventually results in a  timeout.

( Not 100% sure all GSM checks are IPV6 followed by IPV4 but does explain the sporadic results.)

We added values to non IPV 6.0 network device tell the client to respond with NOERROR which will result in the client not checking each DNS server up to root before timing out. Primary cause of the delay

This will lead to no timeouts for IPv6 queries because they will quickly receive the ‘noerror’ response and should therefore generally improve the DNS experience for clients and consequently statistics will also improve immediately. I thought I was really onto something when I said the client have issues with site speeds 🙂

Anyways here is a similar thread which I think is related

http://blogs.technet.com/b/momteam/archive/2013/05/15/local-resolution-time-monitor-in-the-opsmgr-dns-management-pack-may-show-unexpected-results.aspx

Advertisements
Posted in GSM, SCOM 2012 | Leave a comment

Testing Management Packs

Given the testing of MP’s is crucial when you are implementing a new MP or you want to test your alerting following an MP review. I recommend you get your hands on MP Event Analyzer as this tool as is far superior to Event Create which is frustratingly limited in it’s functionality. Kevin Holman nicely articulates it’s usage.

http://blogs.technet.com/b/kevinholman/archive/2011/08/02/how-to-test-fire-any-windows-event-on-any-server-from-any-application.aspx

One thing though is in order to use it .Net framework 4.0 is required on the host you run it on… In order to test any MP you will of course either have to break the application (Not really feasible in most circumstances) or simulate the errors so of course the MP Eventlog Analyzer will need to execute on the host to be tested. I have in the past handed finished packs to testing team with the exported test fire script so that the tests simply load the pack fire the script and hey presto alerts and functionality checked. This is more of a heads up really when liaising with test teams.

There is also a tool named NTEvent Log Explorer which can be used that will run on host machine but may have some limited functionality http://blogs.technet.com/b/momteam/archive/2008/04/02/eventlog-explorer.aspx

Posted in Management Packs | Leave a comment

SCOM2012 Dashboard Limitations

Nice Job Cameron. I would recommend loading these into your SCOM 2012 deployment to make use of the all the native features available for displaying key info.

2007 Dashboard for 2012

For dashboard creation in 2012 I use these more than I would like to as I prefer all the new stuff… I just feel the widgets do not expose enough of the options we have grown used to. I love the new widgets and I make use of what they offer but they have some annoying limitations.

I think it remains a bit of an uphill struggle to sell the dashboards due to their inflexibility. I guess I should stop moaning and get on and develop my own. The web widget is a nice addition.

http://gallery.technet.microsoft.com/Web-View-Widget-Management-0d14bd0a

Note: Remember using the old R2 Dashboards are no good for the SCOM2012 SharePoint Widget as this only allows for viewing SCOM2012 Dashboards.

Camerons Link to the 2007 Dashboards http://www.systemcentercentral.com/creating-opsmgr-2007-r2-style-dashboard-views-in-opsmgr-2012-scom-sysctr/ Cameron articulates well but to extend a little further on the limitations of the Widgets where the old 2007 and focusing only on like for like, given some of the widgets cover stuff that are not present in SCOM 2007 like TOP N Performance..

Performance Widget

Does not allow you to collapse. You cannot Change Colors or Personalize or set the Graph for Max Min Y Axis etc.

Still cant resolve this and think may be silverlight but the flashing of performance counters from highlighted to transparent and back again. It is like the selection is cached in the widget and jumps from full view to the selected.

Alert Widget

You cannot set granular alert criteria for alert views. Often having to create groups just so can limit the alerts for certain classes of objects.

Top N Performance

Has a limit of 20 which I would like to see upped so at least a full column can be filled on your standard screen dashboard resolutions.

SLA Dashboards are awful on response times for 15 and upwards. Just drags on SQL DW. Would like to see some improvements here also.

I had gripes with memory usage but this was thankfully resolved in UR3 I think.

Posted in Dashboards, SCOM 2012, Uncategorized | Leave a comment

Note to self. Upload OneNote backlog

Have been so busy of late just not had time to upload odds and ends.

Note to self. Upload my OneNote backlog

Posted in MISC | Leave a comment

SCOM 2012 SP1 UR2

So RU2 released and finally the Path view on TOP N Performance Widget and Details Widget has been fixed for OpsMgr. This has been a major bug bear of mine especially as seem to be creating lots of Dashboards these days.. logged case on TAM well over a year ago and goes to show eventually they do get around to it 🙂

Update I think was the quickest and easiest I have had dealings with. Very smooth.

http://support.microsoft.com/kb/2802159

Path thing is fixed but still think it is a little messy.. At least we have server name and depending on class used will vary the length. Real Estate is important with views and whilst the Path is welcome it just takes too much room up.

TOP N

Posted in Dashboards, SCOM 2012 | Leave a comment