I did a quick performance study on an SBS 2003 server and found that Disk Writes were causing a bottleneck for the server. If you’d like to know how I determined that, see the blog post Discovering Disk I/O is a Problem at http://www.thirdtier.net/blog.

This procedure is appropriate for any Windows server, not just SBS 2003.

We’re lucky in SBS 2003 in that there is already a populated Group Policy that enables Auditing because it’s Object Access Auditing that we’re going to use to determine whether or not the disk writes happening on our server are from a legitimate source or not. If you don’t have an Auditing Group Policy I would suggest creating one. It’s nice to be able to set the policy for all servers in your network when you’re doing a bottleneck study like this is one. In our case there were three servers on the network and we investigated all of them. This blog post will focus on the SBS 2003 server.

The first step was to edit the Small Business Server Auditing Policy. I changed the Audit Object Access from Not Defined to Success. Then executed a gpupdate /force to push out the updated policy to my servers. Setting this group policy will allow me to choose which folders to audit on my server.

The second step is to configure my security log. The audit policy results will post events into that log file.

In Event Viewer,  properties of the Security Log set the Maximum log size to a large enough number to accommodate the logs files for the period that you want to capture. This log file is likely to be big so I would make it at least double the default, if not more. Also for the duration of this study you will want to make sure that the log file doesn’t get overwritten and instead choose Do not overwrite events. Remember to change your settings back when you are finished reviewing the log files.

The Third Step is to turn on Auditing for the folders that you want to record. In my case, we are interested in Disk Writes. So the folder I am going to target are places where applications and users store their data.

Choose a data folder and view its properties. Select the Security tab and press the Advanced button. In the Advanced Security Settings move to the Auditing tab. Press the Add button and select the events that you’d like to monitor. I chose Traverse Folder/Execute File, List Folder/Read Data, Create Files/Write Data and Create Folders/Append Data. Be sure to choose as few as required to get the information you need. The log files in auditing operations can become huge if you’re not careful. Apply your settings and repeat this for each folder that you want to audit.

Step four is to wait. You now wait for the file access to occur and be logged to the security log.

Step five is to review the security log to determine where the disk write are coming from. This is the hard part. It’s a manual sift through the audit logs to determine the ratio and frequency of writes from all of the various sources. I thought I was looking for a single source of huge number of disk writes to the server and I did indeed find one.

In my case it was the time clock software, receiving data from the physical and software time clocks through IIS, writing that data to its database and updating its reporting engine. It’s a very busy application bustling with data moving from here to there and back again. We found the cause our disk bottleneck. Now to make a recommendation on what to do about it. 

—–

So who wrote this blog and what do they do for a living anyway?
We’re Third Tier. We provide advanced Third Tier support for IT Professionals.
Get Support Blog Twitter Facebook LinkedIN

WordPress Tags: Disk,performance,server,bottleneck,Problem,Windows,Policy,Object,servers,Audit,folders,events,Event,Viewer,Maximum,size,settings,Third,users,data,Execute,File,List,Read,Create,Files,Write,Append,database

We all get those calls from clients complaining that the network is slow at certain times of the day occasionally. We usually start by looking for errors or checking out the computer of the people complaining the most. Perhaps we even look at the applications they are running as a possible source of the problem. But sometimes, it really is the “network” that is slow, at least as users experience the network. The problem is actually that the server is bogged down and having trouble keeping up with the demands placed upon it.

Today’s modern servers are usually loaded with CPU’s and RAM. So a quick look at Windows Task Manager shows that the server isn’t busy at all. CPU and Page File use are not stressed, considering this server performs many functions in the network.

This particular server is about 5 years old. The business has grown over that time and applications have been added to the server over the years. I’m suspecting that while CPU and RAM have kept up with the company, that the disks have not. A performance study on disk performance is in order.

The subject server is an SBS 2003 server running on hardware that is about 5 years old. There are 4 hard drives configured in a RAID5. There is a RAID controller card installed in the server. This procedure is not SBS specific, it will work on any server where you suspect Disk performance problems. In addition to the SBS software suite this server also houses User Shared Folders and a Time Clock application. The subject company is in the manufacturing industry and they have about 80 employees and 45 computers.

The first thing that we need to do is setup performance monitoring to record the disk performance covering the period when users most often report that the server is slow.

To record potential Disk I/O issues I’m going to setup a Counter Log using these items:

• Physical Disk: Avg. Disk Queue Length  (Less than 2)
• Physical Disk: % Disk Read Time
• Physical Disk: % Disk Write Time
• Physical Disk: % Disk Time (under 35)

I know from reading Microsoft’s information on Disk I/O testing in TechNet that a well running server should perform at certain levels for each of these. Those values are noted after the name of each counter above.

To Create and Schedule a Disk Counter Log

1. Open Performance Monitor
2. Expand Performance Logs and Alerts
3. Right click on Counter Logs and Choose New
4. Type a Name for the log file you are about to create and a location in which to save it
5. Add the suggested counters shown in the list above
6. Schedule the task to run for several hours covering the period when users report slowness

Let the task run for the period you specified then return to view the results.

Here’s snippet of what my results looked like.

I find the graph difficult to read, so I prefer to view a text report instead.

Now to do a bit of calculation. In order to get an accurate Ave Disk Queue length I need to divide with by the number of disks, which in the case of this server is 4.

Comparing our results with the specifications, we can see that our Ave Disk Queue Length is within tolerance, the Disk Read Time is somewhat over but our Disk Write Time is through the roof! We have a disk bottleneck.

What you decide to do with this information is entirely up to you. What I did was get curious to find out what was writing to my disks all the time. Is this legitimate business use of the server? To see the answer to that question and learn one way to determine where those disk writes of coming from see the blog post Determining Where All Of Those Disk Writes of Coming From. It will be posted soon to http://www.thirdtier.net/blog

—–

So who wrote this blog and what do they do for a living anyway?
We’re Third Tier. We provide advanced Third Tier support for IT Professionals.
Get Support Blog Twitter Facebook LinkedIN

WordPress Tags: Disk,computer,server,servers,Windows,Task,performance,RAID,User,Time,Clock,computers,Counter,Queue,Read,Write,Microsoft,TechNet,Monitor,Logs,Alerts,bottleneck,Writes,blog