Hello AskPerf folks!  This is just a quick blog post to inform you of a great posting from one of our Disk Experts on our Core team.  It is very informative and talks about measuring Physical Disk IO Latency with Perfmon.  Check it out!

Measuring Disk Latency with Windows Performance Monitor (Perfmon)

http://blogs.technet.com/b/askcore/archive/2012/02/07/measuring-disk-latency-with-windows-performance-monitor-perfmon.aspx

-Blake Morrison

Hello AskPerf Blog Readers! This morning’s post is going to be more of a reference point for all those WMI junkies out there.  Have you ever wanted to test WMI using the command line (outside of VBSCRIPT)?  If so, then WMIC is the tool for you.  In its simplest form, WMIC is a utility that allows you to interact with WMI from a WMI-aware command-line shell.  All WMI objects and their properties, including their methods, are accessible through the shell, which makes WMIC an advanced systems management console.

This blog will assume that you already have a grasp of WMI and are looking for more WMI testing options via the Command Line.  With that said, the following is a list of WMIC queries that you can run on your Workstations/Servers.

To execute these queries, run “WMIC” at a command prompt, followed by one of the following alias/es:

In troubleshooting WMI issues here on the Performance team, I often run the following commands to test WMI locally and remotely:

WMIC CPU GET NAME

WMIC /NODE:SERVERNAME CPU GET NAME

Additional Resources

• WMIC – Take Command-line Control over WMI
• Security with WMIC
• WMIC verbs

-Blake Morrison

Good morning AskPerf world!  Jeff Hagler from the Remote Resources team here to chat about printing.  How many of you have ever dealt with an issue where you just knew that something was wrong with your print spooler but could not quite put a finger on it?  Maybe print jobs were slow, certain users could print to some printers but not others, or maybe nobody could print at all?

Back in the day, we included a tool with the Windows 2003 Resource Kit called Cleanspl that was able to make sure that the spooler service was configured correctly and clear of any third party dependencies that may be referencing bad/missing files.  Cleanspl is a great tool, but by being hidden in the resource kit, it was rarely used.  Up until now, this tool did not exist/work in recent Operating Systems like Windows Server 2008..

We are happy to announce the same functionality in Cleanspl (and a bit more) has been built into one of our Fixits that will run on Windows XP/2003 and Windows Vista/2008 and can be downloaded from here.   If you are running Windows 7 or Server 2008 R2, have no fear as we have a similar tool for you in the works right now, and we’ll update this blog with a link as soon as it becomes available.

This Fixit package consists of two modes: Light and Full.  Light will make changes to the spooler that should not require your end user to make any changes, while the Full mode will restore the print spooler to defaults and will require a complete reinstall of your print drivers and printers.  Think of the Light version as something you can run on any system (print server, terminal server, normal client system) as a first troubleshooting step, and the Full version as something to reserve for systems that get their printers and print drivers pushed down from Print Servers (via Group Policy and Point and Print).  Let’s cover the key part of the previous statement again – do not run the Full version on a Print Server as it will delete all the printers.  Basically, it puts your Print Environment back to the way it was when the OS was installed.

When you first run the Fixit on your system, you will be prompted with the normal installation/licensing agreement screens.  You will then receive the dialog box below.  To run the Light version, just click Next.  To run the Full version, you will need to check the box to “Reset the print spooler to defaults”.

Note the Light version does not require a reboot but the Full version does.  Again, do not run the Full version on a Print Server as it will delete all the printers.

Here are the details on what actually happens during each of these cleanups:

Light Mode

• Set the print spooler service back to its defaults (automatically start, run as Local System, etc.)
• Remove any dependencies that the spooler may have on 3^rd party services
• Delete any print jobs from %windir%\system32\spool\printers (shd and spl files)
• Turn on logging of print errors and warning to the event log (Printers -> File -> Server Properties -> Advanced Tab -> “Log spooler error events” and “Log spooler warning Events” will be checked)
• Make sure the standard print monitors are present (USB, Local Port, and TCPIP) under [HKLM\CurrentControlSet\Control\Print\Monitors] (and WSD on 2008/Vista only)
• Ensure that the winprint print processor is present for the architecture of the computer – for example: [HKLM\CurrentControlSet\Control\Print\Environments\Windows x64\Print Processors\winprint]
• Start the print spooler service
• If any of the core print spooler binaries are missing (spoolsv.exe, spoolss.dll, localspl.dll, win32spl.dll), the package attempts to copy that file from dllcache - if they aren’t present in dllcachce, we log that in the desktop log file
• If ntprint.inf is missing, we log that to the desktop log file
• If there is a bogus path for the spool directory DefaultSpoolDirectory under [HKLM\SYSTEM\CurrentControlSet\Control\Print\Printers], we log that information to the desktop log file

Full Mode

• Does all of the Light steps first
• Delete *.spl and *.shd files from a non-default spool folder
• Delete *.spl and *.shd files from the default spool folder:  %windir%\System32\spool\printers
• Delete printer shares
• Delete per-machine printers from [HKLM\SYSTEM\CurrentControlSet\Control\Print\Printers] (except on Vista do not delete "Microsoft XPS Document Writer" and “Fax”)
• Delete per-machine connections from [HKLM\SYSTEM\CurrentControlSet\Control\Print\Connections] and the “Connections” key itself
• Delete per-user connections under <USER HIVE>\Printers
• Delete Monitors under [HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors] (except Local Port)
• Delete each driver under [HKLM\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows <arch>\Drivers\Version-3] (except on Vista don’t delete “Microsoft XPS Document Writer” and “Microsoft Shared Fax Driver”)
• Vista: delete all files under “%windir%\System32\spool\drivers\<arch>\3” except for driver files associated with “Microsoft XPS Document Writer” and “Microsoft Shared Fax Driver”.  Ignore subdirectories.
• XP: delete the “%windir%\System32\spool\drivers” folder
• Set back the defaults : [HKLM\SYSTEM\CurrentControlSet\Control\Print\Providers\LanMan Print Services\Servers] (AddPrinterDrivers=0)
• Delete [HKLM\SYSTEM\CurrentControlSet\Control\Print\Providers\Internet Print Provider\Ports]
• Add back these default values:
  • HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\USB Monitor
  • HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\Standard TCP/IP Port
  • (XP) HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\BJ Language Monitor
  • (XP) HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\PJL Language Monitor
  • (Vista) HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\WSD Port
  • (If Fax driver was installed) HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors\Microsoft Shared Fax Monitor

As you can see the light mode of this tool is a great first pass at troubleshooting a myriad of Print Spooler issues, with minimal risk/downtime for your users.  Give it a go and let us know how it works for you.

-Jeff Hagler

Good morning AskPerf!  Yesterday we announced the availability of the Windows 8 Consumer Preview (beta) for download/testing.  This is huge for our Client OS business and we encourage you to download and play around with it.  Please see the following links for information and the download location for the Windows 8 beta.

Windows 8 Consumer Preview

Windows 8 Download Link

Welcome to Windows 8 – The Consumer Preview

Running the Consumer Preview: system recommendations

Windows 8 Consumer Preview Product Guide for Business

We will be Blogging more about the exciting new features from a “Performance” standpoint over the coming months, so stay tuned!

-Blake Morrison

This morning Bill Laing, Corporate Vice President of Server and Cloud Announced the beta release of Windows Server “8”.  I think it is really important for anyone who is interested in hearing more about what is coming in this beta with Hyper-V and our Virtualization development efforts; please go read what Bill has to say.

In the coming days and months, there will be more Windows Server “8” blog posts coming from Bill Laing and his leadership team on the Windows Server Blog.

Also, stay tuned in the coming days and weeks for more deeper dives in the supporting technologies here.  I have been waiting to start talking, so hope you have some time carved out to learn more about Windows Server “8” technologies.

Download Windows Server “8” Beta here

-Blake Morrison

We often get asked from time to time how to force printers that a user has installed on their workstation using the IPP website to use RPC in lieu of the Internet printing protocol. There is a document available on TechNet that explains how to accomplish this:

http://technet.microsoft.com/en-us/library/cc757981(WS.10).aspx

(It’s as simple as changing the IE Zone Security to Medium-Low)

Often though, we run into customers who have already customized the settings in their IE Zone security and would still like to force the use of RPC for IPP printers without resetting those IE Zone settings to Medium-Low.

So what settings is the IPP printer really checking to accomplish this?

The process Windows uses to decide how to connect to a printer looks like this:

1. The OS checks if the following Registry Key is set:

[HKEY_CURRENT_USER\Printers\Settings\PreferredConnection]

If this key exists and its value is 0, the printer connection is using HTTP.

2. If the Registry Key is not set we are looking for two Settings:

a. Java Permissions

This Setting cannot be made inside IE. You have to configure the Settings with GPO.

Java Permissions need to be at least set to Medium Safety to use RPC.

b. ActiveX

Additionally the OS checks the Value from “Initialize and script ActiveX controls not marked as safe for scripting”.

If the Value is disabled we will use RPC otherwise we will use HTTP for the Print Connection.

A last pre-requisite is that the Protected Mode in IE needs to be turned off to use RPC

-Bernd Hogen

Good morning AskPerf!  There are several issues that have been associated with a high number of inactive Terminal Server ports.  Delayed logon times to RDP sessions, failure of printers to redirect, and slow server performance due to registry bloat from all the ports. These inactive TS ports accumulate because the Remote Desktop Services Device Redirector service creates a new port every time an RDP session is established, but the ports are not always recycled.  Every RDP session can possibly create a new port, and every ended session means a new inactive port. Performance degradation is known to occur when 250 or more TS ports exist in the registry. Increasingly large numbers of redirected devices will exacerbate performance delays.

To eliminate any issues these TS ports may cause, we have a new Windows Server 2008 R2 hotfix.  The Hotfix can be downloaded here - KB 2655998 Long logon time when you establish an RD session to a Windows Server 2008 R2-based RD Session Host server if Printer Redirection is enabled.  This hotfix will prevent inactive TS ports from accumulating in the future, but you also have to clean up any currently inactive TS ports using the Fixit tool included in KB2655998.  The FixIT tool itself can also be downloaded directly from http://go.microsoft.com/?linkid=9799054.  The FixIt tool will remove the entries already accumulated under the TS ports registry key below:

[HKLM\SYSTEM\CurrentControlSet\Control\DeviceClasses\{28d78fad-5a12-11d1-ae5b-0000f803a8c2}\##?#Root#RDPBUS#0000#{28d78fad-5a12-11d1-ae5b-0000f803a8c2}]

Because the Hotfix and the FixIt perform two different functions, you must install the Hotfix and run the FixIt one time on your 2008 R2 Remote Desktop Server. 

For down-level operating systems such as Server 2003 and Server 2008 Terminal Servers, we highly recommend a scheduled task be created to execute the Inactive TS Port FixIt on a bi-weekly basis, since the hotfix only supports Windows Server 2008 R2 at this time.

The FixIt is an MSI package so you can run it silently with no user interaction required.  Follow these simple steps:

1. Locate the MSI file in Windows Explorer and note the exact path to it

2. Click Start and type in the following:

msiexec /package “path” /quiet

For example, if the MSI file MicrosoftFixit50833.msi is located in the “temp” folder of your C drive, then the path would be C:\temp\MicrosoftFixit50833.msi and the command would look like this:

msiexec /package “C:\temp\MicrosoftFixit50833.msi” /quiet

3. To add logging and check the results, use this command:

msiexec /package "c:\temp\MicrosoftFixit50833.msi" /quiet /log c:\temp\MSI50833.log

Additional Resources

• MSIEXEC commands http://support.microsoft.com/kb/314881
• Available Updates for Terminal Services (Remote Desktop Services) in Windows Server 2008 http://support.microsoft.com/kb/2312539
• Available Updates for Terminal Services (Remote Desktop Services) in Windows Server 2008 R2 http://support.microsoft.com/kb/2601888

-Jess Cunningham

Good morning AskPerf! Sanket Jagtap here from Platforms Team here to discuss an interesting issue I recently worked.  My customer had a Windows 2008 R2 Server, and could not connect locally to WMI.  When they ran “wmimgmt.msc” and tried to connect, the following error appeared:

Failed to connect to <local computer> because "Win32: The system cannot find the path specified."

When this failed, the following events were recorded to the Application log:

Log Name: Application
Source: Microsoft-Windows-WMI
Date: <Date/Time>
Event ID: 28
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: <Computer Name>
Description: Failed to Initialize WMI Core or Provider SubSystem or Event SubSystem with error number 0x80070003. This could be due to a badly installed version of WMI, WMI repository upgrade failure, insufficient disk space or insufficient memory.

Log Name: Application
Source: Microsoft-Windows-WMI
Date: <Date/Time>
Event ID: 43
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: <Computer Name>
Description: Windows Management Instrumentation ADAP failed to connect to namespace \\.\root\cimv2 with the following error 0x80070003

We checked the WMI Security tab, and the Root namespace was missing, as well as under the Advanced tab:

During the course of troubleshooting, we tried running “winmgmt /verifyrepository”, however it failed with a similar error:

C:\>winmgmt /verifyrepository
WMI repository verification failed
Error code: 0x80070003
Facility: Win32
Description: The system cannot find the path specified.

At this point, we decided to run the WMIDiag Tool (WMIDiag 2.1) to help troubleshoot this issue. The WMIDiag log reported the same error multiple times for most of the Root namespace:

1843 12:43:16 (0) ** - Root, 0x80070003 - The system cannot find the path specified.
Error Result: 0x80070003 ( -2147024893 )
ID Defined as: ERROR_PATH_NOT_FOUND
Message Text: The system cannot find the path specified.

We then popped open the Registry and browsed to the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WBEM\CIMOM]

We compared a working to a non-working registry and made the following discover:

Working Machine

Non-Working Machine

The registry “Type” was set to REG_SZ instead of REG_EXPAND_SZ. We took a backup of the CIMON folder, then deleted the old “Repository Directory” value and created a new one using the REG_EXPAND_SZ type. After a restart, WMI now successfully connected to the Repository/Default Namespace.

Unfortunately, we were unable to determine what changed this value, however it could have been a user change, some type of script, or caused by an application.

Additional Resources

• WMIDiag 2.1 is here!
• WMI Troubleshooting: The Repository on Vista / Server 2008

- Sanket Jagtap

A new printing hotfix rollup was released that updates the core print spooler components localspl.dll, splwow64.exe, spoolsv.exe, win32spl.dll, and Winprint.dll.  This rollup is designed to reduce known issues such as printing performance, print spooler crashes, connectivity to print queues, and print driver installation. 

To download the update and get more information on the fixes included in this rollup, please see KB article 2647753:  Description of an update rollup for the printing core components in Windows 7 and in Windows Server 2008 R2

Many of the printing related support cases at Microsoft are resolved by one of the included hotfixes.   We highly recommend installing this hotfix rollup when troubleshooting any printing related issues. 

Hello AskPerf and Happy Friday!  This is just a quick post to let you know that Flavio, one of our Disk Experts on the Core team, has posted another informative blog post; this time on Performance Monitor Disk Counters.  Check it out and have a great weekend!

Windows Performance Monitor Disk Counters Explained

http://blogs.technet.com/b/askcore/archive/2012/03/16/windows-performance-monitor-disk-counters-explained.aspx

-Blake Morrison

Good morning AskPerf! Digvijay here to chat about print server migration. All server admins have probably had an opportunity to migrate print servers. This process can be fast and easy when migrating between the same versions of operating systems; and can be done with a bit of effort moving between Operating Systems with different architecture (e.g., moving from 32-bit to 64-bit versions of Server 2008). However, things really get challenging when we have to migrate from an older Operating System to the latest Operating System on a different architecture. (e.g., from Windows 2003 Server x86 to Windows Server 2008 R2 x64). Fortunately, there are new tools such as the Print Management Console that ships with Server 2008 R2 or, if you prefer the command line interface, then you can use PRINTBRM.EXE. However, there can be some hurdles getting all the printers migrated successfully if we do not understand the Printer restoration process and fulfill necessary pre-requisites.

Things to keep in mind during printer migrations:

1. A print queue cannot function without a driver of the same server architecture (x86 or x64) on which it exists. 

2. Unless the print processor is migrated successfully, the print queue will not show up.

3. If you are migrating from a 32-bit Operating System to a 64-bit Operating System, it is important that you have 64-bit drivers pre-installed for all of your printers, along with the 32-bit drivers. (Remember that for a 64-bit client to print to print server hosted on 32-bit Operating System, you need the 64-bit version of the driver.)

4. When a cross-architecture migration includes the migration of printer language monitors, an error will occur during the process of restoring the printers to the destination server using the PRINTBRM command-line tool. The reason for the error is that language monitor driver architecture must always be the same as the source server architecture. Therefore, when migrating from x86-based architecture to x64-based platforms, language monitor migration cannot be successful. An error posted to the event log will state that the source architecture is not the same as that of the destination server. More info about this behavior can be found in the Print Services Migration Guide.

5. When using PRINTBRM, always run commands from an Administrative command prompt

If performing a 32-bit to 64-bit migration and all the existing client machines are 32-bit, then you will have to install 64-bit drivers for all the 32-bit drivers before starting the migration. It is common to have third-party 32-bit printer drivers that do not have 64-bit equivalents. For those instances, we typically suggest using universal drivers if available but contacting the printer vendor for their input is the safest bet.

During the Printer restore in cross-platform scenarios, if the backup file does not contain driver binaries for target server platform, PRINTBRM will attempt to install the drivers from the target server’s driver store if a matching driver is available. If a matching driver for the target platform is not available (either in the backup or on the server) then the print queue will not be created and there will be errors. See below:

We will see relevant events in the event viewer also:

So to make it easy for everyone to understand, here is the straightforward way to successfully restore all the printers:

1. Install 64-bit drivers on the Windows 2003 x86 Print server (source) or install all the required 64-bit drivers on the 2008/R2 Server (target). Keep in mind that the driver name string must be an exact match. If the names do not match exactly, the migration will treat it as if the driver is not present.

2. Take a backup of the printers on the Windows 2003 source server using the Print Management Console or printbrm.exe run from the Windows 2008 R2 Server.

3. Import the backup to the 2008 R2 server.

NOTE We need to preinstall any driver that has different components when installed on x86 than when installed on x64. As long as the driver is installed then the printer installation will not be blocked unless there is a non-existent Monitor value in the Printer registry key.  If the printer (not the driver) requires this setting, then we will need to install the driver that uses this monitor on the x64 machine.  If there is no x64 monitor provided by the vendor or if the vendor provides a new x64 monitor that uses a different name, then you will need to add a printer using the x64 version of the driver on the new machine since the vendor providing the driver did not provide compatibility to the new OS.

Keeping it simple

To avoid most of these driver and component pitfalls, it’s often easiest to switch all the print processors and print monitors on the source server over to some basic defaults ones (eg: cleaning up the PRINT registry before starting with the migration). This will speed up the process and maximize the number of queues successfully migrated.

Microsoft has provided a Spooler FixIT for Windows 2003/2008 to automate the spooler cleanup process. See our previous AskPerf blog entry for information about the functionality and usage of this tool.

We also recommend removing old and unused drivers from the server before taking a backup.

However, if you do not want to go through this lengthy process of fulfilling these prerequisites, are on a time crunch, want to quickly restore the print queue information on the target server and deal with the driver worries later, then read on.

In order to quickly get the queues migrated to the new server, we can create them using the “Generic / Text only” printer driver without restoring any of original the drivers on the target. Remember that we need to have 64-bit drivers present either in the package or on the target server for the restore to finish and the print queue to be available for printing.

Here are the steps to successfully migrate all of your print queues using the in-box “Generic/Text Only” print driver:

1. On the source Windows 2003 Server, change all print processors to winprint using the following setprinter.exe command:

SetPrinter.exe \\Servername 2 pPrintProcessor="Winprint"

This can also be accomplished using WMI:

wmic printer set PrintProcessor = WinPrint

2. Install the latest update for Printbrm on the target 2008R2 server:

KB 2636591  An update to improve the restore operation performance of Printbrm.exe in Windows 7 or in Windows Server 2008 R2

3. Install a local printer on the target 2008 R2 server using the Generic / Text Only driver and share it to enable the firewall exceptions for the Print Services.

4. From the Windows 2008R2 server, take a backup of the 2003 print server using the nobin switch

(we are using the nobin switch since we don’t want to migrate any of the print drivers):

Printbrm.exe –b –s \\servername –f nobin.printerexport –nobin

5. Next, we have to modify the contents of the backup files. Expand the printerexport backup to a directory and the replace the BrmDrivers.xml, BRMLMons.xml and PProcs.xml files. The command to extract the files to a directory is:

printbrm.exe  -r -d c:\temp\expand  -f nobin.printerexport

The Expanded directory will look similar to this:

• BrmDrivers – contains a list of every driver installed on the computer and the driver files for each driver
• BrmForms – contains a list of all the installed forms
• BrmLMons – will usually contain either Windows NT x86 or Windows x64 as the architecture and a list of port monitors and the port monitor files installed
• BrmPorts – contains a list of all the installed printer ports
• BrmPrinters – contains a list of all printers that have been installed
• BrmSpoolerAttrib – contains information about the spooler directory path and also indicates whether or not the source server was a cluster server

6. Now we need to edit the existing xml files and delete the information about unused print processors, language monitors and drivers in them and then create a new file without these components.

NOTE You may ask if we already set all the printers to use WinPrint as default print processor using the earlier methods, then why do I need to do this? The answer is that when we take a backup using PRINTBRM, the tool copies all the files from their respective directories inside C:\Windows\System32\spool\. Hence even if we have replaced the information on the queues, these files will be backed up and at the restore operation, they will also be restored. So even though we took a backup using the nobin switch, the xml files contains information about all the drivers. When you attempt to restore, the server will try to look for these drivers and restore them. Also, the -nobin switch only omits the driver files, the Language monitor and the Print processors are still backed-up.

For creating new BrmDrivers.xml, BRMLMons.xml and PProcs.xml files, we can use the following templates:

BrmDrivers.xml

===========

<PRINTERDRIVERS>
</PRINTERDRIVERS>

===========

BRMLMons.xml

==========

<LMONS Arch="Windows x64"/>

==========

PProcs.xml

========

<PPROCS>

</PPROCS>

=========

7. Once we have saved the new xml files in place, we will re-pack the backup using this command to get a new backup file that we will use to restore on the target server:

printbrm.exe  -b -d c:\temp\new  -f newbackupfile.printerexport

8. Now that we have a clean printer backup, we will use the BRMConfig file to create all the printers using the “Generic / Text only” driver. The BRMConfig.xml file needs to have a mapping of all the drivers present on the source server that we want to restore with the Generic / Text Only driver.

A sample BRMConfig file may look like this:

=============

<BrmConfig>

<PLUGINS>

</PLUGINS>

<DriverMap>

<DRV old="HP Universal Printing PCL 6" new="Generic / Text Only"/>

<DRV old="HP Laserjet 5000 PCL 6" new="Generic / Text Only"/>

<DRV old="HP Universal Printing PS 6" new="Generic / Text Only"/>

</DriverMap>

</BrmConfig>

==================

For restoring all the printers from the backup, we need to mention all the existing drivers here.

NOTE The name of the driver has to be an exact match.

9. Once you have your brmconfig.xml file finished, run this command to start the restore:

printbrm –r –s \\2008R2Servername  –f newbackupfile.printerexport -c c:\temp\brmconfig.xml

NOTE We need to provide the absolute path of the BrmConfig.xml file here.

10. If all the mentioned instructions have been followed, you will see all the print queues getting created on the server using the Generic / Text Only driver.

This is where we started from:

And this is what we have accomplished:

11. Verify that the printers are working fine by sending some test pages to the printers.

12. After the queues are created on the target server, you can install the 64bit drivers for the printers as per your convenience and then switch the queues again.

Simplifying the BRMConfig.xml file creation

Considering most typical production scenarios, Print Servers you have hundreds of print queues and many print drivers, creating the BrmConfig.XML file for mapping each driver to Generic / Text Only driver can be real time consuming process. To ease the pain of creating this file yourself, there’s a BRMC tool that will come in handy. Download the tool from https://brmc.codeplex.com/ and run it from the directory where the BRMDrivers.XML is located. (So run it from the expanded location). Once the tool is run, and it creates a BRMConfig.xml file, pack the backup and then run the restore using the BrmConfig.XML file (to to step 8 and then go to step 10).

The BRMC tool creates the brmconfig.xml file that has the driver mappings. It also creates clean BrmDrivers.xml, BRMLMons.xml and PProcs.xml in the same directory. Be aware, the tool overwrites the original files, so save the original .printerexport file as a backup of BrmDrivers.xml, BRMLMons.xml and PProcs.xml.

Troubleshooting

If you encounter any errors during a Print Migration restore, check the event logs for possible causes. Common ones may be Print Processor is unknown, Unknown Port, or Print driver is unknown which usually means that the spooler was not cleaned properly on the source server when the initial backup was generated. Look for more in-depth troubleshooting of these kinds errors in a future blog.

I hope this information will come in handy the next time you are working through a printer migration. Until next time…

Addition Resources

• Print Services Migration Guide
• AskPerf Blog: Installing cross-architectural print drivers from the server locally
• AskPerf Blog: Windows 7 / Windows Server 2008 R2: PrintBRM
• Two Minute Drill: PRINTBRM and the Configuration File

-Digvijay

Good morning AskPerf!  Kapil Patry here from the Microsoft Platforms Support Team.  I am blogging today about an issue I recently worked.  This particular issue dealt with the creation of a task using Task Scheduler on a Windows 2008 R2 Server (can occur on Windows 7 as well).  When I attempted to create a scheduled task, the following error appeared:

Task Scheduler

An error has occurred for the task <task name>. Error message: The following error was reported: A specified logon session does not exist. It may have already been terminated..

This error came up whether I was logged on as a Local Administrator, Domain Administrator, or any other user that had rights to log on locally.

After extensive research, I found that the above error will only occur if the following Security Policy is enabled and you select the “Run whether user is logged on or not” Security option on the General tab, when creating a new task:

Network access: Do not allow storage of passwords and credentials for network authentication

To resolve this issue, simply Set this policy to Disabled:

The new version of Task Scheduler (Windows Vista onwards) uses Windows Credential Manager to store the credentials of the account that is specified to perform a task.  If the Network access: Do not allow storage of passwords and credentials for network authentication policy is enabled and applied, Credential Manager cannot store the credentials locally, thus this error message appears.

NOTE you will not receive this error if the “Run only when user is logged on” Security option on the General tab is selected (we do not store passwords in this scenario).

Addition Resources

• Windows Vista Task Scheduler
• Task Security Context
• Task Scheduler 2.0

-Kapil Patry

In previous versions of Windows, clients with remote connections would often initiate network traffic to retrieve remote information from a print server, resulting in a resource-consuming call to the server.  This behavior was more significant when Client Side Rendering (CSR) was used that caused increased communication with the remote server. Many applications use certain APIs to query the print spooler for print information at startup, at document creation time, and when displaying available fonts. This often resulted in visible delays or hangs in the applications from the user’s perspective. In Windows 7 and Windows Server 2008 R2, the performance of many spooler API calls has been improved through more aggressive caching that will reduce the remote server calls and improve end user performance.

In general, the intent of the enhancements has been to decouple applications that use client print APIs (winspool.drv) from the print server to avoid application UI delays. Some of the APIs that are used by major client applications such as the print dialog and Microsoft Office have had their caching behavior improved. OpenPrinter, EnumJobs, and some forms of GetPrinterDriver are the main APIs that will see caching improvements.

Overview of CSR caching behavior

The CSR provider maintains a cache of print connection data for remote connections. This cache is persisted in the registry and loaded at provider startup.  The cache follows a “pull” model and in general operates asynchronously.  Calls requesting remote connection data are serviced from the local cache immediately where possible, and an asynchronous work item is queued that will synchronize with the remote server if necessary. Only in the case where the remote data is not cached locally will the call go over the network synchronously. The asynchronous synchronization with the remote server is done only periodically; most cache entries have an expiry time of 120 seconds. Cache coherency is enforced by checking Change IDs associated with both the local cache and the remote connection. If a request is made for cached data last accessed over 120 seconds ago, the remote Change ID will be compared to the local one, incurring a single RPC call of overhead. If they differ the cache will be updated from the remote server, incurring one or more RPC calls.

Caching behavior of common APIs

The table below shows the caching characteristics of common print APIs. The cache refresh interval for each caching mode is described in the next section.

Caching behavior of key print APIs

Registry keys for cache refresh intervals

Two registry key values control the cache refresh behavior for print APIs. Caching mode “Normal” is controlled by the following registry key value and defaults to 120 seconds:

HKLM\Software\Policies\Microsoft\Windows NT\Printers\SyncCacheAgeUser

Caching mode “Volatile” is controlled by the following registry key value and defaults to 0 seconds:

HKLM\Software\Policies\Microsoft\Windows NT\Printers\SyncCacheAgeVolatile

-Bernd Hogen

Good morning Folks!  Today is a quick post to inform you of a new Blog from the Windows Server team that talks about the new RDS (Remote Desktop Services) features in Windows Server 2012.  Check it out!

Windows Server 2012 Remote Desktop Services (RDS)

Have a great weekend!

Good day AskPerf!  Last Thursday we announced the availability of the Windows 8 Release Preview (a little late on the blog posting I know).  If you’ve been living under a rock, then this is new news to you.  If not, then I’m sure you may have already downloaded it and stated playing around with it.  With that said, here are some links to get you started:

Windows 8 Release Preview (main page)

Delivering the Windows 8 Release Preview (blog post by Steven Sinofsky)

Download page

-Blake Morrison

Good morning AskPerf! One of our large call generators that we still see is with PTE depletions on Windows Server 2003 Terminal Servers (x86). Scaling your W2K3 Terminal Server can be challenging as the load increases with additional users and the limits of the x86 Memory Architecture. Often Terminal Servers run out of kernel resources due to load instead of any type of memory leak. One of those resources is depleting System PTEs (Page Table Entries). So, what are System PTEs?

System PTEs are a kernel memory structure used to map the following:

• I/O Space
• Kernel Stacks
• Memory Descriptor Lists (MDLs)

Other operating system internals that use PTEs:

• Paged Pool Memory
• Non-Paged Pool Memory
• PFN Database
• File Cache
• Drivers
• Sessions

NOTE The normal range of PTE’s are as follows (can vary from these values):

• ~60,000 free at boot – 4GB or less memory system with /3GB and no /userva setting in the boot.ini
• ~170,000 free at boot - 2GB (or 4GB – 16GB) system with multiple NIC’s and HBA’s with default settings
• ~260,000 free at boot – 4GB system on simple hardware (VM’s for example with show high total PTE’s because the hardware is very simple)

Free System PTE’s can be viewed by adding the counter “Memory\Free System Page Table Entries” in Perfmon. Free System PTE’s should be kept above 10,000.

When a system starts running low on free PTEs, < 5000 , unusual things can start occurring on the server with little to no warnings. First, applications or drivers may get memory denied type errors. Secondly, applications or the Server itself might hang if threads cannot be created. Another symptom could be errors in the event logs, similar to the following:

Event Type: Information
Event Source: dmio
Event Category: None
Event ID: 29
Date: MM/DD/YYYY
Time: HH:MM:SS AM/PM
User: N/A
Computer: Computer_Name
Description:
dmio: Harddisk9 read error at block 445136247: status 0xC000009A

Event Type: Information
Event Source: dmio
Event Category: None
Event ID: 30
Date: MM/DD/YYYY
Time: HH:MM:SS AM/PM
User: N/A
Computer: Computer_Name
Description:
dmio: Harddisk2 write error at block 411779656: status 0xC000009A

0xC000009A is a Windows status code that translates to STATUS_INSUFFICIENT_RESOURCES.

A program is composed of 1 or more threads. Each of those threads has a stack (the kernel mode part of a user mode program) and each of the threads use some PTE’s. Normal threads use 4 while UI threads (threads related to displaying info on the screen) use 16. Those can end up using a majority of the PTE’s on a system. Let’s consider the following example:

Customer was previously able to have 50 logged on users to their terminal servers. Now, they are only able to have ~35 users before new users start receiving errors when connecting to the TS.

Let’s look at the relevant data to determine the cause of the problem. Perfmon data captured for several hours shows how load and memory resources vary.

The screenshot below highlights the number of sessions on the system. We start with 36 sessions and get as high as 38 sessions until the end, where we fall down to 5 user sessions.

Now let’s look at free PTEs. Free System PTEs start at 22,000 and get as low as 4000 as more users log in. As users start logging off, we shoot up to ~145,000 free System PTEs. The fact that Free System PTEs increase shows that this is a load issue and not a Memory leak.

Finally, we see threads that have an inverse pattern of Free System PTEs. When we get above 11,000 threads, PTEs decrease. We saw a maximum of 948 processes running at the time of this capture.

Note Most Windows 2003 32 bit systems will only get to 11,000 – 14,000 threads before PTE’s are depleted

11962 threads / 948 processes = 12 threads per process is a normal number (this can vary with different types of applications)

11962 threads / 38 sessions = 314 threads per user session

How do we troubleshoot this type of problem? Glad you asked!

First, we need to determine what the number of free PTEs at boot. I gave ranges above, but in general we should look for at least 170,000 free PTEs when the machine starts up. If your server is not around this number, then the following could be at play here:

The Server has greater than 16 GB of memory. I have never seen a 32 bit terminal server that could use more than 16 GB of memory before running out of some type of kernel memory resource. Having more memory takes additional kernel memory for the PFN database which is used to map physical memory addresses. Either physically remove the memory or use /maxmem=16200 in the boot.ini file to limit memory to 16 GB.

The Server is using the /3GB switch in the boot.ini. Using /3GB (4Gig tuning) on a terminal server will not work well. Do not try to run programs that need 3GB of memory on a terminal server. Move them to another server. While you can increase the amount of PTE’s by using /USERVA=3030 (or a value down to 2800) in the boot.ini, other kernel memory pools will be limited and the system will not scale.

Complex hardware including multiple NIC’s and HBA’s (reserves addresses at boot). Some NICs and HBAs can reserve a lot of virtual addresses. This will reduce the amount of kernel memory including PTEs. Disable or remove any unneeded hardware from the system.

If the number of free PTEs at boot look normal, then decline with load, the following should be reviewed:

How many processes is each user running? Lite users may only be running 4 while heavy users may be running 20 or more. A few heavy users can make the difference between getting 50+ users on a system and only getting 30 users.

Are there any programs that can be removed? It is not uncommon for new programs to get added but old programs are still installed. Try removing those that are not needed.

Is there an individual program that is creating a lot of threads? Threads per process can be viewed by adding the thread column in task manager. I had one customer that had a custom program that was creating over 350 threads per instance - almost all of them UI threads (350 threads * 16 PTE’s = 5600 PTE’s per instance of this program).

Note On average ½ of threads will be UI threads. So on a system with 10000 threads they will use the following:

5,000 threads * 4 PTE’s + 5,000 threads * 16 PTE’s = 100,000 PTE’s used just for thread stacks

After looking at both the number of the PTEs being created at boot and removing any unneeded processes, then you will have optimized the number of Free System PTEs. If free PTEs continue to decrease into the danger area, then the Server load will need to be reduced by limiting the number of logged on users. If you still experience issues, then an x64 OS could be in your future.

-Leonard Severt

Hello everyone! This is Harshad Joshi from the Performance Team at Microsoft. In today’s post, I am going discuss an interesting feature of Remote Desktop Services configured for users via Group Policy Preferences. Recently, I had a request from a customer to change the mouse pointer for a set of users, but only within a Remote Desktop Session. What they wanted was a particular Mouse Pointer scheme shown whenever a particular set of users remote to any server in the infrastructure.  The scheme name in Mouse Properties window under control panel is “Windows Black (extra-large) (system scheme)”. The primary mouse cursor in this scheme looks like this:

Now, it’s fairly easily to change the mouse scheme for a user account when the user is logging onto the computer locally; but to assign a particular Mouse pointer scheme only in a Remote Desktop Session (RDP), we’ll need to make use of the Group Policy Preferences (GPP). Given that GPP is an Active Directory Domain policy, we can do this for Domain Accounts.  Additionally, GPP is only available if the domain has at least one Domain Controller running Windows Server 2008 or Windows Server 2008 R2 as its operating system. GPP was not available on Windows Server 2003 or earlier operating systems.

These are the steps of how to modify a domain users’ mouse scheme using Group Policy Preferences.

1. Create a group of users for which you want to change the mouse pointer in a Remote Desktop Services session. It’s best if you create this group under a separate Organizational Unit (OU) so that you can link a GPO to this OU.

2. Open Group Policy Management Console (gpmc.msc) and locate the concerned OU.

3. Right Click and select option, “Create a GPO I this domain, and link it here…”.

4. Give it a name which can help you in identifying the reason behind this GPO and click Ok.

5. Once the GPO is shown under the OU, right click on it and click Edit…

6. Group Policy Management Editor will open.

7. In the Group Policy Management editor, navigate to User configuration > Preferences> Windows Settings> Registry. Refer to below screenshot:

Before creating or updating any registry item, you should review the existing registry values that are already present and update those. The screenshot below shows the registry keys along with their values which are already present for the “Windows Aero (system scheme)” :

Since this is the default cursor scheme you will see in windows 7, all you need to do is update these registry keys instead of creating new items.

Below screenshot shows one of the registry keys as an example:

For registry keys which are not already present, you need to choose create option under Action. An example screenshot of create item option is show below:

Below table shows what registry values need to be created for changing the default Mouse cursor to the Windows Black using GPP for domain accounts:

This is how you can see the registry items after created defined using GPP:

Note down the color difference in the icons for registry items. Yellow ones are the Updated registry keys whereas Green ones are the Created registry keys. Link this GPO to the OU on which you need to apply.

Additional References

• Group Policy Preferences Overview white paper
• Group Policy Preferences: Getting Started
• How Terminal Services Works
• Remote Desktop Services Features

- Harshad Joshi

Hello AskPerf! Michael Smith here with a quick post on what can cause Win32_QuickFixEngineering to return "No Instances Available". A captured Procmon log while running WMIC QFE or Enumerating Instances with WBEMTEST might reveal something similar to the below picture:

Here it appears that WMI calls into the Servicing stack to get information about installed updates, which makes sense. So what happened? We can see TrustedInstaller could not find the mum file for KB2618444 and it started writing to the CBS.log. Since this seems to indicate a problem with Servicing we should run the System Update Readiness Tool, aka CheckSUR, and check the following logs:

%SYSTEMROOT%\Logs\CBS\CheckSUR.log

%SYSTEMROOT%\Logs\CBS\CheckSUR.persist.log

NOTE Sometimes simply running CheckSUR can resolve some of servicing issues it detects. In this particular case, CheckSUR could not find the mum files either.

Let's take a look at the CheckSUR.persist.log below:

Checking System Update Readiness.

Binary Version 6.1.7601.21645

Package Version 14.0

2012-03-07 11:10

Checking Windows Servicing Packages

Checking Package Manifests and Catalogs

Checking Package Watchlist

Checking Component Watchlist

Checking Packages

(f) CBS MUM Missing 0x00000002 servicing\Packages\Package_for_KB2618444~31bf3856ad364e35~amd64~~6.1.1.2.mum

(f) CBS MUM Missing 0x00000002 servicing\Packages\Package_for_KB2618451_RTM~31bf3856ad364e35~amd64~~6.1.1.0.mum

Checking Component Store

Summary:

Seconds executed: 169

Found 2 errors

CBS MUM Missing Total count: 2

Unavailable repair files:

servicing\packages\Package_for_KB2618444~31bf3856ad364e35~amd64~~6.1.1.2.mum

servicing\packages\Package_for_KB2618451_RTM~31bf3856ad364e35~amd64~~6.1.1.0.mum

servicing\packages\Package_for_KB2618444~31bf3856ad364e35~amd64~~6.1.1.2.cat

servicing\packages\Package_for_KB2618451_RTM~31bf3856ad364e35~amd64~~6.1.1.0.cat

Here we can see that CheckSUR could not find these two mum files. So what do we do now? Fortunately, my colleague Joseph Conway, has a great blog on how to troubleshoot this issue. All we need to do is manually download the updates for KB2618444 and KB2618451 and copy the .msu files over to “%windir%\Windows\CheckSUR\Packages” (you will need to create this directory) and then run CheckSUR again which will take care of the rest!

After repairing the Servicing issue, WMI was able to return instances from Win32_QuickFixEngineering:

C:\>wmic path win32_quickfixengineering get HotFixID,InstalledOn /format:list

HotFixID=KB976932
InstalledOn=2/11/2011

HotFixID=KB980915
InstalledOn=10/21/2011

HotFixID=KB982018
InstalledOn=7/21/2011

… (Truncated for brevity)

Hope you enjoyed this two minute drill.

-Michael Smith

Hello AskPerf! I’m Craig Marcho, a Senior Support Escalation Engineer in the Microsoft Platforms Core Team. There has been an increase in cases lately with Windows 7 Clients and Windows Server 2008 R2 Print Servers where Clients will show print queues as being offline, while at the same time, other Clients can print just fine and the Print Server shows the queue as online. While there are a few things that can cause this behavior, all of our normal troubleshooting steps were not providing relief for this particular issue.

We found that this issue occurs because a restricted client thread that runs in the spooler detects an offline print server. When a client thread detects that a print server is offline, Windows registers a polling loop to check the status of the print server. After the polling loop is registered, Windows queries the print server periodically to check whether it is back online.

In rare cases, a client thread that has a restricted token detects that a print server is offline. In this situation, Windows registers the polling loop in the context of this thread. However, the thread does not have sufficient rights to query the print server. Therefore, the polling request fails. Restarting the spooler or the client machine will obviously kill this thread running under the restricted context and will query the server with the correct security.

The hotfix became available on July 11^th and you may download it here:

A network printer is displayed as offline incorrectly on a computer that is running Windows 7 or Windows Server 2008 R2

So if you have been experiencing this issue, or know someone who has, please spread the word that a fix is now available.

-Craig Marcho

Hello AskPerf!  Not sure if you’ve heard about this new feature with Windows 8 Server beta or not, but you can now manage your machines via Windows PowerShell in a web browser.  This means, you can use Windows PowerShell from a large variety of devices such as mobile phones, tables, and computers that do not have Windows PowerShell installed.

Check out their blog post here:

Introducing Windows PowerShell Web Access in Windows Server 8 Beta

-Blake Morrison